Website page analysis, how does the notebook do web page analysis? , Website page analysis homework

Hello everyone, today Xiaobian paid attention to a more interesting topic, that is, the problem of website page analysis, so the editor compiled a related answer to introduce website page analysis, let Let's see it together. How does the notebook do web page analysis? How does the notebook do web page analysis? First of all, if you want to use a program to grab web pages and automatically save them locally, you need to be able to use socket programming, or learn to use the libcurl library. These are much more useful than learning html language. When you don’t do web scraping, this knowledge is still very useful . Moreover, different web pages have different content and rules may also be different. For example, the example URL you gave, I right-clicked to view the source code, there is no so-called , tags, that is, its form is not realized through these tags. So, let you look at these labels, it is the opposite. Web page analysis, in the final analysis, is still string processing and analysis. Therefore, if you really want to learn, it is better to learn regular expressions and string processing related functions, as well as function libraries, such as tidy library. Regular expressions are used to match a type of string, which is convenient for finding patterns and processing. You will know how powerful and useful it is after learning a little bit. Moreover, regular expressions have nothing to do with language, and can be used in any language, so it is not a loss to learn this. There are no regular expression-related functions in the standard C library. Generally speaking, two regular expression libraries are used in C, one is the POSIX C regular library, and the other is the perl regular expression library PCRE. In comparison, PCRE is more powerful, and the POSIX C regular library is sufficient. Secondly, when analyzing web pages, one must also have a certain understanding of algorithms: (1) Analysis algorithms based on network topology: based on the links between web pages, through known web pages or data, to have direct or indirect links to them Algorithms to make evaluations on relational objects (which can be webpages or websites, etc.). It is further divided into three types: web page granularity, website granularity, and web page block granularity. (2) Webpage analysis algorithm based on webpage content: The analysis algorithm based on webpage content refers to the webpage evaluation based on the characteristics of webpage content (text, data and other resources). The content of the web page has developed from the original hypertext-based to the later dynamic page (or hidden web) data, and the data volume of the latter is about 400~500 times that of the directly visible page data (PIW, publiclyIndexable Web). . (Non-original) At this point, the above is the editor's introduction to the problem of website page analysis. I hope that the one-point answer about website page analysis will be useful to everyone.