python crawler php displays different content to visitors and crawler

I heard that this method will violate some operating principles of search engines, and may be punished by various search engines, or even delete the website. So I have just removed this treatment until I am sure that it is not cheating. Friends who are enterprising can continue to use it. , but do so at your own risk.
The homepage and archive pages of this blog display articles in the form of a list, and the content of the article is only loaded when the visitor clicks to expand the article. Because the content part of the article contains a large amount of text and pictures, it requires a lot of loading time and traffic. Displaying web pages to visitors as quickly as possible can retain a large number of visitors. For mobile users, loading time and traffic are more important.
Generally speaking, the homepage of a website is the page most visited by search engines, and should be used as much as possible It is possible to display meaningful content to them, but if the article is displayed in the form of a list, visitors and search engines can only obtain the article title information. The article content or summary (especially the first sentence of the article) is important for SEO It is extremely important, so we have to try to send this content to the crawler.
Well, we can use the User Agent to determine whether the visitor is a crawler. If so, the article will be displayed in a general form, otherwise the article list will be displayed in a list form. You can use The following PHP method determines whether it is a crawler:

Copy code The code is as follows:

function is_crawler() {
$userAgent = strtolower($_SERVER['HTTP_USER_AGENT']);
$spiders = array(
'Googlebot', // Google crawler
'Baiduspider', // Baidu crawler
'Yahoo! Slurp', // Yahoo crawler
'YodaoBot', // Youdao crawler
'msnbot' // Bing crawler
// More Multiple crawler keywords
);
foreach ($spiders as $spider) {
$spider = strtolower($spider);
if (strpos($userAgent, $spider) !== false) {
return true;
}
}
return false;
}

This is the method I use, each crawler is sorted from high to low by the accessed comments. Then different content is displayed to crawlers and natural people through the following method

The above introduces how python crawler php displays different content to visitors and crawlers, including python crawler content. I hope it will be helpful to friends who are interested in PHP tutorials.