Home > Backend Development > PHP Tutorial > PHP determines whether the visitor is a search engine crawler

PHP determines whether the visitor is a search engine crawler

WBOY
Release: 2016-07-25 08:44:24
Original
1100 people have browsed it

We can use HTTP_USER_AGENT to determine whether it is a spider. Search engine spiders have their own unique signs, some of which are listed below.

  1. function is_crawler() {
  2. $userAgent = strtolower($_SERVER['HTTP_USER_AGENT']);
  3. $spiders = array(
  4. 'Googlebot', // Google crawler
  5. 'Baiduspider', // Baidu crawler
  6. 'Yahoo! Slurp', // Yahoo crawler
  7. 'YodaoBot', // Youdao crawler
  8. 'msnbot' // Bing crawler
  9. // More crawler keywords
  10. );
  11. foreach ($spiders as $spider) {
  12. $spider = strtolower($spider);
  13. if (strpos($userAgent, $spider) !== false) {
  14. return true;
  15. }
  16. }
  17. return false;
  18. }
Copy code

The php code below comes with more spider logos
  1. function isCrawler() {
  2. echo $agent= strtolower($_SERVER['HTTP_USER_AGENT']);
  3. if (!empty($agent)) {
  4. $ spiderSite= array(
  5. "TencentTraveler",
  6. "Baiduspider+",
  7. "BaiduGame",
  8. "Googlebot",
  9. "msnbot",
  10. "Sosospider+",
  11. "Sogou web spider",
  12. "ia_archiver",
  13. "Yahoo! Slurp",
  14. "YoudaoBot",
  15. "Yahoo Slurp",
  16. "MSNBot",
  17. "Java (Often spam bot)",
  18. "BaiDuSpider",
  19. "Voila",
  20. "Yandex bot",
  21. "BSpider",
  22. "twiceler",
  23. "Sogou Spider",
  24. "Speedy Spider",
  25. "Google AdSense",
  26. "Heritrix",
  27. "Python-urllib",
  28. "Alexa (IA Archiver)",
  29. "Ask",
  30. "Exabot",
  31. "Custo",
  32. "OutfoxBot/YodaoBot",
  33. "yacy",
  34. "SurveyBot",
  35. "legs",
  36. "lwp-trivial",
  37. "Nutch",
  38. "StackRambler",
  39. " The web archive (IA Archiver)",
  40. "Perl tool",
  41. "MJ12bot",
  42. "Netcraft",
  43. "MSIECrawler",
  44. "WGet tools",
  45. "larbin",
  46. "Fish search",
  47. );
  48. foreach($spiderSite as $val) {
  49. $str = strtolower($val);
  50. if (strpos($agent, $str) !== false) {
  51. return true;
  52. }
  53. }
  54. } else {
  55. return false;
  56. }
  57. }
  58. if (isCrawler()){
  59. echo "Hello Spider! ";
  60. }
  61. else{
  62. echo "You are not a spider spirit! ";
  63. }
Copy code

php


source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template