php实现递归抓取网页类实例_PHP

WBOY
Release: 2016-05-30 14:59:19
Original
991 people have browsed it

本文实例讲述了php实现递归抓取网页类的方法。分享给大家供大家参考。具体如下:

<&#63;php
class crawler{
 private $_depth=5;
 private $_urls=array();
 function extract_links($url)
 {
  if(!$this->_started){
   $this->_started=1;
   $curr_depth=0;
  }else{
   $curr_depth++;
  }
  if($curr_depth<$this->_depth)
  {
   $data=file_get_contents($url);
   if(preg_match_all('/((&#63;:http|https)://(&#63;:www.)*(&#63;:[a-zA-Z0-9_-]{1,15}.+[a-zA-Z0-9_]{1,}){1,}(&#63;:[a-zA-Z0-9_/.-&#63;&:%,!;]*))/',$data,$urls12))
   {
    foreach($urls12[0] as $k=>$v){
     $check=get_headers($v,1);
     if(strstr($v,$url) && $check[0]=='HTTP/1.1 200 OK' && !array_search($v,$this->_urls) && $curr_depth<$this->_depth){
      $this->_urls[]=$v;
      $this->extract_links($v);
     }
    }
   }
  }
  return $this->_urls;
 }
}
&#63;>
Copy after login

希望本文所述对大家的php程序设计有所帮助。

Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template