Home > php教程 > php手册 > body text

php正则匹配获取指定url网页页面超级链接地址

WBOY
Release: 2016-06-13 11:32:37
Original
1210 people have browsed it

 在数据采集与页面分析中,常需要抓取给定url页面的内容,或者第二、第三层次深度页面内容。

这里是一个测试例子的实现,仅供参考。

代码如下:


/*
匹配给定页面链接
return:array match[link,content,all]
*/
function match_links($host, $document) {
 $pattern = '/(.*?)/i'; 
 preg_match_all($pattern, $document, $m); 
 return $m;

 preg_match_all("']+))[^>]*>?(.*?)'isx",$document,$links);
 while(list($key,$val) = each($links[2])) {
  if(!empty($val))
   if(preg_match("/http/",$val)){
    $match['link'][] = $val;
   }
   else {
    $match['link'][] = $host . $val;
   }
 }
 while(list($key,$val) = each($links[3])) {
  if(!empty($val))
   if(preg_match("/http/",$val)){
    $match['link'][] = $val;
   }
   else {
    $match['link'][] = $host . $val;
   }
 }
 while(list($key,$val) = each($links[4])) {
  if(!empty($val))
   $match['content'][] = $val;
 }
 while(list($key,$val) = each($links[0])) {
  if(!empty($val))
   $match['all'][] = $val;
 }
 return $match['link'];
}

/*
从给定url中获取页面文本内容
*/
function get_content_from_url($url) {
 $str = @file_get_contents($url);
 if(mb_check_encoding($str, "GBK"))
  $str = iconv("GBK","UTF-8", $str);
 $str = strip_tags($str); // 过滤html标签
/* 
 $str = preg_replace( "@<script>@is", "", $str );<br /> $str = preg_replace( "@<iframe(.*?)@is", "", $str );<br /> $str = preg_replace( "@<style(.*?)@is", "", $str );<br /> $str = preg_replace( "@<(.*?)>@is", "", $str );<br /> */<br /> //过滤非汉字字符<br /> preg_match_all('/[x{4e00}-x{9fff}]+/u', $str, $matches);<br /> $str = join(',', $matches[0]);<br /> if(!$str)<br /> return NULL;<br /> <br /> return $str;<br /> }</script>

function get_content($url,$depth) {
 if(!$url || $depth   return false;

 while($depth > 1){
  $str = @file_get_contents($url);
  if(!$str)
   return false;

  $parseurl = parse_url($url); 
  if($parseurl['host'])
   $host = $parseurl[scheme] . "://" . $parseurl['host'];
  
  $arrlink = match_links($host,$str);
  $arr_url = array_unique($arrlink);

  $depth--;
  foreach($arr_url as $url){
   $content .= get_content($url, $depth); //递归调用
  }
 }

 $content .= get_content_from_url($url);
  
 return $content;
}

source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Recommendations
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template