PHP obtains remote web page content (fopen, curl method)-PHP Tutorial-php.cn

PHP obtains remote web page content (fopen, curl method)

WBOY

Release： 2016-07-25 08:54:23

Original

1233 people have browsed it

$handle = fopen ("http://bbs.it-home.org/", "rb");
$contents = "";
while ( !feof($handle)) {
$contents .= fread($handle, 8192);
}
fclose($handle);
echo $contents; //Output the obtained content.
?>
// For PHP 5 and above, you can use the following code
$handle = fopen("http://bbs.it-home. org", "rb");
$contents = stream_get_contents($handle);
fclose($handle);
echo $contents;
?>

Copy the code

the above code The error failed to open stream: HTTP request failed! is prone to occur.

Solution: Some people say that in php.ini, there are two options: allow_url_fopen =on (indicating that remote files can be opened through url), user_agent="PHP" (indicating which script to access the network through, by default there is ";" in front of it, that is, remove it Yes. ) Restart the server. But some people still have this warning message. There is still one step left to achieve a perfect solution. You have to set the user_agent in php.ini. The default user_agent in php is PHP. We changed it to Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) to simulate the browser

user_agent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

I encountered this problem at work and solved it perfectly, so I share it with everyone.

PHP obtains remote web page content (fopen, curl method)

2. Implemented through curl

$url = "http://bbs.it-home.org";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT,10);
$dxycontent = curl_exec($ch);
echo $dxycontent;
?>

Copy code

linux system in It can be downloaded using the following code exec("wget {$url}");

Attached, the difference between PHP’s external resource grabbing functions fopen / file_get_contents / curl. fopen/file_get_contents will re-do the DNS query for each request and does not cache the DNS information. But CURL will automatically cache DNS information. Requests for web pages or images under the same domain name only require one DNS query. This greatly reduces the number of DNS queries. So the performance of CURL is much better than fopen/file_get_contents.