4 commonly used methods of crawling network data in PHP

4 commonly used methods of crawling network data in PHP_PHP Tutorial

WBOY

Release： 2016-07-13 09:52:02

Original

843 people have browsed it

Four commonly used methods of capturing network data in PHP

The name of this section is fsockopen, curl and file_get_contents. Specifically, these three methods are discussed for input and output of network data. Some summary. We have talked a lot about fsockopen before, let’s move on to others. Here is a brief list of some common methods of crawling network data.

1. Use file_get_contents to get the content in get mode:

$url = 'http://localhost/test2.php';

$html = file_get_contents($url);

echo $html;

$url = 'http://localhost/test2.php';

$fp = fopen($url, 'r');

stream_get_meta_data($fp);

$result = '';

while(!feof($fp))

{

$result .= fgets($fp, 1024);

}

echo "url body: $result";

fclose($fp);

$url = 'http://localhost/test2.php';

$html = file_get_contents($url);

$data = array(

'foo'=>'bar',

'baz'=>'boom',

'site'=>'www.jb51.net',

'name'=>'nowa magic');

$data = http_build_query($data);

//$postdata = http_build_query($data);

$options = array(

'http' => array(

'method' => 'POST',

'header' => 'Content-type:application/x-www-form-urlencoded',

'content' => $data

//'timeout' => 60 * 60 // 超时时间（单位:s）

)

);

$url = "http://localhost/test2.php";

$context = stream_context_create($options);

$result = file_get_contents($url, false, $context);

echo $result;

echo $html;

　2. Use fopen to open the url and obtain the content using get method

$url = 'http://localhost/test2.php?site=jb51.net';

$ch = curl_init();

$timeout = 5;

curl_setopt ($ch, CURLOPT_URL, $url);

curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);

$file_contents = curl_exec($ch);

curl_close($ch);

echo $file_contents;

1 2 3 4 5 6 7 8 9 10

$url = 'http://localhost/test2.php'; $fp = fopen($url, 'r'); stream_get_meta_data($fp); $result = ''; while(!feof($fp)) { $result .= fgets($fp, 1024); } echo "url body: $result"; fclose($fp);

　3. Use the file_get_contents function to get the url in post mode 　?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

$data = array( 'foo'=>'bar', 'baz'=>'boom', 'site'=>'www.jb51.net', 'name'=>'nowa magic'); $data = http_build_query($data); //$postdata = http_build_query($data); $options = array( 'http' => array( 'method' => 'POST', 'header' => 'Content-type:application/x-www-form-urlencoded', 'content' => $data //'timeout' => 60 * 60 // Timeout time (unit: s) ) ); $url = "http://localhost/test2.php"; $context = stream_context_create($options); $result = file_get_contents($url, false, $context); echo $result;

4. Use the curl library. Before using the curl library, you may need to check whether the curl extension has been turned on in php.ini 　?

1 2 3 4 5 6 7 8 9

$url = 'http://localhost/test2.php?site=jb51.net'; $ch = curl_init(); $timeout = 5; curl_setopt ($ch, CURLOPT_URL, $url); curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout); $file_contents = curl_exec($ch); curl_close($ch); echo $file_contents;