CURL is a tool that uses URL syntax to transfer files and data. It supports many protocols, such as HTTP, FTP, TELNET, etc. The best part is that PHP also supports the CURL library. Using PHP's CURL library can easily and effectively scrape web pages. You only need to run a script and analyze the web pages you crawled, and then you can get the data you want programmatically. Whether you want to retrieve partial data from a link, take an XML file and import it into a database, or even simply retrieve the content of a web page, CURL is a powerful PHP library.
①: Initialization
curl_init()
②: Setting attributes
curl_setopt() //有一长串CURL 参数可供设置,它们能指定URL请求的各个细节。
③: Execute and get the result
curl_exec()
④: Release the handle
curl_close()
①: GET method to achieve
//初始化 $curl = curl_init(); //设置抓取的url curl_setopt($curl, CURLOPT_URL, 'http://www.baidu.com'); //设置头文件的信息作为数据流输出 curl_setopt($curl, CURLOPT_HEADER, 1); //设置获取的信息以文件流的形式返回,而不是直接输出。 curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); //执行命令 $data = curl_exec($curl); //关闭URL请求 curl_close($curl); //显示获得的数据 print_r($data);
②: POST method to implement
//初始化 $curl = curl_init(); //设置抓取的url curl_setopt($curl, CURLOPT_URL, 'http://www.baidu.com'); //设置头文件的信息作为数据流输出 curl_setopt($curl, CURLOPT_HEADER, 1); //设置获取的信息以文件流的形式返回,而不是直接输出。 curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); //设置post方式提交 curl_setopt($curl, CURLOPT_POST, 1); //设置post数据 $post_data = array( "username" => "coder", "password" => "12345" ); curl_setopt($curl, CURLOPT_POSTFIELDS, $post_data); /执行命令 $data = curl_exec($curl); //关闭URL请求 curl_close($curl); //显示获得的数据 print_r($data);
③: If The obtained data is in json format and is interpreted into an array using the json_decode function.
$output_array = json_decode($data,true); //If the second parameter is true, it will be converted to array form. If not filled in, it will be in the form of an object
If you use json_decode($data) to parse, you will get object type data.
An encapsulated function
//参数1:访问的URL,参数2:post数据(不填则为GET),参数3:提交的$cookies,参数4:是否返回$cookies function curl_request($url,$post='',$cookie='', $returnCookie=0){ $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)'); curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($curl, CURLOPT_AUTOREFERER, 1); curl_setopt($curl, CURLOPT_REFERER, "http://XXX"); if($post) { curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($post)); } if($cookie) { curl_setopt($curl, CURLOPT_COOKIE, $cookie); } curl_setopt($curl, CURLOPT_HEADER, $returnCookie); curl_setopt($curl, CURLOPT_TIMEOUT, 10); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $data = curl_exec($curl); if (curl_errno($curl)) { return curl_error($curl); } curl_close($curl); if($returnCookie){ list($header, $body) = explode("\r\n\r\n", $data, 2); preg_match_all("/Set\-Cookie:([^;]*);/", $header, $matches); $info['cookie'] = substr($matches[1][0], 1); $info['content'] = $body; return $info; }else{ return $data; } }
Although these two functions are not difficult, they are still worth learning. Because these two functions must be used when making interfaces or calling interfaces.
Recommended related articles:
How to crawl Tmall and Taobao product data with php
The above is the detailed content of PHP uses CURL to implement GET and POST request step examples. For more information, please follow other related articles on the PHP Chinese website!