Table of Contents
HTTP
FTP
Local
Home php教程 php手册 php获取远程网页源码的程序代码

php获取远程网页源码的程序代码

Jun 02, 2016 am 09:13 AM
explode fopen foreach request substr

有时我们需要做一些采集需要下载远程网页源码到本来了,在这里我们整理了一些php获取远程网页源码代码,希望对各位会有所帮助。

php的curl函数

基本例子

<?php
// 初始化一个 cURL 对象
$curl = curl_init();
// 设置你需要抓取的URL
curl_setopt($curl, CURLOPT_URL, &#39;http://www.phprm.com&#39;);
// 设置header
curl_setopt($curl, CURLOPT_HEADER, 1);
// 设置cURL 参数,要求结果保存到字符串中还是输出到屏幕上。
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
// 运行cURL,请求网页
$data = curl_exec($curl);
// 关闭URL请求
curl_close($curl);
// 显示获得的数据
var_dump($data);
?>
Copy after login

php fopen函数

<?php
print ("<h1 id="HTTP">HTTP</h1>n");
// open a file using http protocol
if (!($myFile = fopen("http://www.phprm.com/", "r"))) {
    print ("file could not be opened");
    exit;
}
while (!feof($myFile)) {
    // read a line from the file
    $myLine = fgetss($myFile, 255);
    print ("$myLine <BR>n");
}
// close the file
fclose($myFile);
print ("<h1 id="FTP">FTP</h1>n");
print ("<HR>n");
// open a file using ftp protocol
if (!($myFile = fopen("ftp://ftp.php.net/welcome.msg", "r"))) {
    print ("file could not be opened");
    exit;
}
while (!feof($myFile)) {
    // read a line from the file
    $myLine = fgetss($myFile, 255);
    print ("$myLine <BR>n");
}
// close the file
fclose($myFile);
print ("<h1 id="Local">Local</h1>n");
print ("<HR>n");
// open a local file
if (!($myFile = fopen("data.txt", "r"))) {
    print ("file could not be opened");
    exit;
}
while (!feof($myFile)) {
    // read a line from the file
    $myLine = fgetss($myFile, 255);
    print ("$myLine <BR>n");
}
// close the file
fclose($myFile);
?>
Copy after login

file_get_contents函数

<?php
file_get_contents(&#39;http://www.phprm.com/&#39;);
?>
Copy after login

抓取远程网页源码类

<?php
class HTTPRequest {
    var $_fp; // HTTP socket
    var $_url; // full URL
    var $_host; // HTTP host
    var $_protocol; // protocol (HTTP/HTTPS)
    var $_uri; // request URI
    var $_port; // port
    // scan url
    function _scan_url() {
        $req = $this->_url;
        $pos = strpos($req, &#39;://&#39;);
        $this->_protocol = strtolower(substr($req, 0, $pos));
        $req = substr($req, $pos + 3);
        $pos = strpos($req, &#39;/&#39;);
        if ($pos === false) $pos = strlen($req);
        $host = substr($req, 0, $pos);
        if (strpos($host, &#39;:&#39;) !== false) {
            list($this->_host, $this->_port) = explode(&#39;:&#39;, $host);
        } else {
            $this->_host = $host;
            $this->_port = ($this->_protocol == &#39;https&#39;) ? 443 : 80;
        }
        $this->_uri = substr($req, $pos);
        if ($this->_uri == &#39;&#39;) $this->_uri = &#39;/&#39;;
    }
    // constructor
    function HTTPRequest($url) {
        $this->_url = $url;
        $this->_scan_url();
    }
    // download URL to string
    function DownloadToString() {
        $crlf = "rn";
        // generate request
        $req = &#39;GET &#39; . $this->_uri . &#39; HTTP/1.0&#39; . $crlf . &#39;Host: &#39; . $this->_host . $crlf . $crlf;
        // fetch
        $this->_fp = fsockopen(($this->_protocol == &#39;https&#39; ? &#39;ssl://&#39; : &#39;&#39;) . $this->_host, $this->_port);
        fwrite($this->_fp, $req);
        while (is_resource($this->_fp) && $this->_fp && !feof($this->_fp)) $response.= fread($this->_fp, 1024);
        fclose($this->_fp);
        // split header and body
        $pos = strpos($response, $crlf . $crlf);
        if ($pos === false) return ($response);
        $header = substr($response, 0, $pos);
        $body = substr($response, $pos + 2 * strlen($crlf));
        // parse headers
        $headers = array();
        $lines = explode($crlf, $header);
        foreach ($lines as $line) if (($pos = strpos($line, &#39;:&#39;)) !== false) $headers[strtolower(trim(substr($line, 0, $pos))) ] = trim(substr($line, $pos + 1));
        // redirection?
        if (isset($headers[&#39;location&#39;])) {
            $http = new HTTPRequest($headers[&#39;location&#39;]);
            return ($http->DownloadToString($http));
        } else {
            return ($body);
        }
    }
}
//使用方法
$r = new HTTPRequest(&#39;http://www.phprm.com&#39;);
$str = $r->DownloadToString();
?>
Copy after login


Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

What is the difference between using foreach and iterator to delete elements when traversing Java ArrayList? What is the difference between using foreach and iterator to delete elements when traversing Java ArrayList? Apr 27, 2023 pm 03:40 PM

1. The difference between Iterator and foreach is the polymorphic difference (the bottom layer of foreach is Iterator) Iterator is an interface type, it does not care about the type of collection or array; both for and foreach need to know the type of collection first, even the type of elements in the collection; 1. Why is it said that the bottom layer of foreach is the code written by Iterator: Decompiled code: 2. The difference between remove in foreach and iterator. First, look at the Alibaba Java Development Manual, but no error will be reported in case 1, and an error will be reported in case 2 (java. util.ConcurrentModificationException) first

How to determine the number of foreach loop in php How to determine the number of foreach loop in php Jul 10, 2023 pm 02:18 PM

​The steps for PHP to determine the number of the foreach loop: 1. Create an array of "$fruits"; 2. Create a counter variable "$counter" with an initial value of 0; 3. Use "foreach" to loop through the array, and Increase the value of the counter variable in the loop body, and then output each element and their index; 4. Output the value of the counter variable outside the "foreach" loop to confirm which element the loop reaches.

How to solve PHP Warning: fopen(): SSL operation failed in file.php on line X How to solve PHP Warning: fopen(): SSL operation failed in file.php on line X Aug 25, 2023 am 09:22 AM

How to solve PHPWarning:fopen():SSLoperationfailedinfile.phponlineX In PHP programming, we often use the fopen function to open files or URLs and perform related operations. However, when using the fopen function, sometimes you will encounter something similar to Warning:fopen():SSLoperationfailedinfile.p

How to solve PHP Warning: fopen(): failed to open stream: Permission denied How to solve PHP Warning: fopen(): failed to open stream: Permission denied Aug 20, 2023 pm 01:45 PM

How to solve PHPWarning:fopen():failedtoopenstream:Permissiondenied In the process of developing PHP programs, we often encounter some error messages, such as PHPWarning:fopen():failedtoopenstream:Permissiondenied. This error is usually due to incorrect file or directory permissions

How to solve PHP Warning: fopen(): failed to open stream: No such file or directory How to solve PHP Warning: fopen(): failed to open stream: No such file or directory Aug 19, 2023 am 10:44 AM

How to solve PHPWarning:fopen():failedtoopenstream:Nosuchfileordirectory In the process of using PHP development, we often encounter some file operation problems, one of which is "PHPWarning:fopen():failedtoopenstream:Nosuchfileordirectory"

What does php request mean? What does php request mean? Jul 07, 2021 pm 01:49 PM

The Chinese meaning of request is "request". It is a global variable in PHP and is an array containing "$_POST", "$_GET" and "$_COOKIE". The "$_REQUEST" variable can obtain data and COOKIE information submitted by POST or GET.

How to use the urllib.request.urlopen() function to send a GET request in Python 3.x How to use the urllib.request.urlopen() function to send a GET request in Python 3.x Jul 30, 2023 am 11:28 AM

How to use the urllib.request.urlopen() function in Python3.x to send a GET request. In network programming, we often need to obtain data from a remote server by sending an HTTP request. In Python, we can use the urllib.request.urlopen() function in the urllib module to send an HTTP request and get the response returned by the server. This article will introduce how to use

Usage of fopen function in Matlab Usage of fopen function in Matlab Nov 28, 2023 am 11:03 AM

In Matlab, the fopen function is used to open a file and return the file identifier for subsequent reading or writing operations on the file. Select the appropriate permission options to open the file as needed, and promptly close the file when the operation is complete. It should be noted that after opening a file, you need to ensure that the file is closed in time when it is no longer needed to release system resources. In addition, if the file opening fails or an operation error occurs, the error handling mechanism can be used to handle it accordingly.

See all articles