Solution to 403 error in PHP Curl_PHP Tutorial-PHP Tutorial-php.cn

Solution to 403 error in PHP Curl_PHP Tutorial

WBOY

Release： 2016-07-13 10:29:16

Original

1183 people have browsed it

A small PHP application I use, I use curl to grab web pages and process them. In order to facilitate wall penetration, I use Privoxy as a proxy, which makes it easy to choose which websites use proxy and which do not. But today I encountered a strange problem. Visiting Google Baidu and other websites actually returned a 403 error, but accessing some other websites was fine. If you set it to not use a proxy, you can access it normally.

Does Google Baidu not allow proxy connection? Obviously it's impossible, so open curl's information output (curl_setopt($this->mSh, CURLOPT_VERBOSE, 1);) and take a look and get the following results:

Copy code The code is as follows:

* Trying 127.0.0.1... * connected
* Connected to 127.0.0.1 (127.0.0.1) port 8118 (#0)
* Establish HTTP proxy tunnel to www.baidu.com:80
> CONNECT www.baidu.com:80 HTTP/1.0
Host: www.baidu.com:80
User-Agent: Mozilla/4.0 ( compatible; MSIE 6.0; Windows NT 5.1; SV1)
Proxy-Connection: Keep-Alive

< HTTP/1.0 403 Connection not allowable
< X-Hint: If you read this message interactively, then you know why this happens ,-)
<
* The requested URL returned error: 403
* Received HTTP code 403 from proxy after CONNECT
* Closing connection #0
... Failed.

You can see that the proxy server is working normally, and it is indeed baidu that returned 403 error, but the reason must still be on my side. Finally, I got some inspiration from the Internet (1of2, 2of2)─I used proxytunnel instead of proxy.

In the code, there is this sentence:

Copy the code The code is as follows:

curl_setopt($this-> mSh, CURLOPT_HTTPPROXYTUNNEL, true);
curl_setopt($this->mSh, CURLOPT_PROXY, $phost);

There is no detailed explanation in the php document, but there is a detailed explanation in man curl, both are Proxy, proxytunnel (-p parameter) allows other protocols to be transmitted through the http proxy, while proxy (-x parameter) can only use the http protocol. So I guess that Google Baidu's server is at odds with curl's proxytunnel, so 403 is returned.

After disabling the first sentence of the above two lines of code, curl access returns to normal.

What is strange is that it is different under several operating systems. A MAC OSX requires explicitly disabling proxytunnel. Curl version:

Copy codeThe code is as follows:

$ curl --version
curl 7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2 .3
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz

The other ubuntu is not affected at all and can be used no matter what. , curl version:

Copy code The code is as follows:

$ curl --version
curl 7.18.2 (i486 -pc-linux-gnu) libcurl/7.18.2 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.10
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

centos on the MT host is fine too, curl version:

Copy code The code is as follows:

$ curl --version
curl 7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

It seems that it is not entirely a curl version problem, MAC OSX is indeed different.

There is another reason why curl will return a 403 error. If it is set:

Copy code The code is as follows:

curl_setopt($ch, CURLOPT_NOBODY, true);

needs to be set immediately:

Copy code The code is as follows:

curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');

Otherwise, a 403 error will be returned because the http server does not allow the HEAD command. Reference: Trouble with a cURL request in PHP (http://forums.devshed.com/php-development-5/trouble-with-a-curl-request-in-php-445222.html). The reason why curl on MAC OSX is special may be due to this reason.