PHP Collection Snoopy Detailed Explanation
PHP Collection Tool Snoopy Application Detailed Explanation
Snoopy is a PHP class that is used to simulate the functions of a browser and can obtain web content and send forms. The correct operation of Snoopy requires that your server's PHP version is 4 or above and supports PCRE (Perl Compatible Regular Expressions), which is supported by basic LAMP services.
1. Some features of Snoopy:
1. Fetch the content of the web page fetch
2. Fetch the text content of the web page (remove HTML tags) fetchtext
3. Fetch the links of the web page, form fetchlinks fetchform
4. Support proxy host
5. Support basic username/password verification
6. Support setting user_agent, referer (source), cookies and header content (header file)
7. Support browser redirection and control the redirection depth
8. Can expand the links in the web page into high-quality URLs (default)
9. Submit data and get the return value
10. Support tracking HTML framework
11. Support passing cookies when redirecting, it requires PHP4 or above, because It is a PHP class and does not need to be extended. It is the best choice when the server does not support curl.
2. Class method:
fetch($URI)
————–
This is the method used to grab the content of the web page. The $URI parameter is the URL address of the crawled web page. The fetched results are stored in $this->results. If you are scraping a frame, Snoopy will track each frame and store it in an array, then store it in $this->results.
fetchtext($URI)
——————
This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the text content in the web page.
fetchform($URI)
——————
This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the form content (form) in the web page.
fetchlinks($URI)
————-
This method is similar to fetch(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return links in the web page.
By default, relative links will be automatically completed and converted into complete URLs.
submit($URI,$formvars)
——————-
This method sends a confirmation form to the link address specified by $URL. $formvars is an array that stores form parameters.
submittext($URI,$formvars)
————————–
This method is similar to submit(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return the page after login. text content.
submitlinks($URI)
————-
This method is similar to submit(). The only difference is that this method will remove HTML tags and other irrelevant data, and only return links in the web page.
By default, relative links will be automatically completed and converted into complete URLs.
3. Class attributes: (The default value is in brackets)
$host The connected host
$port The connected port
$proxy_host The proxy host used, if any
$proxy_port The proxy host port used, if any
$agent User agent camouflage (Snoopy v0.1)
$referer source information, if any
$cookies cookies, if any
$rawheaders Other header information, if any
$maxredirs Maximum weight Number of redirects, 0=not allowed (5)
$offsiteok whether or not to allow redirects off-site. (true)
$expandlinks Whether to complete all links to complete addresses (true)
$user authentication user name, if any If so
$pass authentication user name, if any
$accept http acceptance type (image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*)
$error Where to report the error, if any
$response_code The response code returned from the server
$headers The header information returned from the server
$maxlength The longest returned data length
$read_timeout The read operation timeout (requires PHP 4 Beta 4+)
Set to 0 for no timeout
$timed_out If a read operation times out, this attribute returns true (requires PHP 4 Beta 4+)
$maxframes The maximum number of frames allowed to be tracked
$status The status of the captured http
$temp_dir The web server can write Temporary file directory (/tmp)
$curl_path cURL binary directory, if there is no cURL binary, set it to false
Four. The following is the demo
include "Snoopy.class.php";
$snoopy = new Snoopy;
$snoopy- >proxy_host = "www.baidu.com";
$snoopy->proxy_port = "8080";
$snoopy->agent = "(compatible; MSIE 4.01; MSN 2.5; AOL 4.0; Windows 98)";
$snoopy->referer = "http://www.baidu.com/";
$snoopy->cookies["SessionID"] = 238472834723489l;
$snoopy->cookies["favoriteColor"] = "RED";
$snoopy->rawheaders["Pragma"] = "no-cache";
$snoopy->maxredirs = 2;
$snoopy->offsiteok = false;
$snoopy->expandlinks = false;
$snoopy->user = "joe";
$snoopy->pass = "bloe";
if($snoopy->fetchtext("http://www.baidu.com"))
{
echo "
".htmlspecialchars($snoopy->results)."n";