Complete collection of PHP collection programs_PHP tutorial

I haven’t officially published anything on the forum for a long time. Today I will share my collection code with you!

Things:

The idea of ​​the collection program is very simple and can be roughly divided into the following steps:
1. Get the remote file source code (file_get_contents or use fopen).
2. Analyze the code to get the content you want (here Use regular matching, usually to get paging).
3. Download and store the content obtained from the root.

The second step here may have to be repeated several times. For example, we need to analyze the paging address first, and then analyze the content of the inner page to get what we want.

Code:

I remember posting part of the code before and today I will simply post it here
Copy the PHP content to the clipboard
PHP code:
@$nl=file_get_contents($rs[url ]);//Capture remote content
preg_match_all("/var url = "gameswf/(.*?).swf";/is",$nl,$connect);//Perform formal matching to obtain what you want Contents
mysql_query("insert...insert database part");

The above code is the code used for all collections. Of course, you can also use fope to do it. I personally like to use file_get_contents.

Now I will share my method of downloading flash images to local, it is too simple with two lines of code
Copy PHP content to the clipboard
PHP code:
if(@copy($url,$ newurl)){
echo ok;
}

I have posted an image download function on the forum before and I will post it here for everyone.
Copy PHP content to the clipboard
PHP code:
/*Local image function*/
function getimg($url,$filename){
/*Determine whether the url of the image is empty, if it is empty, stop the function*/
if($url==""){
return false;
}
/*Get the extension of the image and store it in the variable $ext*/
$ext=strrchr($url,".");
/*Determine whether it is legal Picture file*/
if($ext!=".gif" && $ext!=".jpg"){
                                                                                                                                                                                                                                    through 🎜>         $img=file_get_contents($url); Go to the pointed file*/
fwrite($fp,$img);
/*Close the file*/
fclose($fp);
/*Return the new file name of the picture*/
         return $filename.$ext;
}


Share your personal collecting thoughts:

1. Don’t use sites that are protected against hotlinking. In fact, you can fake your origins, but the cost of collecting such sites is too high

2. Collect sites as quickly as possible, preferably locally

3. When collecting, there are many times when you can store part of the data in the database first, and wait for the next step of processing later.

4. You must handle errors when collecting. I usually skip it if the collection fails three times. In the past, I would often get stuck picking out a piece of content just because I couldn't pick it up.

5. You must make good judgment before entering the database, check the legality of the content, and filter unnecessary strings.

http://www.bkjia.com/PHPjc/508292.html

www.bkjia.com

true

http: //www.bkjia.com/PHPjc/508292.htmlTechArticleI haven’t published anything formally on the forum for a long time. Today I will share my collection code with you! Idea: The idea of ​​the collection program is very simple and can be roughly divided into the following steps...