PHP writing and collection experience_PHP tutorial-PHP Tutorial-php.cn

PHP writing and collection experience_PHP tutorial

WBOY

Release： 2016-07-13 17:38:00

Original

974 people have browsed it

I haven’t officially published anything on the forum for a long time. Today I will share with you my collection code!

Idea:

The idea of the collection program is very simple and can be roughly divided into the following steps:

1. Get the remote file source code (file_get_contents or use fopen).

2. Analyze the code to get the content you want (use regular matching here, usually to get pagination).

3. Download and store the content obtained from the root.

The second step here may have to be repeated several times. For example, we need to analyze the paging address first, and then analyze the content of the inner page to get what we want.

Code:

I remember that I posted some of the code before. Today I will simply post it here

Copy PHP content to clipboard

PHP code:

@$nl=file_get_contents($rs[url]);//Catch remote content
preg_match_all("/var url = "gameswf/(.*?).swf";/is",$nl,$connect);//Perform formal matching to get the content you want
mysql_query("insert...insert database part");

The above code is the code used for all collections. Of course, you can also use fope to do it. I personally like to use file_get_contents.

Now I’ll share my method of downloading flash images to local, it’s so simple with just two lines of code

Copy PHP content to clipboard

PHP code:

if(@copy($url,$newurl)){
echo ok;
}

I have posted an image download function on the forum before and I will post it here for everyone

Copy PHP content to clipboard

PHP code:

/*Local image function*/
function getimg($url,$filename){
/*Determine whether the url of the image is empty, and if it is empty, stop the function*/
　　if($url==""){
　　　　return false;
　　}
/*Get the extension of the image and store it in the variable $ext*/
　　$ext=strrchr($url,".");
/*Determine whether it is a legal image file*/
if($ext!=".gif" && $ext!=".jpg"){
　　　　return false;
　　}
/*Read pictures*/
$img=file_get_contents($url);
/*Open the specified file*/
　　$fp=@fopen($filename.$ext,"a");
/*Write the image to the pointed file*/
　　fwrite($fp,$img);
/*Close file*/
　　fclose($fp);
/*Return the new file name of the image*/
Return $filename.$ext;
}

Share your personal collection experience:

1. Don’t use sites that are protected against hotlinking. In fact, you can fake your origins, but the cost of collecting such sites is too high

2. Collect sites as quickly as possible, preferably locally

3. When collecting, there are many times when you can store part of the data in the database first, and wait for the next step of processing later.

4. You must handle errors when collecting. I usually skip it if the collection fails three times. In the past, I would often get stuck picking out a piece of content just because I couldn't pick it up.

5. You must make good judgment before entering the database, check the legality of the content, and filter unnecessary strings.