Collect Tmall New Balance flagship store product data, Tmall New Balance flagship store

Collect Tmall New Balance flagship store product data, Tmall New Balance flagship store_PHP tutorial

WBOY

Release： 2016-07-13 10:14:37

Original

1093 people have browsed it

Collect product data of Tmall New Balance flagship store, Tmall New Balance flagship store

1. Download Fire Spider Collector

Download address of Fire Spider Collector:

http://firespider.duapp.com/FireSpiderWeb/index.html

2. Installation

Fire Spider Collector includes a client and a server installation package. The installation is very convenient and can be completed in the next step. For specific installation steps, please refer to the installation manual provided.

3. Start

Start the server: There is a shortcut "Fire Spider Server Startup" on the desktop, right click -> Run as administrator.

Start the client: Double-click the desktop shortcut "FireSpider"

4. Trial

After installation, several collection tasks are included by default. Open the client, as shown in the figure below, select a task and click "Perform Collection".

You can see that the collection has started.

If you need to stop collection, select the collection task and click "Exit" next to the "Perform Collection" button.

5. Establish New Balance Task

After installation, a task of collecting Luotuo Tmall stores - "Luotuo-List Page" is included by default. We just need to refer to this and build our own task. For example, if I want to collect "New Balance Flagship Store", search for "New Balance" on Tmall, enter the New Balance Tmall store, and find the New Balance list page. The link is as follows:

http://newbalance.tmall.com/search.htm?spm=a220m.1000858.1000725.4.vvTbdj&rn=bda5ce8edbc3307bf802e68880861de3&user_number_id=520557274

Next, create a task to collect New Balance

Output http://localhost:8090/FireSpider/html/index.html in the browser to open the management center

Create a new task. Except for the "task name", "link" and "data upload link", they are actually the same as the "Camel-List Page". Link fill in the link to the New Balance list page just now.

Pay attention to the "Data Upload Link" field: This is when the collector collects data, it will push the data to this link in POST mode. Because the collector doesn't know what we use the collected data for, it has to leave the data to us for processing. We can open a REST service locally, such as apache, tomcat, iis, etc.

Here we fill in the "data upload link" as "http://localhost/myphp/index.php", here we use PHP. I will talk about this PHP script later.

If you just want to experience the collection, the "data upload link" can also be left blank.

Click to save

6. Build WAMP

This step is not necessary if you do not need to process the collected data yourself. What we are talking about here is PHP custom collection data processing, other languages can also be skipped.

6.1 Install wamp

No need to say more about this, just keep going to the next step.

6.2 Creating Alias

Enter alias name

Enter the directory corresponding to alias

6.3 index.php script

[php] view plaincopy

// Note that PHP files must be saved in UTF-8 BOM-free format
// Get the data from POST
// Because firespider does not set the content type when posting, so you need to use this method to get the post data, you cannot use $_POST
$content = file_get_contents("php://input");
// JSON decoding
$p = json_decode($content);
// You can write your own logic later. This is just saved to a file
$james=fopen("e:/test/json-".$p->urlId.".txt", "w ");
if(!$james) {
echo'file not exist';
exit;
}
fwrite($james, $content);
fclose($james);

7. Collect New Balance

After the task is created, you can start collecting. Open the collector client, on the task tab, select the "New Balance" we just created, and click "Execute Collection"

After the collection starts, we can see the collected data in josn-xxx.txt under e:/test/.