Crawl Douban data in Node.js (detailed tutorial)

This article gives you a detailed analysis of the process of crawling Douban data with Node.js and the specific steps through examples. Interested friends can refer to it.

I always thought that my vue was good, and I always thought that webpack was good. When I was browsing node in MOOC today, I realized that I was still far behind. As we all know, vue-cli is based on webpack, and webpack is based on node. If you don’t know node, how can you understand webpack? So I asked myself a question to crawl Douban data, which is still in its infancy. Today I will briefly talk about the data crawled from Douban and display it in your own way on another page. I will follow up later.

1. Problems that need to be solved

1. Building services

2. How to process the crawled data

3. How to automatically open the default browser

2. Build a service

It’s good to build a service There are several ways. I used http at the beginning, but the disadvantage of http is that it cannot parse URLs of https protocol, so I used express. To parse URLs of https protocol, I used the request package. Douban’s URL is https,

Today I crawled the URL https://movie.douban.com/chart; as shown below, there are three parts I want to get, the picture, the movie name, and the movie link.

3. How to process the crawled data

How to process the data we crawled using request? The cheerio package allows us to process crawled html data like Jq.

①. First parse the data and get the html data of the crawled web page;

②. Then use the cheerio package to operate the crawled data and get the data you want.

③. Get the data, create html, and output it to the page. As shown below, the string splicing method I used is a bit stupid, and I haven't found a better method yet.

4. How to automatically open the default browser

I don’t know if you have looked at the configuration of webpack in vue-cli. Automatically Open the browser and use the opn package used by vue-cli.

This package is very convenient to use. Just import the package and call opn (url) directly;

5. Display

The above is what I compiled for everyone. I hope it will be helpful to everyone in the future. .

Related articles:

javaScript implements the pop-up floating layer effect when the mouse is hovering over text

Use Angular CLI for unit Methods of testing and E2E testing

##Detailed explanation of Build and Serve using Angular CLI

The above is the detailed content of Crawl Douban data in Node.js (detailed tutorial). For more information, please follow other related articles on the PHP Chinese website!