Use Python's Scrapy framework to crawl beautiful women's pictures in ten minutes

Scrapy is written in Python, lightweight, simple and lightweight, and very convenient to use. Using Scrapy can easily complete the collection of online data. It has completed a lot of work for us without having to expend great efforts to develop it ourselves. This article introduces the use of Python's Scrapy framework to crawl beautiful pictures in ten minutes. Friends who need it can refer to it

Introduction

scrapy is a python The following is a crawler framework that is rich in functions and quick and easy to use. You can use scrapy to quickly develop a simple crawler. A simple example given by the official is enough to prove its power:

rapid development

The 10-minute countdown starts below:

Of course, before starting, you can take a look at the scrapy introductory article we wrote before "Writing Python Crawlers from Zero Basics: Using the Scrapy Framework to Write Crawlers

1. Initialize the project

scrapy startproject mzt
cd mzt
scrapy genspider meizitu meizitu.com

##2. Add spider code:

Define scrapy.Item, add image_urls and images, and prepare for downloading images.

Modify start_urls to the initial page, add parse to process the list page, and add parse_item to process the item page.

3. Modify the configuration file:

DOWNLOAD_DELAY = 1 # 添加下载延迟配置
ITEM_PIPELINES = {'scrapy.pipelines.images.ImagesPipeline': 1} # 添加图片下载 pipeline
IMAGES_STORE = '.' # 设置图片保存目录

4 . Run the project:

scrapy crawl meizitu

Look at the project operation renderings

Wait for a while, it’s time to harvest

##More uses Python’s Scrapy framework to crawl beautiful women in ten minutes For pictures related articles, please pay attention to the PHP Chinese website!