python - scrapy 处理 文章 分页的内容
PHP中文网
PHP中文网 2017-04-18 10:32:48
0
3
398

如一篇文章有2-3页,然后想把这些内容页爬下来,拼接成一页,然后再放入数据库。
文章url如:article_1.html,article_2.html
item有:item['title'],item['content']
item['content']就是拼接成一页的内容。
大概怎么写呢?

PHP中文网
PHP中文网

认证0级讲师

reply all(3)
大家讲道理

Find the paging interface url

大家讲道理

Find the link to the next page and add it to the list of crawled URLs

洪涛

You can write regular rules in the rules to automatically scan for matching URLs

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template
About us Disclaimer Sitemap
php.cn:Public welfare online PHP training,Help PHP learners grow quickly!