python爬虫 - python访问豆瓣突然遇到403forbidden,用过edge浏览器也无法访问？

Question

如题，使用requests访问，昨天写代码的时候两个多小时一直在访问豆瓣主页，没有什么问题，今天用相同的程序就变成了403forbidden，chrome、edge访问网址也变成了403forbidden..不知是什么原因..

Answer 1

It’s good to learn to crawl Douban. If you want to apply it, please use their open platform http://developers.douban.com/

Answer 2

I caught him too hard and was blocked by Douban~

Answer 3

How much effort does it take to create some UGC? How can you allow you to grab it casually?
You can use high-density proxy IP to capture. We have captured Taobao's data before, but we banned it after capturing it a few times. Later, we used a proxy to solve the problem perfectly, and the other party couldn't ban it at all.

Answer 4

Douban limits the crawling frequency through cookies. After analyzing the cookies, it can be forged. Last month, 1.07 million pages were crawled in two hours.