python - 代理导致Scrapy无法访问网站?
大家讲道理
大家讲道理 2017-04-18 09:36:37
0
3
367
大家讲道理
大家讲道理

光阴似箭催人老,日月如移越少年。

reply all(3)
大家讲道理

Turn off Lanten~

巴扎黑

Answer source address: CSDN blog: Scrapy usage problems

At the end of the answer, it is mentioned that proxy problems cause some websites to be inaccessible. The solution is to modify it in settings.py:
DOWNLOADER_MIDDLEWARES = {

'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None,

}

After testing and modification, the program can run

洪涛

It should be that Lanten has changed the system's proxy, causing your crawler to be forced to use this proxy IP to access

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template