RT, when using python to crawl, everything is normal at the beginning, but suddenly the web page freezes, and after waiting for a while, it returns to normal. However, when crawling again at this time, it will report "DNS lookup failed". I have tried many websites. , but the web page opens normally.
Please tell me the reasons and solutions. Kneel down and thank you! !
http is a protocol, not part of the address, so DNS resolution fails. Let’s start from this aspect.
For example, if you want to parse Baidu’s website address:
Correct address: www.baidu.com
Incorrect address: http://www.baidu.com