python socket 超时设置 errno 10054
python socket.error: [Errno 10054] 远程主机强迫关闭了一个现有的连接。问题解决方案：
前几天使用python读取网页。因为对一个网站大量的使用urlopen操作，所以会被那个网站认定为攻击行为。有时就不再允许下载。导致urlopen()后，request.read()一直卡死在那里。最后会抛出errno 10054.
这个错误是connection reset by peer.也就是传说的远端主机重置了此连接。原因可能是socket超时时间过长；也可能是request = urllib.request.urlopen(url)之后，没有进行request.close()操作；也可能是没有sleep几秒，导致网站认定这种行为是攻击。
具体解决方案如下面的代码：
01.import socket 02.import time 03.timeout = 20 04.socket.setdefaulttimeout(timeout)#这里对整个socket层设置超时时间。后续文件中如果再使用到socket，不必再设置 05.sleep_download_time = 10 06.time.sleep(sleep_download_time) #这里时间自己设定 07.request = urllib.request.urlopen(url)#这里是要读取内容的url 08.content = request.read()#读取，一般会在这里报异常 09.request.close()#记得要关闭
因为urlopen之后的read()操作其实是调用了socket层的某些函数。所以设置socket缺省超时时间，就可以让网络自己断掉。不必在read()处一直等待。
当然你也可以再外层再写几个try,except,例如：
try:
time.sleep(self.sleep_download_time)
request = urllib.request.urlopen(url)
content = request.read()
request.close()
except UnicodeDecodeError as e:
print('-----UnicodeDecodeError url:',url)
except urllib.error.URLError as e:
print("-----urlError url:",url)
except socket.timeout as e:
print("-----socket timout:",url)
一般来说就莫有问题了。我测试了几千个网页的下载，然后才说出此话。不过如果是下载成千上万的，我做了下测试，ms还是会跳出这个异常。可能是time.sleep()的时间太短，也可能是网络突然中断。我使用urllib.request.retrieve()测试了一下，发现不停的下载数据，总会有失败的情况出现。
简单的处理方法是：首先参照的我的文章：python检查点简单实现 。先做一个检查点。然后将以上会跑出异常的那段代码while True一下。参见下面的伪代码：
def Download_auto(downloadlist,fun,sleep_time=15):
while True:
try: # 外包一层try
value = fun(downloadlist,sleep_time) # 这里的fun是你的下载函数，我当函数指针传进来。
# 只有正常执行方能退出。
if value == Util.SUCCESS:
break
except : # 如果发生了10054或者IOError或者XXXError
sleep_time += 5 #多睡5秒，重新执行以上的download.因为做了检查点的缘故，上面的程序会从抛出异常的地方继续执行。防止了因为网络连接不稳定带来的程序中断。
print('enlarge sleep time:',sleep_time)
不过对于找不到相应的网页，又要做另外的一番处理：
# 打印下载信息
def reporthook(blocks_read, block_size, total_size):
if not blocks_read:
print ('Connection opened')
if total_size < 0:
print ('Read %d blocks' % blocks_read)
else:
# 如果找不到，页面不存在，可能totalsize就是0，不能计算百分比
print('downloading:%d MB, totalsize:%d MB' % (blocks_read*block_size/1048576.0,total_size/1048576.0))
def Download(path,url):
#url = 'http://downloads.sourceforge.net/sourceforge/alliancep2p/Alliance-v1.0.6.jar'
#filename = url.rsplit("/")[-1]
try:
# python自带的下载函数
urllib.request.urlretrieve(url, path, reporthook)
except IOError as e: # 如果找不到，好像会引发IOError。
print("download ",url,"/nerror:",e)
print("Done:%s/nCopy to:%s" %(url,path))
如果大家还是遇到了问题...请再评论里注上其他解决方案。
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1359
52
What is the function of C language sum?
Apr 03, 2025 pm 02:21 PM
There is no built-in sum function in C language, so it needs to be written by yourself. Sum can be achieved by traversing the array and accumulating elements: Loop version: Sum is calculated using for loop and array length. Pointer version: Use pointers to point to array elements, and efficient summing is achieved through self-increment pointers. Dynamically allocate array version: Dynamically allocate arrays and manage memory yourself, ensuring that allocated memory is freed to prevent memory leaks.
Who gets paid more Python or JavaScript?
Apr 04, 2025 am 12:09 AM
There is no absolute salary for Python and JavaScript developers, depending on skills and industry needs. 1. Python may be paid more in data science and machine learning. 2. JavaScript has great demand in front-end and full-stack development, and its salary is also considerable. 3. Influencing factors include experience, geographical location, company size and specific skills.
Is distinctIdistinguish related?
Apr 03, 2025 pm 10:30 PM
Although distinct and distinct are related to distinction, they are used differently: distinct (adjective) describes the uniqueness of things themselves and is used to emphasize differences between things; distinct (verb) represents the distinction behavior or ability, and is used to describe the discrimination process. In programming, distinct is often used to represent the uniqueness of elements in a collection, such as deduplication operations; distinct is reflected in the design of algorithms or functions, such as distinguishing odd and even numbers. When optimizing, the distinct operation should select the appropriate algorithm and data structure, while the distinct operation should optimize the distinction between logical efficiency and pay attention to writing clear and readable code.
How to understand !x in C?
Apr 03, 2025 pm 02:33 PM
!x Understanding !x is a logical non-operator in C language. It booleans the value of x, that is, true changes to false, false changes to true. But be aware that truth and falsehood in C are represented by numerical values rather than boolean types, non-zero is regarded as true, and only 0 is regarded as false. Therefore, !x deals with negative numbers the same as positive numbers and is considered true.
Can C language user identifiers contain spaces?
Apr 03, 2025 pm 01:51 PM
C language identifiers cannot contain spaces because they can cause confusion and difficulty in maintaining. The specific rules are as follows: they must start with letters or underscores. Can contain letters, numbers, or underscores. Cannot contain illegal characters (such as special symbols).
What does sum mean in C language?
Apr 03, 2025 pm 02:36 PM
There is no built-in sum function in C for sum, but it can be implemented by: using a loop to accumulate elements one by one; using a pointer to access and accumulate elements one by one; for large data volumes, consider parallel calculations.
How to apply snake nomenclature in C language?
Apr 03, 2025 pm 01:03 PM
In C language, snake nomenclature is a coding style convention, which uses underscores to connect multiple words to form variable names or function names to enhance readability. Although it won't affect compilation and operation, lengthy naming, IDE support issues, and historical baggage need to be considered.
Does H5 page production require continuous maintenance?
Apr 05, 2025 pm 11:27 PM
The H5 page needs to be maintained continuously, because of factors such as code vulnerabilities, browser compatibility, performance optimization, security updates and user experience improvements. Effective maintenance methods include establishing a complete testing system, using version control tools, regularly monitoring page performance, collecting user feedback and formulating maintenance plans.
