使用python实现模拟登陆并爬取返回页面的时候出现了乱码,目标网页的编码使用utf-8
相关代码:
#coding=utf-8
import urllib
import urllib2
headers={
'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding':'gzip, deflate',
'Accept-Language':'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3',
'Connection':'keep-alive',
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73 Safari/537.36'
}
payload={
'_eventId':'submit',
'lt':'_cF2A0EB3F-D044-046C-6F4A-C828DE0ACE8E_k8B4BE5F5-4CAD-375D-0DDC-FB84A18445DF',
'password':'',
'submit':'登 录',
'username':''
}
payload=urllib.urlencode(payload)
request = urllib2.Request(posturl, payload, headers)
print request
response = urllib2.urlopen(request)
text = response.read()
print text
控制台输出信息:
第一次遇见这种乱码比较懵逼
urllib2는 압축 문제를 처리하지 않으므로 다음과 같이 gzip을 사용하여 압축을 풀어야 합니다
으아악요약하자면 urllib2는 상대적으로 낮은 레벨이므로 요청을 사용하는 것이 좋습니다