Home > Backend Development > Python Tutorial > Python extracts the most popular Q&A content on Zhihu

Python extracts the most popular Q&A content on Zhihu

大家讲道理
Release: 2016-11-09 11:29:25
Original
1129 people have browsed it

#-*- coding: utf-8 -*-
import urllib.request
import re
from _io import open
def yunpan_search():
    url = "https://www.zhihu.com/explore"
    req = urllib.request.Request(url, headers = {
        'Connection': 'Keep-Alive',
        'Accept': 'text/html, application/xhtml+xml, */*',
       'Accept-Language': 'en-US,en;q=0.8,zh-Hans-CN;q=0.5,zh-Hans;q=0.3',
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko'
})
    opener = urllib.request.urlopen(req)
    html = opener.read()
    html = html.decode('utf-8')
    rex = &#39;(?<=<textarea class="content hidden">\n).*?(?=<span class="answer-date-link-wrap">)&#39;
    m = re.findall(rex,html,re.S)
    f = open(&#39;/root/Desktop/zhihu.txt&#39;,&#39;w&#39;)
    for i in m:
        f.write(i)
        f.write(&#39;\n\n&#39;)
    f.close()
    print("抓取成功!")
    file = open(&#39;/root/Desktop/zhihu.txt&#39;,&#39;r+&#39;)
    fullfile = file.readlines()
    text = []
    p = re.compile(r&#39;\w*&#39;, re.L)
    pp = re.compile(r"(&;)*")
    for line in fullfile:
        lines = p.sub(&#39;&#39;,line)
        liness = pp.sub(&#39;&#39;,lines)
        text.append(liness)
    file.seek(0)
    file.truncate(0)
    file.writelines(text)
    file.close()
    print("处理成功!")
 
if __name__==&#39;__main__&#39;:
    yunpan_search()
Copy after login

source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template