使用BeautifulSoup爬虫程序获取百度搜索结果的标题和url示例

WBOY
Release: 2016-06-06 11:28:41
Original
2598 people have browsed it

熟悉Java的jsoup包的话,对于Python的BeautifulSoup库应该很容易上手。

代码如下:


#coding: utf-8
import sys
import urllib
import urllib2
from BeautifulSoup import BeautifulSoup

question_word = "吃货 程序员"
url = "http://www.baidu.com/s?wd=" + urllib.quote(question_word.decode(sys.stdin.encoding).encode('gbk'))
htmlpage = urllib2.urlopen(url).read()
soup = BeautifulSoup(htmlpage)
print len(soup.findAll("table", {"class": "result"}))
for result_table in soup.findAll("table", {"class": "result"}):
    a_click = result_table.find("a")
    print "-----标题----\n" + a_click.renderContents()#标题
    print "----链接----\n" + str(a_click.get("href"))#链接
    print "----描述----\n" + result_table.find("div", {"class": "c-abstract"}).renderContents()#描述
    print

Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template