Python下载懒人图库JavaScript特效
Jun 10, 2016 pm 03:11 PM
python
download
这是一个简单的Python脚本,主要从懒人图库下载JavaScript特效模板,在脚本中使用了gevent这个第三方库,使用的时候需要先安装。
#!/usr/bin/python # -*- coding: utf-8 -*- import urllib,os,sys import gevent,re from gevent import monkey from bs4 import BeautifulSoup gevent.monkey.patch_socket() ''' Description:Python 爬虫抓取懒人图库的JS脚本模板 Author:admin Create-Date:2015-05-25 Version:1.0 ''' HTTP_URL = 'http://www.lanrentuku.com%s' DOWNLOAD_URL = HTTP_URL[:-2] + '/js/d%szip' reg=r'\d{1,}\.+' def encode(text): return text.encode("utf8") def createDirectory(curPath): myPath = os.path.join(getSubDirectory(), u'JS代码模板') if not os.path.exists(myPath): os.mkdir(myPath) return os.path.join(myPath, curPath) def getSubDirectory(): return os.getcwd() def schedule(a, b, c): per = 100.0 * a * b / c if per > 100 : per = 100 sys.stdout.write('%.1f%%\r' % per) sys.stdout.flush() def geturllist(url): url_list = {} html = urllib.urlopen(url) content = html.read() html.close() # 用BeautifulSoup解析 decodeHtml = BeautifulSoup(content) try: aTags = decodeHtml.find_all('div', {'class':'list-pngjs'})[0].find_all('a') except IndexError, e: print e aTags = None # 获取链接地址和标题 if aTags is not None: for a_tag in aTags: url_list[HTTP_URL % a_tag.get('href')] = a_tag.get_text() return url_list def download(down_url): try: m=re.search(reg,down_url[0]) name = DOWNLOAD_URL % m.group(0) urllib.urlretrieve(name,createDirectory(down_url[1] + name[-4:]),schedule) except Exception, e: print e.message def getpageurl(xUrl): # 进行列表页循环 return [xUrl % page for page in xrange(1,49)] if __name__ == '__main__': jobs = [] pageurl = getpageurl('http://www.lanrentuku.com/js/p%s.html') # 爬取所有链接 for i in pageurl: for k in geturllist(i).items(): jobs.append(gevent.spawn(download, k)) gevent.joinall(jobs)
Copy after login
以上所述就是本文的全部内容了,希望大家能够喜欢。
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot Article
How Long Does It Take To Beat Split Fiction?
3 weeks ago
By DDD
Repo: How To Revive Teammates
3 weeks ago
By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
3 weeks ago
By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
1 weeks ago
By 尊渡假赌尊渡假赌尊渡假赌

Hot tools Tags

Hot Article
How Long Does It Take To Beat Split Fiction?
3 weeks ago
By DDD
Repo: How To Revive Teammates
3 weeks ago
By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
3 weeks ago
By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
1 weeks ago
By 尊渡假赌尊渡假赌尊渡假赌

Hot Article Tags

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

What are the advantages and disadvantages of templating?

Google AI announces Gemini 1.5 Pro and Gemma 2 for developers

For only $250, Hugging Face's technical director teaches you how to fine-tune Llama 3 step by step

Share several .NET open source AI and LLM related project frameworks

A complete guide to golang function debugging and analysis
