python – Über TwistedPipline-Fehler in Scrapy

Question

> python3.5 / pycharm 

crawlspider 返回的 item 是绝对没有问题的，接上TwistedPiplineIch habe gerade einen Fehler gemeldet. Kann mir bitte ein Lehrer bei der Analyse des Fehlercodes helfen?

# TwistedPipline
class MysqlTwistedPipline(object):
    def __init__(self, dbpool):
        self.dbpool = dbpool

    @classmethod
    def from_settings(cls, settings):
        dbparms = dict(
            host = settings["MYSQL_HOST"],
            db = settings["MYSQL_DBNAME"],
            user = settings["MYSQL_USER"],
            passwd = settings["MYSQL_PASSWORD"],
            charset='utf8',
            cursorclass=MySQLdb.cursors.DictCursor,
            use_unicode=True,
        )
        dbpool = adbapi.ConnectionPool("MySQLdb", **dbparms)

        return cls(dbpool)

    def process_item(self, item, spider):
        query = self.dbpool.runInteraction(self.do_insert, item)
        query.addErrback(self.handle_error, item, spider) 

    def handle_error(self, failure, item, spider):
        print (failure)

    def do_insert(self, cursor, item):
        insert_sql, params = item.insert_values()
        print (insert_sql, params)
        cursor.execute(insert_sql, params)
        return item
        

Fehlercode

2017-05-01 00:06:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.lagou.com/jobs/2108656.html> (referer: https://www.lagou.com/zhaopin/Python/)
2017-05-01 00:06:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.lagou.com/jobs/2108656.html>
None
[Failure instance: Traceback: <class 'TypeError'>: connect() argument 3 must be str, not None
D:\Python35\lib\threading.py:914:_bootstrap_inner
D:\Python35\lib\threading.py:862:run
D:\Python35\lib\site-packages\twisted\_threads\_threadworker.py:46:work
D:\Python35\lib\site-packages\twisted\_threads\_team.py:190:doWork
--- <exception caught here> ---
D:\Python35\lib\site-packages\twisted\python\threadpool.py:250:inContext
D:\Python35\lib\site-packages\twisted\python\threadpool.py:266:<lambda>
D:\Python35\lib\site-packages\twisted\python\context.py:122:callWithContext
D:\Python35\lib\site-packages\twisted\python\context.py:85:callWithContext
D:\Python35\lib\site-packages\twisted\enterprise\adbapi.py:464:_runInteraction
D:\Python35\lib\site-packages\twisted\enterprise\adbapi.py:36:__init__
D:\Python35\lib\site-packages\twisted\enterprise\adbapi.py:76:reconnect
D:\Python35\lib\site-packages\twisted\enterprise\adbapi.py:431:connect
D:\Python35\lib\site-packages\MySQLdb\__init__.py:86:Connect
D:\Python35\lib\site-packages\MySQLdb\connections.py:204:__init__
]
 

Answer 1

从提示看出,你的配置(settings)的时候有1个参数是None,而不是字符串