python - pyspider使用redis分布式

Question

在部署pyspider分布式时,尝试连接redis集群
配置项为： "message_queue":"redis://127.0.0.1:6380/db"
在启动时：报如下错误
[E 170221 09:40:39 tornado_fetcher:529] MOVED 12068 127.0.0.1:6380

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pyspider/fetcher/tornado_fetcher.py", line 515, in queue_loop
    if self.outqueue.full():
  File "/usr/lib/python2.6/site-packages/pyspider/message_queue/redis_queue.py", line 50, in full
    if self.maxsize and self.qsize() >= self.maxsize:
  File "/usr/lib/python2.6/site-packages/pyspider/message_queue/redis_queue.py", line 40, in qsize
    self.last_qsize = self.redis.llen(self.name)
  File "/usr/lib/python2.6/site-packages/redis/client.py", line 1219, in llen
    return self.execute_command('LLEN', name)
  File "/usr/lib/python2.6/site-packages/redis/client.py", line 573, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/usr/lib/python2.6/site-packages/redis/client.py", line 585, in parse_response
    response = connection.read_response()
  File "/usr/lib/python2.6/site-packages/redis/connection.py", line 582, in read_response
    raise response
ResponseError: MOVED 12068 127.0.0.1:6380

经查找，得到：因为redis由单节点变为集群，而python的redis连接包暂时还不支持redis集群连接方式，需要更换连接包（redis-py-cluster）。

在安装完redis-py-cluster之后，依旧报上面的错误。

Answer 1

The library used by link redis is written in the code, and it will not be switched after you install it.
Even if you want to change the code, make sure the API is compatible.