python - There is no problem when debugging pyspider. When I click run, encoding problems are reported.

Question

There are no problems during debugging. If you click run, encoding problems will be reported. In the same two collections, one always reports an error, and the other one is completely fine

taskid

d7221a2be620c4ef60e874a1d93e79d1

lastcrawltime

1499144004.75187 (20 minutes ago)

updatetime

1499144004.7518892 (20 minutes ago)

exetime

1499144014.7518687 (20 minutes ago)

track.fetch 1.32ms

{
  "content": "",
  "encoding": null,
  "error": "'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)",
  "headers": {},
  "ok": false,
  "redirect_url": null,
  "status_code": 599,
  "time": 0.0013222694396972656
}

track.process 0.83ms

'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)
 = self.gen.throw(*exc_info)
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py", line 378, in http_fetch
        response = yield gen.maybe_future(self.http_client.fetch(request))
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py", line 1055, in run
        value = future.result()
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py", line 238, in result
        raise_exc_info(self._exc_info)
      File "<string>", line 4, in raise_exc_info
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py", line 214, in _process_queue
        curl.info["headers"])
      File "/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py", line 306, in _curl_setup_request
        for k, v in request.headers.get_all()])
    Exception: 'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)

{
  "exception": "'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)",
  "follows": 0,
  "logs": " = self.gen.throw(*exc_info)\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/pyspider/fetcher/tornado_fetcher.py\", line 378, in http_fetch\n        response = yield gen.maybe_future(self.http_client.fetch(request))\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/gen.py\", line 1055, in run\n        value = future.result()\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/concurrent.py\", line 238, in result\n        raise_exc_info(self._exc_info)\n      File \"<string>\", line 4, in raise_exc_info\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py\", line 214, in _process_queue\n        curl.info[\"headers\"])\n      File \"/root/workspaces/pyspider3/lib/python3.5/site-packages/tornado/curl_httpclient.py\", line 306, in _curl_setup_request\n        for k, v in request.headers.get_all()])\n    Exception: 'ascii' codec can't encode character '\uff09' in position 94: ordinal not in range(128)\n",
  "ok": false,
  "result": null,
  "time": 0.0008292198181152344
}

schedule

{
  "age": 10,
  "exetime": 1499144014.7518687,
  "retried": 3
}

process

{
  "callback": "index_page"
}

fetch

{}

Answer 1

There is something wrong with the headers setting, I will delete it and it will be fixed in seconds.

Answer 2

#!/usr/bin/env python
# -*- encoding: utf-8 -*-

Look at the first two lines of your code, right?