python - scrapy cannot crawl data and cannot create txt files

Question

Answer 1

It’s a problem with xpath parsing path, thank you everyone

Answer 2

There should be a problem with the writing mode, try ab.

Answer 3

Is it running on Linux? Is there any permission issue?

Answer 4

Looking at the log output, it seems that scrapy exited early when it encountered robot.txt. Try changing ROBOTSTXT_OBEY to ROBOTSTXT_OBEY = False in settings. If it still doesn’t work, stop and debug it