Don’t be limited to the means of implementation. Technology is dead, but people are alive.
The essence of a crawler is to simulate people to do operations. As long as people can see it, it can see it; as long as people can do it, it can do it. Take your time and think about it.
Capture and analyze the packets. In fact, I think the search page is just a web page. You can capture the packets and take a look.
Don’t be limited to the means of implementation. Technology is dead, but people are alive.
The essence of a crawler is to simulate people to do operations. As long as people can see it, it can see it; as long as people can do it, it can do it. Take your time and think about it.