I did a 0 o'clock ticket grab for the Taiwan Railway. It was also done with PHP at first. Later I found that there were a lot of tickets to grab at 0 o'clock every day. I made it concurrent and executed at the same time. I used nodejs to call php and solved the problem. Concurrency issues, although crude, can be used. . . It used to take 5 minutes to complete 20 tickets, but now it only takes more than 30 seconds to complete 20 tickets. . .
I have written a lot of crawlers using PHP, and I am very good at processing data, but PHP itself is not suitable for things like crawlers. Maybe Python is worth a try, I haven’t tried it
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
python+1
Recommend an article
Blog crawler technology sharing
I did a 0 o'clock ticket grab for the Taiwan Railway. It was also done with PHP at first. Later I found that there were a lot of tickets to grab at 0 o'clock every day. I made it concurrent and executed at the same time. I used nodejs to call php and solved the problem. Concurrency issues, although crude, can be used. . . It used to take 5 minutes to complete 20 tickets, but now it only takes more than 30 seconds to complete 20 tickets. . .
I have written a lot of crawlers using PHP, and I am very good at processing data, but PHP itself is not suitable for things like crawlers. Maybe Python is worth a try, I haven’t tried it
Why do so many people recommend node.js? Could it be that there were no crawlers when there was no node.js a few years ago?
Python
strongsupport
The simplest one is javascript. It is completely scripted to write a crawler. It is recommended to use the Archer Cloud crawler framework.
You can also write crawlers using javascript and write and execute them in the cloud. Source code address:
https://github.com/ShenJianSh...
I have seen a crawler written in Go language. According to the author, the efficiency is super high
Nodejs, Java, C#, VB, PHP, Python, Ruby, Perl