There are two different situations: all the URLs are known before the request and all the URLs are not known before the request The first situation: first remove the duplicates from the requested image URL array, and then initiate the request Second Two cases: put the requested URL in an object, and then check whether it has been requested. If it has been requested, no request will be initiated
In addition, the mechanism of NodeJS itself is not multi-threaded, but a single-process event loop, so simple flag bits can be used to implement the sentinel mechanism
Either you have requested it and then don’t request again. However, there is no guarantee that the requested download will be successful. If the download fails, request again
Is the name of the file saved after downloading the same? If they are the same, check whether the file exists and then download it. This can avoid repeated downloads.
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
There are two different situations: all the URLs are known before the request and all the URLs are not known before the request
The first situation: first remove the duplicates from the requested image URL array, and then initiate the request
Second Two cases: put the requested URL in an object, and then check whether it has been requested. If it has been requested, no request will be initiated
In addition, the mechanism of NodeJS itself is not multi-threaded, but a single-process event loop, so simple flag bits can be used to implement the sentinel mechanism
Either you have requested it and then don’t request again. However, there is no guarantee that the requested download will be successful. If the download fails, request again
Is the name of the file saved after downloading the same? If they are the same, check whether the file exists and then download it. This can avoid repeated downloads.
md5 and url