Program execution times out when performing batch image processing in PHP

This batch processing is because the project itself has been outsourced many times, and the pictures are stored everywhere, and there is no special generation of thumbnails and uniform-sized pictures. This time, the pictures are extracted from the original mixed image and text data. , store it in the specified directory, crop the size and generate the corresponding thumbnail, and store the result in the database.

In the process of batch program processing, it is inevitable that the program execution time will be too long, resulting in 504 errors, etc. Adjusting the server configuration is actually not a good solution. After all, it will affect the production environment...

But I am lazy and don’t want to perform paging request processing for the pictures extracted once^_^

By the way, the maximum number of pictures extracted from the data processed at one time is 3000~

Now let’s get to the point. Our main purpose is to extend the program execution time as much as possible so that all the pictures that should be captured are captured, and the processing is completed and written to the database.

Our environment is PHP5.6 based on nginx. First of all, we should pay attention to the code. One is to put set_time_limit(0), which avoids adjusting php.ini. Of course, there are some basic things such as maximum memory, etc. It still needs to be adjusted, and I won’t introduce it here; another one must set curl’s CURLOPT_TIMEOUT (I use CURL to capture remote images). Considering that the images are on multiple known servers, there must be a problem if they cannot be captured after a timeout. Simply give up, if you don't set this timeout, the runtime will keep reporting errors to you.

The next step is the server. The first is the nginx program execution time:

fastcgi_connect_timeout 900s;

fastcgi_send_timeout 900s;

fastcgi_read_timeout 900s;

This configuration is in nginx.conf. If necessary, modify the value but not append it. Set the time according to your own situation and restart the service after adjustment.

MYSQL will also report a timeout error. Modify my.ini to add or modify the configuration:

wait_timeout=2880000

interactive_timeout=2880000

max_allowed_packet=100M

After the adjustment, restart mysql to take effect. After the above settings, various situations of timeout errors have been basically eliminated. My program ran for 2 days and 3 nights, collecting and generating a total of 86G images.

I would like to emphasize again that this operation is not very recommended, especially in a production environment, it will cause many problems. The most important thing is to restore the above settings after completing the batch task~

Related recommendations ：

Summary of issues related to image processing in WeChat mini programs

(Advanced) PHP common image processing classes

Introduction to the usage of phpThumb parameters of PHP image processing class

The above is the detailed content of Program execution times out when performing batch image processing in PHP. For more information, please follow other related articles on the PHP Chinese website!