How to resolve concurrency conflicts in crond script execution?

In planned tasks, you will occasionally see repeated executions:

For example, our company’s planned tasks:

*/2 * * * * root cd /opt/xxxx/test_S1/html/xxxx/admin; php index.php task testOne >/dev/null 2>&1*/2 * * * * root cd /opt/xxxx/test_S1/html/xxxx/admin; php index.php task testTwo >/dev/null 2>&1

This is executed every two minutes There is no guarantee that each started process can be completely executed and closed within two minutes. If the processes continue to accumulate, system resources may be exhausted, causing the system to crash.

Example:

Create a new test.php file with the following code:

<?php
sleep(70);?>

Add a scheduled task:

*/1 * * * * root cd /home/ganjincheng;php test.php

Waiting for execution, accumulation occurs

root     26722  0.0  0.0   9232  1064 ?        Ss   12:05   0:00 /bin/sh -c cd /home/ganjincheng;php test.php
root     26744  0.0  0.0 112304  8840 ?        S    12:05   0:00 php test.php
root     29102  0.0  0.0   9232  1060 ?        Ss   12:06   0:00 /bin/sh -c cd /home/ganjincheng;php test.php
root     29116  0.1  0.0 112304  8840 ?        S    12:06   0:00 php test.php
root     29906  0.0  0.0 103320   904 pts/3    S+   12:06   0:00 grep test.php

Solution

The first is to control concurrency in the code

This method is to control the code Make a transformation. Add the judgment of whether there is process execution. For example, the following code:

<?php  
$lockfile = &#39;/tmp/mytest.lock&#39;;  
   
if(file_exists($lockfile)){  
    exit();  
}
file_put_contents($lockfile, date("Y-m-d H:i:s"));
   
sleep(70);
 
unlink($lockfile);  
?>

There is a problem with this way of determining whether a file does not exist. That is, it is possible that the program has not been executed to the end, that is, the previously created mytest.lock file has not been deleted. This will result in the program not being able to execute properly in the future.

Second, database control concurrency

The first solution can be transferred to redis and memache for key value judgment.

If the planned task is to access the database, you can perform table lock operations. Occasionally we can also use the uniqueness of the unique index and the joint index to avoid repeated insertions

The third way is to determine whether the process exists

Example:

$fp = popen("ps aux | grep 'test.php' | wc -l", "r");
$proc_num = fgets($fp);if ($proc_num > 3) { //这里要注意为什么进程数要大于3，实际操作一遍你就明白了exit;
}
sleep(70);

One drawback of this method is that the ps command must be written accurately. Avoid counting processes that are not executing the test.php script. For example:
We open the test.php file through vim. This will cause the above command to incorrectly count. So when we accidentally open the test.php file in vim, it cannot be executed.

The fourth method is to use the flock command of Linux

Let Linux help us judge. The flock command provides the file lock function. The command parameters are as follows:

[root@qkzj_Multi-Purpose_1A_113.107.248.124 ganjincheng]# flock -h
flock (util-linux-ng 2.17.2)
Usage: flock [-sxun][-w #] fd#
       flock [-sxon][-w #] file [-c] command...
       flock [-sxon][-w #] directory [-c] command...  -s  --shared     Get a shared lock
  -x  --exclusive  Get an exclusive lock
  -u  --unlock     Remove a lock
  -n  --nonblock   Fail rather than wait  -w  --timeout    Wait for a limited amount of time  -o  --close      Close file descriptor before running command  -c  --command    Run a single command string through the shell  -h  --help       Display this text  -V  --version    Display version

Configuration example:

*/1 * * * * root flock -xn /tmp/mytest.lock -c 'php ./test.php'

The above is the detailed content of How to resolve concurrency conflicts in crond script execution?. For more information, please follow other related articles on the PHP Chinese website!