Build a highly available enterprise-level scheduled task scheduling system based on Swoole

With the continuous development of the Internet industry and the continuous advancement of technology, scheduled task scheduling systems are becoming more and more important in various large-scale enterprise-level application scenarios. Enterprises need a highly available and easily scalable scheduled task scheduling system to regularly handle daily business processes, such as data backup, email sending, regular statistics, etc., to ensure the stability and reliability of the system. This article will introduce how to build a highly available enterprise-level scheduled task scheduling system based on the Swoole framework.

Swoole is a coroutine network communication engine based on the PHP language, which can make PHP programs have the same high concurrency and high performance features as Node.js. Swoole provides rich network communication and asynchronous IO functions, which can provide powerful support for enterprise-level applications. Below we will introduce in detail how to use Swoole to build a highly available enterprise-level scheduled task scheduling system.

1. Design Ideas

When designing a scheduled task scheduling system, we need to consider the following aspects:

1. Task management: Responsible for managing and scheduling all tasks Tasks include task creation, task modification, task deletion, task running status management, etc.

2. Task execution: Responsible for specific task execution, including calling specified business logic code, recording task execution logs, handling task exceptions, etc.

3. Task scheduling: Responsible for allocating tasks to corresponding executors according to predetermined time intervals and rules.

4. Task monitoring: Responsible for monitoring the running status of all tasks, discovering and handling abnormal problems in a timely manner, and ensuring the stability and reliability of the system.

Based on the above ideas, we can divide the entire system into the following layers:

Task scheduling layer: responsible for the scheduling and distribution of tasks, and assigning tasks to the corresponding executors.

Message queue layer: used to store task information and execution results to improve system processing capabilities and stability.

Execution layer: The specific task executor is responsible for executing the specified task and writing the results to the message queue.

Monitoring layer: Monitor the running status of the entire system and detect and handle abnormalities in a timely manner.

2. Technical Architecture

1. Task Scheduling

Task scheduling is the core part of the entire system, and tasks need to be scheduled and allocated according to predetermined rules and time intervals. We can use Swoole's timers and coroutines to implement task scheduling functions. First, we need to start a Swoole process to execute scheduled task scheduling logic:

$scheduler = new Scheduler();
$scheduler->add(function () use ($taskManager) {

$taskManager->assignTask();

}, '', SWOOLE_TIMER_INTERVAL * 1000);

Among them, $taskManager is the task management object. In its assignTask() function, we can start from Select the appropriate task from the task list and assign it to the corresponding executor:

public function assignTask()
{

$now = time();
foreach ($this->tasks as $task) {
    if ($task->nextExecTime == 0) {
        $task->nextExecTime = strtotime($task->cron);
    }
    if ($task->nextExecTime <= $now) {
        $task->nextExecTime = strtotime($task->cron, $now);
        $this->executeTask($task);
    }
}

}

In executeTask( ) function, we can put the task information into the message queue and wait for the executor to process:

public function executeTask($task)
{

// 将任务信息放入消息队列中
$this->queue->push($task);

}

2. Task execution

Task execution is another core part of the entire system. It needs to call the corresponding business logic code based on the task information and write the execution results into the message queue. Since exceptions may occur during task execution, it is necessary to handle exceptions during execution and record execution logs. We can use Swoole's coroutines and asynchronous IO functions to achieve high-performance task execution functions. First, we need to start several Swoole sub-processes as task executors:

for ($i = 0; $i < SWOOLE_PROCESS_NUM; $i ) {

$worker = new Worker();
$worker->onWorkerStart = function ($worker) use ($queue) {
    while (true) {
        // 从消息队列中获取任务信息
        $task = $queue->pop();
        if (!$task) continue;
        // 执行任务
        $result = $this->execute($task);
        // 将执行结果写入消息队列中
        $this->queue->push($result);
    }
};
$worker->listen();

}

In the execute() function, we can call the corresponding business logic code based on the task information, and perform exception handling and logging:

public function execute($task)
{

// 调用业务逻辑代码
try {
    $result = $this->doTask($task);
    return $result;
} catch (Exception $e) {
    // 异常处理
    $errMsg = sprintf("Task failed: %s, error message: %s", $task->name, $e->getMessage());
    $this->log($errMsg);
    return false;
}

}

3. Message Queue

The message queue is the communication hub of the entire system, used to store task information and execution results, and improve system processing capabilities and stability. We can use the coroutine and asynchronous IO functions provided by Swoole to implement high-performance message queue functions. First, we need to start a Swoole process as a message queue:

$queue = new Channel();
$server = new Server('0.0.0.0', 9501, SWOOLE_PROCESS, SWOOLE_SOCK_TCP);
$server->on('receive', function ($server, $fd, $from_id, $data) use ($queue) {

// 将消息放入消息队列中
$queue->push($data);

});
$server-> start();

During the specific task execution process, we can write the task information and execution results into the message queue and wait for other components to process:

// Put the task information Enter the message queue
$this->queue->push($task);

4. Monitoring system

The monitoring system is an integral part of the entire system. Use It is used to monitor the running status of the entire system, detect and handle abnormal problems in a timely manner, and ensure the stability and reliability of the system. We can use Swoole's process management and signal processing functions to implement the functions of the monitoring system. We can start a Swoole process as a monitoring process:

$monitor = new Monitor();
$monitor->start();

In the start() function of the Monitor class , we can use Swoole's process management and signal processing functions to implement the functions of the monitoring system:

public function start()
{

// 注册信号处理函数
pcntl_signal(SIGUSR1, array($this, 'handleSignal'));
while (true) {
    $cpuUsage = $this->getCpuUsage();
    $memUsage = $this->getMemUsage();
    $this->log(sprintf('CPU usage: %.2f%%, Memory usage: %.2fMB', $cpuUsage, $memUsage));
    sleep(MONITOR_INTERVAL);
}

}

其中，getCpuUsage()函数用于获取当前进程的CPU使用率，getMemUsage()函数用于获取当前进程的内存使用情况，handleSignal()函数用于处理信号并进行相应的处理。

三、系统部署

在系统部署方面，我们可以使用Docker容器化的方式，来实现系统的快速部署和迁移。首先，我们需要构建一组Docker镜像：

docker build -t task-scheduler:latest .
docker build -t task-executor:latest .
docker build -t task-queue:latest .
docker build -t task-monitor:latest .

其中，task-scheduler镜像用于运行任务调度进程，task-executor镜像用于运行任务执行进程，task-queue镜像用于运行消息队列进程，task-monitor镜像用于运行监控进程。

接着，我们可以使用docker-compose来启动和管理整个系统：

version: '3'
services:
scheduler:

image: task-scheduler:latest
restart: always

executor:

image: task-executor:latest
restart: always
scale: 5

queue:

image: task-queue:latest
restart: always

monitor:

image: task-monitor:latest
restart: always

其中，scheduler服务用于启动任务调度进程，executor服务用于启动任务执行进程，queue服务用于启动消息队列进程，monitor服务用于启动监控进程。可以根据实际情况，调整服务的数量和启动参数。

四、总结

本文介绍了如何基于Swoole框架构建一套高可用的企业级定时任务调度系统，其中涵盖了任务调度、任务执行、消息队列和监控等方面。Swoole的高性能和异步IO特性，为企业级应用提供了强大的支持，能够满足各种大规模应用的需求。通过本文的介绍，相信读者可以更好地了解Swoole框架的应用和实践。

The above is the detailed content of Build a highly available enterprise-level scheduled task scheduling system based on Swoole. For more information, please follow other related articles on the PHP Chinese website!