Table of Contents
Articles you may be interested in:
Home Backend Development PHP Tutorial Source code analysis explains why Laravel repeatedly executes the same queue task

Source code analysis explains why Laravel repeatedly executes the same queue task

Jul 04, 2018 pm 05:58 PM
laravel task queue

Laravel's queue service provides a unified API for various background queue services. The following article introduces to you through source code analysis the reasons why Laravel repeatedly executes the same queue task. The article introduces it through sample code. For details, friends in need can refer to it. Let’s take a look below.

Preface

Laravel’s queue service provides a unified API for various background queue services. Queues allow you to delay the execution of time-consuming tasks, such as sending an email. This can effectively reduce the request response time.

Found the problem

Use Redis in Laravel to process queue tasks. The functions provided by the framework are very powerful, but recently I encountered a problem, that is It is found that a task is executed multiple times. Why is this?

Let’s talk about the reason first:

Because in Laravel, if a queue (task) execution time is greater than After 60 seconds, it will be considered an execution failure and rejoined in the queue, which will cause the same task to be executed repeatedly.

The logic of this task is to push content to users. Users need to be retrieved and traversed based on the queue content, and sent through the request backend HTTP interface. For example, if there are 10,000 users, if the number of users is large or the interface processing speed is not that fast, the execution time will definitely be greater than 60 seconds, so the task will be re-added to the queue. The situation is even worse. If the previous tasks are not executed within 60 seconds, they will be re-added to the queue, so that the same task will be executed not only once, but multiple times.

Let’s find the culprit from the Laravel source code.

Source code file: vendor/laravel/framework/src/Illuminate/Queue/RedisQueue.php

/**
 * The expiration time of a job.
 *
 * @var int|null
 */
protected $expire = 60;
Copy after login

The $expire member variable is a fixed value. Laravel believes that a queue should be executed no matter how long it takes 60 seconds. Method to get the queue:

public function pop($queue = null)
{
 $original = $queue ?: $this->default; 
 $queue = $this->getQueue($queue); 
 $this->migrateExpiredJobs($queue.':delayed', $queue); 
 if (! is_null($this->expire)) {
  $this->migrateExpiredJobs($queue.':reserved', $queue);
 } 
 list($job, $reserved) = $this->getConnection()->eval(
  LuaScripts::pop(), 2, $queue, $queue.':reserved', $this->getTime() + $this->expire
 ); 
 if ($reserved) {
  return new RedisJob($this->container, $this, $job, $reserved, $original);
 }
}
Copy after login

There are several steps to get the queue, because if the queue fails to execute or the execution times out, it will be put into another collection and saved. , in order to retry, the process is as follows:

1. Re-rpush the queue that failed due to execution from the delayed collection to the currently executed queue.

2. Re-rpush the queue due to execution timeout from the reserved collection to the currently executed queue.

3. Then take the task from the queue and start executing it, and put the queue into the reserved ordered collection.

The eval command is used here to execute this process, and several lua scripts are used.

Get the task from the queue to be executed:

local job = redis.call('lpop', KEYS[1])
local reserved = false
if(job ~= false) then
 reserved = cjson.decode(job)
 reserved['attempts'] = reserved['attempts'] + 1
 reserved = cjson.encode(reserved)
 redis.call('zadd', KEYS[2], ARGV[1], reserved)
end
return {job, reserved}
Copy after login

You can see that Laravel is getting the task to be executed by Redis When queuing, a copy will be put into an ordered collection at the same time, and the expiration timestamp will be used as the score.

Only when the task is completed, the task will be removed from the ordered set. The code to remove the queue from this ordered collection is omitted. Let's take a look at how Laravel handles queues whose execution time is greater than 60 seconds.

This is the operation performed by this lua script:

local val = redis.call('zrangebyscore', KEYS[1], '-inf', ARGV[1])
if(next(val) ~= nil) then
 redis.call('zremrangebyrank', KEYS[1], 0, #val - 1)
 for i = 1, #val, 100 do
  redis.call('rpush', KEYS[2], unpack(val, i, math.min(i+99, #val)))
 end
end
return true
Copy after login

Here zrangebyscore finds the score from infinitesimal To the elements of the current timestamp, that is, the tasks added to the collection 60 seconds ago, these elements are then removed from the collection through zremrangebyrank and rpush to the queue.

You should suddenly realize it when you see this.

If a queue has not been executed within 60 seconds, the process will rpush the tasks from the reserved set to the queue again when fetching the queue.

Summary

Articles you may be interested in:

About Laravel Redis Detailed explanation of the problem of multiple processes taking queues at the same time

Detailed explanation of php-msf source code

thinkphp5 URL and routing Detailed explanation of functions and examples

The above is the detailed content of Source code analysis explains why Laravel repeatedly executes the same queue task. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1659
14
PHP Tutorial
1257
29
C# Tutorial
1231
24
Laravel Eloquent ORM in Bangla partial model search) Laravel Eloquent ORM in Bangla partial model search) Apr 08, 2025 pm 02:06 PM

LaravelEloquent Model Retrieval: Easily obtaining database data EloquentORM provides a concise and easy-to-understand way to operate the database. This article will introduce various Eloquent model search techniques in detail to help you obtain data from the database efficiently. 1. Get all records. Use the all() method to get all records in the database table: useApp\Models\Post;$posts=Post::all(); This will return a collection. You can access data using foreach loop or other collection methods: foreach($postsas$post){echo$post->

Laravel Introduction Example Laravel Introduction Example Apr 18, 2025 pm 12:45 PM

Laravel is a PHP framework for easy building of web applications. It provides a range of powerful features including: Installation: Install the Laravel CLI globally with Composer and create applications in the project directory. Routing: Define the relationship between the URL and the handler in routes/web.php. View: Create a view in resources/views to render the application's interface. Database Integration: Provides out-of-the-box integration with databases such as MySQL and uses migration to create and modify tables. Model and Controller: The model represents the database entity and the controller processes HTTP requests.

Solve caching issues in Craft CMS: Using wiejeben/craft-laravel-mix plug-in Solve caching issues in Craft CMS: Using wiejeben/craft-laravel-mix plug-in Apr 18, 2025 am 09:24 AM

When developing websites using CraftCMS, you often encounter resource file caching problems, especially when you frequently update CSS and JavaScript files, old versions of files may still be cached by the browser, causing users to not see the latest changes in time. This problem not only affects the user experience, but also increases the difficulty of development and debugging. Recently, I encountered similar troubles in my project, and after some exploration, I found the plugin wiejeben/craft-laravel-mix, which perfectly solved my caching problem.

Laravel user login function Laravel user login function Apr 18, 2025 pm 12:48 PM

Laravel provides a comprehensive Auth framework for implementing user login functions, including: Defining user models (Eloquent model), creating login forms (Blade template engine), writing login controllers (inheriting Auth\LoginController), verifying login requests (Auth::attempt) Redirecting after login is successful (redirect) considering security factors: hash passwords, anti-CSRF protection, rate limiting and security headers. In addition, the Auth framework also provides functions such as resetting passwords, registering and verifying emails. For details, please refer to the Laravel documentation: https://laravel.com/doc

Laravel's geospatial: Optimization of interactive maps and large amounts of data Laravel's geospatial: Optimization of interactive maps and large amounts of data Apr 08, 2025 pm 12:24 PM

Efficiently process 7 million records and create interactive maps with geospatial technology. This article explores how to efficiently process over 7 million records using Laravel and MySQL and convert them into interactive map visualizations. Initial challenge project requirements: Extract valuable insights using 7 million records in MySQL database. Many people first consider programming languages, but ignore the database itself: Can it meet the needs? Is data migration or structural adjustment required? Can MySQL withstand such a large data load? Preliminary analysis: Key filters and properties need to be identified. After analysis, it was found that only a few attributes were related to the solution. We verified the feasibility of the filter and set some restrictions to optimize the search. Map search based on city

Laravel framework installation method Laravel framework installation method Apr 18, 2025 pm 12:54 PM

Article summary: This article provides detailed step-by-step instructions to guide readers on how to easily install the Laravel framework. Laravel is a powerful PHP framework that speeds up the development process of web applications. This tutorial covers the installation process from system requirements to configuring databases and setting up routing. By following these steps, readers can quickly and efficiently lay a solid foundation for their Laravel project.

Laravel and the Backend: Powering Web Application Logic Laravel and the Backend: Powering Web Application Logic Apr 11, 2025 am 11:29 AM

How does Laravel play a role in backend logic? It simplifies and enhances backend development through routing systems, EloquentORM, authentication and authorization, event and listeners, and performance optimization. 1. The routing system allows the definition of URL structure and request processing logic. 2.EloquentORM simplifies database interaction. 3. The authentication and authorization system is convenient for user management. 4. The event and listener implement loosely coupled code structure. 5. Performance optimization improves application efficiency through caching and queueing.

How to learn Laravel How to learn Laravel for free How to learn Laravel How to learn Laravel for free Apr 18, 2025 pm 12:51 PM

Want to learn the Laravel framework, but suffer from no resources or economic pressure? This article provides you with free learning of Laravel, teaching you how to use resources such as online platforms, documents and community forums to lay a solid foundation for your PHP development journey from getting started to master.

See all articles