PHP message queue implementation and application

As we all know, when designing a website, you will encounter "mass text messages" to users, "a large number of logs in the order system", "flash sale design", etc. The server cannot handle this instantaneous burst of pressure. In this situation To ensure the normal and effective use of the system, the help of "message queue" is needed. This article mainly studies through the idea of ​​message queue.

Mainly understand the following knowledge:

1. What is a queue and what can it do?

　2. What are the application scenarios for alignment?

　3. How to use queues to uncouple services?

　4. How to use Redis queue to eliminate high pressure?

　5. How to use the professional alignment system RabbitMQ?

The main content is summarized as follows

　@The concept, principle and scenario of message queue

　@Decoupling case: queue processing order system and distribution system

　@ Traffic peak-cutting case: Redis’s List type realizes flash sale

@RabbitMQ: A more professional message system implementation solution

1. Understanding the message queue

1.1 Message queue concept

Essentially, message queue is a middleware with a queue structure. That is to say, the message can be returned directly after being put into this middleware, and does not need to be processed immediately by the system. In addition, There will be a program that reads the data and processes it one by one in sequence.

That is to say, when you encounter a situation where concurrency is particularly large and takes a long time, and the processing results do not need to be returned immediately, using message queues can solve such problems.

1.2 Core structure

Enqueues messages from a business system, inserts messages into the message queue one by one, and directly returns a successful result after the insertion is successful. There will be a message processing system in the future, which will take out and process the records in the message system one by one, completing a dequeuing process.

1.3 Application Scenario

Data redundancy: For example, in the order system, strict data conversion and recording are required in the future. The message queue can store these data persistently in the queue, and then there are orders , the subsequent processing program obtains it, and after the subsequent processing is completed, this record is deleted to ensure that each record can be processed.

System decoupling: After using the messaging system, the enqueue system and the dequeue system are separated, which means that as long as it crashes one day, it will not affect the normal operation of the other system.

Traffic peak shaving: For example, flash sales and rush sales, we can use message queues in conjunction with caching, which can effectively withstand the amount of instantaneous visits and prevent the server from being overwhelmed and causing a crash.

Asynchronous communication: The message itself can be returned directly after being queued.

Scalability: For example, the order queue can not only process orders, but can also be used by other businesses.

Sorting guarantee: Some scenarios need to be processed in the order of products, such as single in and single out to ensure that data is processed in a certain order. It is possible to use message queues.

The above are common usage scenarios of message queue. Of course, message queue is just a middleware and can be used in conjunction with other products.

1.4 Common queue implementation advantages and disadvantages

Queue medium

1. Database, such as mysql (high reliability, easy to implement, slow speed)

2 , cache, such as redis (fast, low efficiency when a single message package is too large)

3. Messaging system, such as rabbitMq (highly professional, reliable, high learning cost)

Message Processing trigger mechanism

　1. Infinite loop reading: easy to implement, unable to recover in time in case of failure; (more suitable for flash sale, more centralized, centralized operation and maintenance)

　2. Scheduled tasks : Pressure is evenly distributed, with a processing upper limit; currently a popular processing trigger mechanism. (The only disadvantage is that you need to pay attention to the interval and data. Don't wait until the previous task is not completed and the next task starts again)

　3. Daemon process: similar to php-fpm and php-cg, requires shell basics

2. Decoupling Case: Queue Processing "Order System" and "Distribution System"

Let’s briefly talk about program decoupling: Program decoupling is to avoid the problem of who should be rescued first when your wife and your mother fall into the water at the same time (smirking)

For orders For the process, we can design two systems, one is the "order system" and the other is the "delivery system". We should all have seen it when shopping online. After I submit an order, I can see my goods in the background. In progress. At this time, a "delivery system" needs to be involved.

If we design the "order system" and "delivery system" together when doing the architecture, there will be some problems. First of all, for the order system, the pressure on the system will be relatively high, but " The distribution system" does not necessarily have to respond immediately to these pressures.

Secondly, we do not want the failure of the order system to cause a failure of the distribution system, which will affect the normal operation of both systems at the same time. So we hope to decouple these two systems. After the two systems are separated, we can communicate between the two systems through an intermediate "queue table".

2.1 Architecture design

## 1. First, the order system will receive the user’s order, and then process the order.

　2. These order information will then be written to the queue table. This queue table is the key to communicating between the two systems.

　3. A program executed regularly by the distribution system reads the queue table for processing.

　4. After processing by the distribution system, the processed records will be marked.

2.2 Program flow

#3. Traffic peak clipping case: Redis’s list type realizes flash sales

　redis Based on memory, its speed will be very fast. Redis is a very good supplement to the database because it is durable. Redis will periodically write data to the hard disk, so it does not have to worry about power outages. In this respect, it has more advantages than another cache memcache. In addition, redis provides five data types (string, doubly linked list, hash, set, ordered set)

In general, do flash sales Redis is a good choice for cases where cases, rush purchases, and cases that require queuing are instantly higher than yours.

3.1 List type in redis data type

The list in redis is a doubly linked list, and data can be appended from the head or tail.

* LPUSH/LPUSHX: Insert the value into the head of the (/existing) list

* RPUSH/RPUSHX: Insert the value into the tail of the (/existing) list

* LPOP: Remove and get the first element of the list

* RPOP: Remove and get the last element of the list

* LTRIM: Keep the elements in the specified range

　* LLEN: Get the length of the list

　* LSET: Set the value of the list element by index

　* LINDEX: Get the element in the list by index

　* LRANGE: Get the elements within the specified range of the list

3.2 Architecture design

A simple structure flash kill program design.

　1. First record which user participated in the flash sale and record his time.

　2. Save the user's ID in the redis list and let it queue up. If it is stipulated that only the first 10 users can successfully participate, if the number in the list is enough, it will not be allowed to continue to add data. In this way, the length of the redis list will only be 10

　3. Finally, slowly write the data in redis into the database to reduce the pressure on the data

3.3 Code-level design

　1. When the user starts the flash sale, write the request of the flash sale program into Redis (uid, time_stamp).

　2. If it is stipulated that only 10 people can succeed in the flash sale, check the length of the data stored in Redis. If it exceeds the upper limit and discard it directly, it means that the flash sale is completed.

　3. Finally, the 10 pieces of data stored in Redis are processed in an infinite loop, and then the data is slowly fetched and stored in the mysql database.

In the flash sale area, the pressure on the database is particularly high. If we do not have such a design, it will cause a writing bottleneck in MySQL. We use a queue list in Redis, and then put the flash sale request into Redis. Finally, we slowly write the data into the database through the warehousing program. In this way, the traffic can be balanced and there will be no impact on mysql. Too much pressure.

4. RabbitMQ

Here we will explain some uses of RabbitMQ. First of all, when we talked about the flash sale case before, we mentioned the lock mechanism to prevent other programs from processing the same record. If our system architecture is very complex, there are multiple programs reading a queue in real time, or I have multiple sending programs that operate one or more queues at the same time, and I even want these programs to be distributed on different machines. In this case, using redis queue is somewhat inadequate. What to do at this time? We need to introduce some more professional message queue systems, which can better solve the problem.

4.1 The architecture and principles of RabbitMQ

　

Features: Complete implementation of AMQP, cluster simplification, persistence, cross-platform

Use of RabbitMQS

1. RabbitMQ installation (rabbitmq-server, php-amqplib)

2. Producer sends message to message channel

3. Consumer Processor message

Work queue

　　　

　　 Idea: The producer sends it to the message system, and the message system encapsulates the task into a message queue, and then uses the same queue for multiple consumers

　This is not only It solves the decoupling between producers and consumers, and can also realize the sharing of consumers and tasks, reducing the pressure on the server.

5. Summary

The above mainly focuses on learning the concepts, principles, and scenarios of message queues. Decoupling cases and peak clipping cases, as well as understanding the simple use of RabbitMQ.

6. Question

What is the biggest difference between redis and message server selection.

My understanding is that Redis processes requests one by one. Redis is a single thread. It is different from the message server IO implementation. One is synchronous and the other is asynchronous. Redis uses synchronous blocking, while the message server Use asynchronous non-blocking.

Advantages and disadvantages of common queue implementations

Queue medium:

Mysql: high reliability, easy to implement, slow speed
Redis: fast speed, single large message package Low time efficiency
Message system: highly professional, reliable, high learning cost (for example: RabbtiMQ)

Trigger mechanism of message processing:

Infinite loop reading: easy to implement, Unable to recover in time when a fault occurs;
Scheduled tasks: pressure is evenly distributed, and there is an upper limit on processing capacity. (The biggest flaw: The time interval of positioning tasks and the data processed need to be accurately grasped. The next task cannot be considered to have been started before the previous task is completed.)

Daemon process: similar to PHP-FPM and PHP-CGI, shell knowledge is required

Related recommendations:

php implementation of message queue class instance sharing

The above is the detailed content of PHP message queue implementation and application. For more information, please follow other related articles on the PHP Chinese website!