Distributed transactions: reliable message eventual consistency scheme

Affairs must be familiar to everyone, such as ACID that is often mentioned, but for the subsequent distribution Regarding the content of distributed transactions, let’s first talk aboutACID, then introduce what distributed transactions is, and finally focus on reliable messages distributed transaction solutions.

What is a transaction

A transaction in the strict sense should have Atomicity, consistency, isolation and durability, referred to as ACID.

1. Atomicity (Atomicity) can be understood as all operations within a transaction are either executed or not executed.
2. Consistency (Consistency) can be understood as the data satisfies the integrity constraint, that is, there will be no data in an intermediate state. For example, your wallet has 100 , my wallet has 100, and you give me 50. At this time, the money in your wallet should be 50, and the money in my wallet should be 150. There will be no intermediate state where my money is added and your money is not deducted.
3. Isolation refers to the fact that multiple transactions will not interfere with each other when executed concurrently, that is, the data within one transaction will not affect other transactions. is isolated.
4. Durability (Durability) refers to the fact that after a transaction is completed, the data is saved forever, and other subsequent operations or failures will not affect the transaction. have an impact on the results.

In a popular sense, a transaction is to make some update operations either succeed or fail.

What is a distributed transaction

Distributed transaction As the name suggests, it is to implement transactions in a distributed system. It is actually composed of multiple local transactions.

A large operation is composed of different small operations. These small operations are distributed on different servers. Distributed transactionsIt is necessary to ensure that these small operations either all succeed or all fail. In essence, distributed transactions are to ensure data consistency of different databases.

Common distributed transaction solutions include the following: 2PC, 3PC, TCC, local message table, reliable message eventual consistency, best-effort notification , etc.

Today we will focus on The solution for reliable message eventual consistency

What is reliable message eventual consistency Scheme

Reliable message eventual consistency scheme means that when the transaction initiator completes the local transaction, it sends a message to the message middleware, Transaction participants (message consumers) must be able to receive the message and process the transaction successfully. This solution emphasizes that As long as the message is sent to the transaction participant, the final transaction must be consistent.

What are the problems with this method?

This solution is implemented through message middleware. The transaction initiator (message producer) sends the message to the message middleware, and the transaction participants receive the message from the message middleware. Due to the network The uncertainty of communication will lead to distributed transaction problems, as shown below:

1. The atomicity problem of local transactions and messages

As shown in the dotted box above, there are the following situations:

1) The local transaction fails to submit, and the message does not send.
2) The local transaction is successful, the message sending fails, and the local transaction is rolled back.
3) The local message is successful, the message times out, the local transaction is rolled back, and the message ultimately fails.
4) The local message is successful, the message times out, the local transaction is rolled back, and the message is finally successful.

To sum up, there is a fourth situation, which causes local transactions to be inconsistent with the transactions of the message participants.

2. The reliability of messages received by transaction participants.
   

The message middleware and transaction participants must ensure that they can successfully consume the message.

3. Repeated consumption of messages
   

Pay attention to the interface idempotence problem of transaction participants, message participants It may have been consumed successfully, but due to network problems, the message middleware believed that the message was not consumed and initiated a retry.

Solution

1. Local message table
   

Local message table The key is that there is a local record table that stores message logs. You need to start a scheduled task to continuously scan the message log records to ensure that the message can be sent. The specific process is as follows:

The process in the above picture:

1) The local transaction of the transaction initiator is successfully executed, and the local message table Record messages in the log.
2) Start a scheduled task and scan the local message table cyclically.
3) The scheduled task scans the message and sends the message to the message middleware.
4) The message middleware receives the message and returns a successful message to notify the transaction initiator.
5) The transaction initiator deletes the log message after receiving the message and sending it successfully.
6) Transaction participants subscribe to messages and consume messages.
7) Transaction participants handle local transactions.
8) The local transaction is processed successfully and a successful ack is sent to the message middleware.

Points to note:
Transaction participants ensure interface idempotence.

2. RocketMq transaction message scheme
   

Apache RocketMQ Versions after 4.3 officially support transaction messages , provides convenient support for distributed transaction implementation. After RocketMQ 4.3, complete transaction messages were implemented. In fact, it is actually an encapsulation of the local message table. The local message table was moved to MQ internal to solve the problem of message sending and processing on the Producer side. The atomicity issue of local transaction execution.

Implementation process:

1) The transaction initiator sends Half transaction message
2) RocketMq replies Halfsent successfully
3) The transaction initiator executes the local transaction
4) The transaction initiator successfully executes the local transaction, sends commit to RocketMq, and mq delivers the message to the transaction participant; the transaction initiator fails to execute the local transaction and sends rollbackGo to RocketMq, mq delete message.
5) When RocketMq does not receive confirmation information from the transaction initiator within a certain period of time, it will transaction review on the transaction initiator.
6) The transaction initiator queries the local transaction status.
7) The transaction initiator sends commint/rollback to RocketMq based on the queried transaction status.
8) When RocketMq initiates commit and receives a failure or does not receive a successful ack within a certain period of time, it will initiate a retry.

Advantages:

Message data is stored independently, reducing the coupling between the business system and the message system.
The throughput is better than the local message table solution.

Disadvantages:

One message sending requires two network requests (half message commit/rollback).
Need to implement the message review interface.

In fact, each distributed transaction solution has advantages and disadvantages. We need to weigh the pros and cons and choose the one that is most suitable for the business scenario.

The above is the detailed content of Distributed transactions: reliable message eventual consistency scheme. For more information, please follow other related articles on the PHP Chinese website!