Consistent Backends and UX: What are the Barriers to Adoption?

Series of articles

1. Reasons you should pay attention to
2. Probable problems
3. Obstacles to adoption
4. How the new algorithm can help

It is very rare that ultimately consistent databases are better than strong consistency databases. In addition, in multi-region application scenarios that need to be expanded, it is even more doubtful to choose a non-distributed database or a final consistent database. So, what prompts engineers to ignore strongly consistent distributed databases? We have seen many reasons, but the wrong assumption is the driver.

"CAP theorem shows that this is impossible"

As we explain in the first part of this series, the CAP theorem is widely accepted but often misunderstood. When many people misunderstand a famous theorem, it leaves its mark. In this case, many engineers still believe that ultimate consistency is a necessary evil.

"Building a strongly consistent distributed database is too difficult/impossible"

It has gradually been recognized that consistency should not be sacrificed, but many databases still put consistency second. Why is this? Some popular databases offer options for providing higher consistency, but at the cost of very high latency. Their sales information may even claim that providing consistency with low latency in multi-region distributed databases is difficult or even impossible, while the developer community has a deep memory of experiencing very bad latency in non-consistent databases. The combination of the two together consolidates the misunderstanding that strong consistency in distributed databases with relatively low latency is impossible.

"Premature optimization is the source of all evil"

Many engineers build on the principle that “premature optimization is the source of all evil” (Donald Knuth), but this statement only applies to small inefficiencies. Building your startup on a strongly consistent distributed scalable database seems like a premature optimization because initially, your application does not need to be scaled, and probably does not need to be distributed. However, what we are talking about here is not a small inefficiency. When your application becomes popular, scalable or distributed requirements may appear overnight. At that time, your users will experience a bad experience and you will face a huge challenge to change your infrastructure and code.

"It's hard to program a distributed database"

Since distributed databases are new and many databases have serious restrictions, this sentence once made some sense. They do not allow connections, only key-value storage, or require you to query data based on predefined shard keys that you cannot change later. Today, we have distributed databases with flexible models and the flexibility you are used to using traditional databases. This is closely related to the previous point, which ignores the fact that it is equally easy to start programming for a strongly consistent distributed database today than traditional databases, and may be easier in the long run. If it's equally easy, why not optimize from the beginning?

Using a final consistency database is like...

Distributed databases are often created by those who have experienced the ultimate consistency problem. For example, FaunaDB was built by a former Twitter engineer after experiencing how difficult it was to build a scalable system on top of a popular final consistency database like Cassandra. These problems often appear when new companies start to expand, so many young engineers have never experienced them in person.

Sometimes, painful things can teach us lessons that we don’t think we need to know.

— Amy Poehler

Discussing the danger of ultimate consistency often leads engineers who have not yet encountered any problems to make the argument that “this works for me”. Since this usually takes months (or years, if you're lucky), let's look at an analogy.

…Riding a bicycle with loose wheels.

Not long ago, my best friend almost missed the date, so I lent him my bike. I'm glad I helped, he's glad, everything goes well. The pleasure quickly turned into pain when he tried to jump the bike to the sidewalk. You see… I fiddled with the bike earlier that day and forgot to tighten the front wheels. He returned with a huge purple bruise.

The bicycle example is very similar to using a non-strong consistency database. Everything will go well before trying to lift the wheels of the bike (or in other words until your company takes off and starts expanding).

When your application needs to be extended, you usually implement it by copying the service. Once the database becomes a bottleneck, you will replicate your traditional database or migrate to a distributed database. Sadly, at that time, when you start copying the database, the functionality in your application may be interrupted. So far, you haven't noticed these issues because the database runs on a node. At that time, two things might happen:

• Case 1, Build/Fix it around it: Developers quickly realize that the database they are "riding" is unreliable for the functionality they have built or are trying to build. Their choice boils down to canceling features, simplifying features, or changing the database.
• Situation 2, Fall: The developer didn't get a good informing from the vendor (I'm a bad bike vendor for my friends) about the risks, there's now a lack of information to understand the very subtle meaning of what's going on. This is not necessarily due to the lack of engineers’ abilities. Lazy-defined standards and optimistic marketing do a great job of confusing the consistency guarantees of different databases.

Developers who end up in the first case are often experienced in handling the final consistency system. They now either accept that they can't provide certain functionality or they will build a complex and difficult-to-maintain layer on top of the database to get what they need. Essentially, they try to develop a strong consistency database on top of the final consistency database. This is a pity, because others have designed distributed databases from scratch that are not only more efficient but do not require maintenance from your development team!

…Riding a bicycle with loose wheels and invisible.

The developer who ended up in the second situation was riding a partially invisible bike. They didn't realize the wheels were loose, they didn't see the wheels falling off, and once they looked up after falling, they still saw a completely intact bike.

When things go wrong, the complexity of solving these errors is high for the following reasons:

• Determine if it is a final consistency error. The problem may be an application error or an error caused by misunderstanding of the guarantees of the underlying database. To be sure, we need to investigate application logic, and if the application logic is reasonable in a non-distributed environment, then engineers must have the instinct to evaluate whether this situation may occur due to final consistency.
• The reason disappeared. Second, as the database eventually becomes consistent, the root of the problem may have disappeared (the wheels magically reconnected to the bike, and all you see is a perfect bike).
• Fix it! Once the problem is identified, you can find a solution, try building a layer on top of the database (hi, latency and other potential errors), deleting features, or changing the database. The last option is sometimes considered easy. However, even the slightest differences between databases can make this work very challenging. By the time your app takes off, you're already busy. This is not the time when you want to change the database!

…Riding a bike with a loose wheel, invisible, and a group of people standing on your shoulders.

The invisible bike example is still too forgiving. In fact, others may depend on your application. So basically, you're riding an invisible bike while others (your clients) are standing on your shoulders.

Not only will you fall, they will fall with you, falling heavily and painfully on you. You may not even survive this fall; in other words, your company may not be able to withstand the storm of negative feedback from customers.

What is the moral of the story? If you choose a strong consistency (and ultimate consistency) database from the beginning, you don't need to consider working on a complex and resource-intensive project like migrating a database when your customers are already frustrated.

in conclusion

A few years ago, it was reasonable to choose the final consistency database to scale, because there was no other option at that time. However, we now have modern databases that can scale efficiently without sacrificing data consistency or performance. In addition, these modern databases include many other powerful features that go beyond consistency, such as ease of use, serverless pricing model, built-in authentication, timeliness, native GraphQL, and more. With modern databases, you can scale without opening Pandora's box!

And, if you still choose not to use a strongly consistent distributed database after reading this series, at least make sure to tighten your wheels (in other words, read and understand the consistency guarantees of different databases).

Series of articles

1. Reasons you should pay attention to
2. Probable problems
3. Obstacles to adoption
4. How the new algorithm can help

The above is the detailed content of Consistent Backends and UX: What are the Barriers to Adoption?. For more information, please follow other related articles on the PHP Chinese website!