[译]Cassandra 架构简述
本文翻译主要来自Datastax的cassandra1.2文档。。此外还有一些来自于相关官方博客。 该翻译作为实验室大数据组的学习材料的一部分，适合对Cassandra已经有一定了解的读者。 未经本人许可，请勿转载。 。（下文有时候又将Cassandra简称C） Cassandra被设计来
本文翻译主要来自Datastax的cassandra1.2文档。。此外还有一些来自于相关官方博客。
该翻译作为实验室大数据组的学习材料的一部分，适合对Cassandra已经有一定了解的读者。
未经本人许可，请勿转载。
。（下文有时候又将Cassandra简称C）
Cassandra被设计来通过没有单点故障的多节点模式去处理海量数据工作负载。他的架构是基于 理解系统和硬件故障可以而且会发生 的基础上的。C通过所有节点都相同并且数据分布在所有节点上的p2p分布式系统来解决故障问题。集群中的每个节点每秒都在交换信息。每个节点上的commit log 捕获写行为来确保数据的持久化。数据也会被写到一个内存结构中，叫做memtable，然后当内存结构满了的时候就写数据到磁盘文件中，叫做SSTable。所有的写入都是自动分区和复制的。
cassandra是一种面向行的数据库。C的架构允许任何授权的用户连接任意数据中心的任意的节点，并使用cql访问数据。为了简化使用，cql使用和sql类似的语法。从cql的视角出发，database是由tables组成的。典型地，一个集群中 每个应用拥有一个keyspace。开发者可以通过cqlsh调用cql，也可以使用其他驱动。
客户端的读写请求可以到达集群的任意节点。当一个客户连接到一个节点做了一个请求时，那个节点服务器就作为这个特定的客户操作的一个了。协调器扮演了客户应用和拥有用户请求的数据的节点之间的代理（proxy）的角色。协调器决定了集群环中的哪些节点应该响应请求。（更多信息，请查阅关于用户请求）
配置C的关键组件列表：
gossip信息也被每个节点保存在本地，这样当一个节点重启时，它能够立刻使用这些信息。你可能会想清空某个节点上的gossip历史，比如节点ip地址改变了等原因。（译者注：大概就是system.local表）
你必须设置分区器的类型，并且指派给每个节点一个num_tokens值。如果没有使用虚拟节点的话，美国空间，使用initial_token来代替。（译者注：虚拟节点是1.2中新增的）
默认的，美国空间，一个节点被设置为存储他管理的数据到/var/lib/cassandra目录。在一个生产环境中，你需要修改commitlog目录到一个其他硬盘上去（别和data file 在一个硬盘上）。
（该翻译作为实验室大数据组的学习材料的一部分，适合对Cassandra已经有一定了解的读者。未经本人许可，请勿转载。）
：
cassandra使用称为gossip的协议来发现加入C集群中的其他节点的位置和状态信息。这是一个p2p的交流协议，每个节点定期的交换他们自己的和他们所知道的其他人的状态信息。gossip进程每秒都在进行，并与至多三个节点交换状态信息。节点交换他们自己和所知道的信息，于是所有的节点很快就能学习到整个集群中的其他节点的信息。gossip信息有一个相关的版本号，于是在一次gossip信息交换中，旧的信息会被新的信息覆盖重写。
要阻止分区进行gossip交流，那么在集群中的所有节点中使用相同的seed list（译者注：指的是cassandra。yaml中的seeds）。默认的，在重新启动时，一个节点记得他曾经gossip过得其他节点。
注意：种子节点的指定除了启动起gossip进程外，没有其他的目的。种子节点不是一个单点故障，他们在集群操作中也没有其他的特殊目的，除了引导节点以外..
设置Gossip设置
任务：
当一个节点第一次启动的时候，他去yaml中读取配置，得到集群的名字，并得到从哪些seeds中获取其他节点的信息，还有其他的一些参数，服务器空间，比如端口，范围等等。。
属性 描述
cluster_name
listen_address 与其他节点连接的ip
seed_provider
storage_port 内部节点交流端口（默认7000），每个节点之间必须相同
initial_token 在1.1以及之前，决定节点的数据的管理范围
num_tokens 在1.2以及之后，决定节点的数据的管理范围
清理gossip状态:
-Dcassandra.load_ring_state= false 关于故障检测和修复C使用信息来避免路由用户的请求到坏了的节点（C还能避免路由到可用但是性能很差的节点，通过动态snitch技术）Rather than have a fixed threshold for marking failing nodes, Cassandra uses an accrual detection mechanism to calculate a per-node threshold that takes into account network performance, workload, or other conditions. During gossip exchanges, every node maintains a sliding window of inter-arrival times of gossip messages from other nodes in the cluster. In Cassandra, configuring the property adjusts the sensitivity of the failure detector. Use default value for most situations, but increase it to 12 for Amazon EC2 (due to the frequently experienced network congestion).（译者注：这是04年的一篇论文的失效检测算法1.Hayashibara, N., Defago, X., Yared, R. & Katayama, T. The phi; accrual failure detector. in Proceedings of the 23rd IEEE International Symposium on Reliable Distributed Systems, 2004 66–78 (2004). doi:10.1109/RELDIS.2004.1353004）一个节点的当机往往不代表这个节点永远的离开了，因此并不会自动的从环中删除。其他的节点会定期的尝试与失效节点联系看看他们恢复了没。要永久的改变一个节点的关系，administrators must explicitly add or remove nodes from a Cassandra cluster using the .当一个节点返回的时候，他可能错过了他需要维护的副本数据的写入命令。一旦失效检测标记一个节点当机了，错过的写入就会被存储到其他的副本中一段时间，叫做 hinted handoff。 当一个节点当机超过max_hint_windows_in_ms(默认3小时)，hints就不在存储了。这时候你应该等节点启动后运行修复程序了。此外，你应该日常地运行nodetool repair 在所有的节点上，以保证他们的数据的一致性。For more explanation about recovery, see Modern hinted handoff. （该翻译作为实验室大数据组的学习材料的一部分，适合对Cassandra已经有一定了解的读者。未经本人许可，请勿转载。）
数据分配和备份
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
Comparative analysis of deep learning architectures
May 17, 2023 pm 04:34 PM
The concept of deep learning originates from the research of artificial neural networks. A multi-layer perceptron containing multiple hidden layers is a deep learning structure. Deep learning combines low-level features to form more abstract high-level representations to represent categories or characteristics of data. It is able to discover distributed feature representations of data. Deep learning is a type of machine learning, and machine learning is the only way to achieve artificial intelligence. So, what are the differences between various deep learning system architectures? 1. Fully Connected Network (FCN) A fully connected network (FCN) consists of a series of fully connected layers, with every neuron in each layer connected to every neuron in another layer. Its main advantage is that it is "structure agnostic", i.e. no special assumptions about the input are required. Although this structural agnostic makes the complete
This 'mistake' is not really a mistake: start with four classic papers to understand what is 'wrong' with the Transformer architecture diagram
Jun 14, 2023 pm 01:43 PM
Some time ago, a tweet pointing out the inconsistency between the Transformer architecture diagram and the code in the Google Brain team's paper "AttentionIsAllYouNeed" triggered a lot of discussion. Some people think that Sebastian's discovery was an unintentional mistake, but it is also surprising. After all, considering the popularity of the Transformer paper, this inconsistency should have been mentioned a thousand times. Sebastian Raschka said in response to netizen comments that the "most original" code was indeed consistent with the architecture diagram, but the code version submitted in 2017 was modified, but the architecture diagram was not updated at the same time. This is also the root cause of "inconsistent" discussions.
Multi-path, multi-domain, all-inclusive! Google AI releases multi-domain learning general model MDL
May 28, 2023 pm 02:12 PM
Deep learning models for vision tasks (such as image classification) are usually trained end-to-end with data from a single visual domain (such as natural images or computer-generated images). Generally, an application that completes vision tasks for multiple domains needs to build multiple models for each separate domain and train them independently. Data is not shared between different domains. During inference, each model will handle a specific domain. input data. Even if they are oriented to different fields, some features of the early layers between these models are similar, so joint training of these models is more efficient. This reduces latency and power consumption, and reduces the memory cost of storing each model parameter. This approach is called multi-domain learning (MDL). In addition, MDL models can also outperform single
Ten elements of machine learning system architecture
Apr 13, 2023 pm 11:37 PM
This is an era of AI empowerment, and machine learning is an important technical means to achieve AI. So, is there a universal machine learning system architecture? Within the cognitive scope of experienced programmers, Anything is nothing, especially for system architecture. However, it is possible to build a scalable and reliable machine learning system architecture if applicable to most machine learning driven systems or use cases. From a machine learning life cycle perspective, this so-called universal architecture covers key machine learning stages, from developing machine learning models, to deploying training systems and service systems to production environments. We can try to describe such a machine learning system architecture from the dimensions of 10 elements. 1.
What is the architecture and working principle of Spring Data JPA?
Apr 17, 2024 pm 02:48 PM
SpringDataJPA is based on the JPA architecture and interacts with the database through mapping, ORM and transaction management. Its repository provides CRUD operations, and derived queries simplify database access. Additionally, it uses lazy loading to only retrieve data when necessary, thus improving performance.
Software architecture design and software and hardware decoupling methodology in SOA
Apr 08, 2023 pm 11:21 PM
For the next generation of centralized electronic and electrical architecture, the use of central+zonal central computing unit and regional controller layout has become a must-have option for various OEMs or tier1 players. Regarding the architecture of the central computing unit, there are three ways: separation SOC, hardware isolation, software virtualization. The centralized central computing unit will integrate the core business functions of the three major domains of autonomous driving, smart cockpit and vehicle control. The standardized regional controller has three main responsibilities: power distribution, data services, and regional gateway. Therefore, the central computing unit will integrate a high-throughput Ethernet switch. As the degree of integration of the entire vehicle becomes higher and higher, more and more ECU functions will be slowly absorbed into the regional controller. And platformization
1.3ms takes 1.3ms! Tsinghua's latest open source mobile neural network architecture RepViT
Mar 11, 2024 pm 12:07 PM
Paper address: https://arxiv.org/abs/2307.09283 Code address: https://github.com/THU-MIG/RepViTRepViT performs well in the mobile ViT architecture and shows significant advantages. Next, we explore the contributions of this study. It is mentioned in the article that lightweight ViTs generally perform better than lightweight CNNs on visual tasks, mainly due to their multi-head self-attention module (MSHA) that allows the model to learn global representations. However, the architectural differences between lightweight ViTs and lightweight CNNs have not been fully studied. In this study, the authors integrated lightweight ViTs into the effective
How steep is the learning curve of golang framework architecture?
Jun 05, 2024 pm 06:59 PM
The learning curve of the Go framework architecture depends on familiarity with the Go language and back-end development and the complexity of the chosen framework: a good understanding of the basics of the Go language. It helps to have backend development experience. Frameworks that differ in complexity lead to differences in learning curves.
