HBase入门篇4–存储
前几篇文章讲述了 HBase的安装、Hbase命令和API的使用、HBase简单的优化技巧，《HBase入门篇4》这篇文章是讲述把HBase的数据放在HDFS上的点滴过程。目前对与HBase我是一个绝对的新手，如果在文章中有任何我理解有错误的地方请各位指正，谢谢。 Ok，进行正题
前几篇文章讲述了 HBase的安装、Hbase命令和API的使用、HBase简单的优化技巧，《HBase入门篇4》这篇文章是讲述把HBase的数据放在HDFS上的点滴过程。目前对与HBase我是一个绝对的新手，如果在文章中有任何我理解有错误的地方请各位指正，谢谢。
Ok，进行正题 ………
在HBase中创建的一张表可以分布在多个Hregion，也就说一张表可以被拆分成多块，每一块称我们呼为一个Hregion。每个Hregion会保 存一个表里面某段连续的数据，用户创建的那个大表中的每个Hregion块是由Hregion服务器提供维护，访问Hregion块是要通过 Hregion服务器，而一个Hregion块对应一个Hregion服务器，一张完整的表可以保存在多个Hregion 上。HRegion Server 与Region的对应关系是一对多的关系。每一个HRegion在物理上会被分为三个部分：Hmemcache(缓存)、Hlog(日志)、HStore(持久层)。
上述这些关系在我脑海中的样子，如图所示：
1.HRegionServer、HRegion、Hmemcache、Hlog、HStore之间的关系，如图所示：
2.HBase表中的数据与HRegionServer的分布关系，如图所示：
HBase读数据
HBase读取数据优先读取HMemcache中的内容，如果未取到再去读取Hstore中的数据，提高数据读取的性能。
HBase写数据
HBase写入数据会写到HMemcache和Hlog中，HMemcache建立缓存，Hlog同步Hmemcache和Hstore的事务日志，发起Flush Cache时，数据持久化到Hstore中，并清空HMemecache。
客户端访问这些数据的时候通过Hmaster ，每个 Hregion 服务器都会和Hmaster 服务器保持一个长连接，Hmaster 是HBase分布式系统中的管理者，他的主要任务就是要告诉每个Hregion 服务器它要维护哪些Hregion。用户的这些都数据可以保存在Hadoop 分布式文件系统上。 如果主服务器Hmaster死机，那么整个系统都会无效。下面我会考虑如何解决Hmaster的SPFO的问题，这个问题有点类似Hadoop的SPFO 问题一样只有一个NameNode维护全局的DataNode，HDFS一旦死机全部挂了，也有人说采用Heartbeat来解决这个问题，但我总想找出 其他的解决方案，多点时间，总有办法的。
昨天在hadoop-0.21.0、hbase-0.20.6的环境中折腾了很久，一直报错，错误信息如下：
Exception in thread "main" java.io.IOException: Call to localhost/serv6:9000 failed on local exception: java.io.EOFException
10/11/10 15:34:34 ERROR master.HMaster: Can not start master
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1233)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1274)
死活连接不上HDFS，也无法连接HMaster，郁闷啊。
我想想啊，慢慢想，我眼前一亮 java.io.EOFException 这个异常，是不是有可能是RPC 协定格式不一致导致的？也就是说服务器端和客户端的版本不一致的问题？换了一个HDFS的服务器端以后，一切都好了，果然是版本的问题，最后采用 hadoop-0.20.2 搭配hbase-0.20.6 比较稳当。
最后的效果如图所示：
查看大图请点击这里， 上图的一些文字说明：
1.hadoop版本是0.20.2 ,
2.hbase版本是0.20.6,
3.在hbase中创建了一张表 tab1，退出hbase shell环境,
4.用hadoop命令查看，文件系统中的文件果然多了一个刚刚创建的tab1目录,
以上这张图片说明HBase在分布式文件系统Apache HDFS中运行了。
相关文章：
Hbase入门6 -白话MySQL(RDBMS)与HBase之间
Lily-建立在HBase上的分布式搜索
MySQL向Hive/HBase的迁移工具
HBase入门5(集群) -压力分载与失效转发
Hive入门3–Hive与HBase的整合
HBase入门篇4
HBase入门篇3
HBase入门篇2-Java操作HBase例子
HBase入门篇
基于Hbase存储的分布式消息(IM)系统-JABase
–end–
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
A Diffusion Model Tutorial Worth Your Time, from Purdue University
Apr 07, 2024 am 09:01 AM
Diffusion can not only imitate better, but also "create". The diffusion model (DiffusionModel) is an image generation model. Compared with the well-known algorithms such as GAN and VAE in the field of AI, the diffusion model takes a different approach. Its main idea is a process of first adding noise to the image and then gradually denoising it. How to denoise and restore the original image is the core part of the algorithm. The final algorithm is able to generate an image from a random noisy image. In recent years, the phenomenal growth of generative AI has enabled many exciting applications in text-to-image generation, video generation, and more. The basic principle behind these generative tools is the concept of diffusion, a special sampling mechanism that overcomes the limitations of previous methods.
Generate PPT with one click! Kimi: Let the 'PPT migrant workers' become popular first
Aug 01, 2024 pm 03:28 PM
Kimi: In just one sentence, in just ten seconds, a PPT will be ready. PPT is so annoying! To hold a meeting, you need to have a PPT; to write a weekly report, you need to have a PPT; to make an investment, you need to show a PPT; even when you accuse someone of cheating, you have to send a PPT. College is more like studying a PPT major. You watch PPT in class and do PPT after class. Perhaps, when Dennis Austin invented PPT 37 years ago, he did not expect that one day PPT would become so widespread. Talking about our hard experience of making PPT brings tears to our eyes. "It took three months to make a PPT of more than 20 pages, and I revised it dozens of times. I felt like vomiting when I saw the PPT." "At my peak, I did five PPTs a day, and even my breathing was PPT." If you have an impromptu meeting, you should do it
How to install Android apps on Linux?
Mar 19, 2024 am 11:15 AM
Installing Android applications on Linux has always been a concern for many users. Especially for Linux users who like to use Android applications, it is very important to master how to install Android applications on Linux systems. Although running Android applications directly on Linux is not as simple as on the Android platform, by using emulators or third-party tools, we can still happily enjoy Android applications on Linux. The following will introduce how to install Android applications on Linux systems.
All CVPR 2024 awards announced! Nearly 10,000 people attended the conference offline, and a Chinese researcher from Google won the best paper award
Jun 20, 2024 pm 05:43 PM
In the early morning of June 20th, Beijing time, CVPR2024, the top international computer vision conference held in Seattle, officially announced the best paper and other awards. This year, a total of 10 papers won awards, including 2 best papers and 2 best student papers. In addition, there were 2 best paper nominations and 4 best student paper nominations. The top conference in the field of computer vision (CV) is CVPR, which attracts a large number of research institutions and universities every year. According to statistics, a total of 11,532 papers were submitted this year, and 2,719 were accepted, with an acceptance rate of 23.6%. According to Georgia Institute of Technology’s statistical analysis of CVPR2024 data, from the perspective of research topics, the largest number of papers is image and video synthesis and generation (Imageandvideosyn
A must-read for technical beginners: Analysis of the difficulty levels of C language and Python
Mar 22, 2024 am 10:21 AM
Title: A must-read for technical beginners: Difficulty analysis of C language and Python, requiring specific code examples In today's digital age, programming technology has become an increasingly important ability. Whether you want to work in fields such as software development, data analysis, artificial intelligence, or just learn programming out of interest, choosing a suitable programming language is the first step. Among many programming languages, C language and Python are two widely used programming languages, each with its own characteristics. This article will analyze the difficulty levels of C language and Python
How to Install and Run the Ubuntu Notes App on Ubuntu 24.04
Mar 22, 2024 pm 04:40 PM
While studying in high school, some students take very clear and accurate notes, taking more notes than others in the same class. For some, note-taking is a hobby, while for others, it is a necessity when they easily forget small information about anything important. Microsoft's NTFS application is particularly useful for students who wish to save important notes beyond regular lectures. In this article, we will describe the installation of Ubuntu applications on Ubuntu24. Updating the Ubuntu System Before installing the Ubuntu installer, on Ubuntu24 we need to ensure that the newly configured system has been updated. We can use the most famous "a" in Ubuntu system
How to install Podman on Ubuntu 24.04
Mar 22, 2024 am 11:26 AM
If you have used Docker, you must understand daemons, containers, and their functions. A daemon is a service that runs in the background when a container is already in use in any system. Podman is a free management tool for managing and creating containers without relying on any daemon such as Docker. Therefore, it has advantages in managing containers without the need for long-term backend services. Additionally, Podman does not require root-level permissions to be used. This guide discusses in detail how to install Podman on Ubuntu24. To update the system, we first need to update the system and open the Terminal shell of Ubuntu24. During both installation and upgrade processes, we need to use the command line. a simple
From bare metal to a large model with 70 billion parameters, here is a tutorial and ready-to-use scripts
Jul 24, 2024 pm 08:13 PM
We know that LLM is trained on large-scale computer clusters using massive data. This site has introduced many methods and technologies used to assist and improve the LLM training process. Today, what we want to share is an article that goes deep into the underlying technology and introduces how to turn a bunch of "bare metals" without even an operating system into a computer cluster for training LLM. This article comes from Imbue, an AI startup that strives to achieve general intelligence by understanding how machines think. Of course, turning a bunch of "bare metal" without an operating system into a computer cluster for training LLM is not an easy process, full of exploration and trial and error, but Imbue finally successfully trained an LLM with 70 billion parameters. and in the process accumulate
