Table of Contents
一、低效的分片键
1.1 分布差
1.2 缺乏局部性
3. 无法拆分的块
二、理想的分片键
Home Database Mysql Tutorial 【MongoDB】在windows平台下mongodb的分片集群(五)

【MongoDB】在windows平台下mongodb的分片集群(五)

Jun 07, 2016 pm 04:09 PM
mongodb windows Fragmentation

本篇接着上面的四篇继续讲述在window平台下mongodb的分片集群搭建。在分片集群中也照样可以创建索引,创建索引的方式与在单独数据库中创建索引的方式一样。因此这不再多说。本篇主要聚焦在分片键的选取问题上。 分片键通俗来说就是分割海量数据的标记符。 如

本篇接着上面的四篇继续讲述在window平台下mongodb的分片集群搭建。在分片集群中也照样可以创建索引,创建索引的方式与在单独数据库中创建索引的方式一样。因此这不再多说。本篇主要聚焦在分片键的选取问题上。

分片键通俗来说就是分割海量数据的标记符。 如果更高效的划分海量数据往往依赖于分片键的选择。 分片键选得不好,应用程序就无法利用分片集群所提供的诸多优势。在这种情况下,查询和插入得系能都回显著下降。

一、低效的分片键

1.1 分布差

BSON对象ID是每个mongodb文档的默认主键。所有的对象ID最重要的组成部分是时间戳,也就是说对象ID是升序的,遗憾的是升序对于分片键来说是很糟糕的。由于分片是基于范围的。使用升序的分片键后,所有最近插入的文档会落在某个很小的连续范围内。如果想让插入负载分不到多个分片上,就不能使用升序分片键,应需某些随机性更强发的的东西。

1.2 缺乏局部性

升序分片键由明确的方向,完全随机的分片键根部没有方向。前者无法分散插入,而后者则可能将插入分散太慢。假设分片集合中每个文档都包含一个MD5,而MD5字段就是分片键。因为MD5随着文档的不同而进行变化。所有该分片键能确保插入的文档均匀分布在集群的分片上。但是有个问题,对于每个分片的MD5字段索引进行的插入过程中,索引中每个虚拟内存分页都有可能被访问到。这就意外着有可能所有的索引和数据都装在内存中。从而超出了物理内存。

3. 无法拆分的块

随机分片键和升序分片键都不好用,那么就尝试一下粗粒度分片键。举个例子,例如用户Id上传了100张照片,那么分片键就是用户ID,第一原因对于每张照片来说具有随机性,同时可以通过局部性引用来提升效率。但有个问题就是当用户ID上传的照片太大时候,以至于不得不分块。而系统又不能把一个用户的照片拆分成多个快。

二、理想的分片键

通过上面分析,理想的分片键应该满足:
1. 将插入数据均匀分布到各个分片上 2.保证crud操作能够利用局部性 3. 有足够的粒度进行块划分
举个例子:创建一个网站分析系统,一个不错的数据模型就是每个网页每月保存一个文档,随后在那个文档中保持该月每天的数据,每次访问某个页面增加一些计数器字段。下面是于分片键有关的实例分析文档:
{ _id: objectId("34535353245eraf32223sdarwe") domin:"org.mongod" url:"download" perid:"2011-12" }
最简单的分片就是包含每个网页的域名,随后是url{domain:1, url:1}所有来自指定域的页面通常都落在一个分片上,但是一些特殊的域拥有大量页面,在必要时候仍会被拆分到分片上。

备注:本篇内容大多引自《MongoDB in action》 Kyle Banker著
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
1 months ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Can I install mysql on Windows 7 Can I install mysql on Windows 7 Apr 08, 2025 pm 03:21 PM

Yes, MySQL can be installed on Windows 7, and although Microsoft has stopped supporting Windows 7, MySQL is still compatible with it. However, the following points should be noted during the installation process: Download the MySQL installer for Windows. Select the appropriate version of MySQL (community or enterprise). Select the appropriate installation directory and character set during the installation process. Set the root user password and keep it properly. Connect to the database for testing. Note the compatibility and security issues on Windows 7, and it is recommended to upgrade to a supported operating system.

Solutions to the errors reported by MySQL on a specific system version Solutions to the errors reported by MySQL on a specific system version Apr 08, 2025 am 11:54 AM

The solution to MySQL installation error is: 1. Carefully check the system environment to ensure that the MySQL dependency library requirements are met. Different operating systems and version requirements are different; 2. Carefully read the error message and take corresponding measures according to prompts (such as missing library files or insufficient permissions), such as installing dependencies or using sudo commands; 3. If necessary, try to install the source code and carefully check the compilation log, but this requires a certain amount of Linux knowledge and experience. The key to ultimately solving the problem is to carefully check the system environment and error information, and refer to the official documents.

Unable to access mysql from terminal Unable to access mysql from terminal Apr 08, 2025 pm 04:57 PM

Unable to access MySQL from the terminal may be due to: MySQL service not running; connection command error; insufficient permissions; firewall blocks connection; MySQL configuration file error.

How to solve mysql cannot connect to local host How to solve mysql cannot connect to local host Apr 08, 2025 pm 02:24 PM

The MySQL connection may be due to the following reasons: MySQL service is not started, the firewall intercepts the connection, the port number is incorrect, the user name or password is incorrect, the listening address in my.cnf is improperly configured, etc. The troubleshooting steps include: 1. Check whether the MySQL service is running; 2. Adjust the firewall settings to allow MySQL to listen to port 3306; 3. Confirm that the port number is consistent with the actual port number; 4. Check whether the user name and password are correct; 5. Make sure the bind-address settings in my.cnf are correct.

How to solve the problem of missing dependencies when installing MySQL How to solve the problem of missing dependencies when installing MySQL Apr 08, 2025 pm 12:00 PM

MySQL installation failure is usually caused by the lack of dependencies. Solution: 1. Use system package manager (such as Linux apt, yum or dnf, Windows VisualC Redistributable) to install the missing dependency libraries, such as sudoaptinstalllibmysqlclient-dev; 2. Carefully check the error information and solve complex dependencies one by one; 3. Ensure that the package manager source is configured correctly and can access the network; 4. For Windows, download and install the necessary runtime libraries. Developing the habit of reading official documents and making good use of search engines can effectively solve problems.

Navicat's method to view MongoDB database password Navicat's method to view MongoDB database password Apr 08, 2025 pm 09:39 PM

It is impossible to view MongoDB password directly through Navicat because it is stored as hash values. How to retrieve lost passwords: 1. Reset passwords; 2. Check configuration files (may contain hash values); 3. Check codes (may hardcode passwords).

How to copy and paste mysql How to copy and paste mysql Apr 08, 2025 pm 07:18 PM

Copy and paste in MySQL includes the following steps: select the data, copy with Ctrl C (Windows) or Cmd C (Mac); right-click at the target location, select Paste or use Ctrl V (Windows) or Cmd V (Mac); the copied data is inserted into the target location, or replace existing data (depending on whether the data already exists at the target location).

MySQL download prompts disk write errors how to deal with MySQL download prompts disk write errors how to deal with Apr 08, 2025 am 11:51 AM

MySQL download prompts a disk write error. The solution is as follows: 1. Check whether the disk space is insufficient, clean up the space or replace a larger disk; 2. Use disk detection tools (such as chkdsk or fsck) to check and fix disk errors, and replace the hard disk if necessary; 3. Check the target directory permissions to ensure that the user account has write permissions; 4. Change the download tool or network environment, and use the download manager to restore interrupted download; 5. Temporarily close the anti-virus software or firewall, and re-enable it after the download is completed. By systematically troubleshooting these aspects, the problem can be solved.

See all articles