mongodb迁移工具
项目地址 [github]地址戳这里.(https://github.com/xbsura/mongo-migrate) 使用go语言编写,与m-tools保持一致,除了go语言自带的库,还需要mgo库支持. 功能 mongodb实例之间的数据迁移,带有根据源与目的的节点性质,进行自动分片,预split与movechunk,创建索引,
项目地址
[github]地址戳这里.(https://github.com/xbsura/mongo-migrate)
使用go语言编写,与m-tools保持一致,除了go语言自带的库,还需要mgo库支持.
功能
mongodb实例之间的数据迁移,带有根据源与目的的节点性质,进行自动分片,预split与movechunk,创建索引,拷贝数据与同步oplog几个过程.
设计
迁移工具分为三个阶段,初始化阶段,数据拷贝与oplog同步,分别使用InitCollection
类,CopyData
类与OplogSync
类实现.
InitCollection
过程:
- 建立与源,目标的连接,若失败,raise.
- 初始化两个标识位,s(shard),o(oplog),为0, 0.
- 检测源类型,若为mongod,检测是否为复制集,若无法检测,raise;若为复制集,令o=1;若为mongos,查看要迁移的ns是否分片,如果分片,检测数据分片类型,若都为复制集,令o=1.
- 检测目标类型,若为mongod,令s=0.
- 若s=1,在目标进行分片,按照源的chunk进行split并随机movechunk.
CopyData
过程:
1. 建立源与目的的连接.
2. 获取要迁移的ns.
4. 如果o=1,记录源每个数据分片oplog最新值.
5. 如果源ns分片,按照chunk边界从源并发读,并向目标并发写;如果源未分片,使用一个query查询数据并向目标写.
OplogSync
过程:
1. 如果o=0,退出.
2. 建立源各个数据节点,目标节点(单节点)的连接.
3. 在每个源分片查找oplog ts大于等于CopyData记录时间点的记录,并逐条应用到目标节点.
问题
- 日志貌似打印有一点点问题.
- 性能没测试.
- 代码有点烂.
目前进度 :
完成可运行的版本,没有完整测试.
原文地址:mongodb迁移工具, 感谢原作者分享。

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics











Using the chrono library in C can allow you to control time and time intervals more accurately. Let's explore the charm of this library. C's chrono library is part of the standard library, which provides a modern way to deal with time and time intervals. For programmers who have suffered from time.h and ctime, chrono is undoubtedly a boon. It not only improves the readability and maintainability of the code, but also provides higher accuracy and flexibility. Let's start with the basics. The chrono library mainly includes the following key components: std::chrono::system_clock: represents the system clock, used to obtain the current time. std::chron

DMA in C refers to DirectMemoryAccess, a direct memory access technology, allowing hardware devices to directly transmit data to memory without CPU intervention. 1) DMA operation is highly dependent on hardware devices and drivers, and the implementation method varies from system to system. 2) Direct access to memory may bring security risks, and the correctness and security of the code must be ensured. 3) DMA can improve performance, but improper use may lead to degradation of system performance. Through practice and learning, we can master the skills of using DMA and maximize its effectiveness in scenarios such as high-speed data transmission and real-time signal processing.

Measuring thread performance in C can use the timing tools, performance analysis tools, and custom timers in the standard library. 1. Use the library to measure execution time. 2. Use gprof for performance analysis. The steps include adding the -pg option during compilation, running the program to generate a gmon.out file, and generating a performance report. 3. Use Valgrind's Callgrind module to perform more detailed analysis. The steps include running the program to generate the callgrind.out file and viewing the results using kcachegrind. 4. Custom timers can flexibly measure the execution time of a specific code segment. These methods help to fully understand thread performance and optimize code.

The main steps and precautions for using string streams in C are as follows: 1. Create an output string stream and convert data, such as converting integers into strings. 2. Apply to serialization of complex data structures, such as converting vector into strings. 3. Pay attention to performance issues and avoid frequent use of string streams when processing large amounts of data. You can consider using the append method of std::string. 4. Pay attention to memory management and avoid frequent creation and destruction of string stream objects. You can reuse or use std::stringstream.

C performs well in real-time operating system (RTOS) programming, providing efficient execution efficiency and precise time management. 1) C Meet the needs of RTOS through direct operation of hardware resources and efficient memory management. 2) Using object-oriented features, C can design a flexible task scheduling system. 3) C supports efficient interrupt processing, but dynamic memory allocation and exception processing must be avoided to ensure real-time. 4) Template programming and inline functions help in performance optimization. 5) In practical applications, C can be used to implement an efficient logging system.

With the popularization and development of digital currency, more and more people are beginning to pay attention to and use digital currency apps. These applications provide users with a convenient way to manage and trade digital assets. So, what kind of software is a digital currency app? Let us have an in-depth understanding and take stock of the top ten digital currency apps in the world.

C code optimization can be achieved through the following strategies: 1. Manually manage memory for optimization use; 2. Write code that complies with compiler optimization rules; 3. Select appropriate algorithms and data structures; 4. Use inline functions to reduce call overhead; 5. Apply template metaprogramming to optimize at compile time; 6. Avoid unnecessary copying, use moving semantics and reference parameters; 7. Use const correctly to help compiler optimization; 8. Select appropriate data structures, such as std::vector.

To safely and thoroughly uninstall MySQL and clean all residual files, follow the following steps: 1. Stop MySQL service; 2. Uninstall MySQL packages; 3. Clean configuration files and data directories; 4. Verify that the uninstallation is thorough.
