Home Database Mysql Tutorial Linux上从mongoDB中定时导出数据脚本

Linux上从mongoDB中定时导出数据脚本

Jun 07, 2016 pm 05:06 PM
mongodb

目标:编写一个shell脚本,以便能够将mongoDb中的数据导出成为csv格式的文件。要求:1. 为了避免导出巨量数据,仅需要过滤出当月

目标:编写一个shell脚本,,以便能够将mongoDb中的数据导出成为csv格式的文件。

要求:

1. 为了避免导出巨量数据,仅需要过滤出当月数据和上个月的数据即可。

2. 由于当天的数据是不完整的,所以需要排除当天的数据

以下是一个例子

--------------------------------------------------------------------------

#!/bin/bash

#创建保存目录

mypath=micAnalyDir/`date +%Y/%m%d`

mkdir -p $mypath


#计算过滤时间

baseDate=`date --date '1 months ago' +'%m/01/%Y'`

echo $baseDate

declare -i baseTime

baseTime=$((`date --date "$baseDate" +"%s"` * 1000))


#导出指定数据表中的数据

function exportData() {

   exportCmd="/kingdee/mongodb/bin/mongoexport --csv"

   if [ $# -gt 3 ]; then

#这里要注意baseTime和endTime不能加引号,否则会报错

        $exportCmd -d $1 -c $2 -f $3 -q \

 "{\"$4\":{\"\$gte\":{\"\$date\":$baseTime}, \

        \"\$lt\":{\"\$date\":$endTime}}}" -o $mypath/$2.csv

   else

        $exportCmd -d $1 -c $2 -f $3 -o $mypath/$2.csv

   fi

}


#导出第一张表的所有数据

echo export data from $baseDate

echo 'export ossDev.T_User'

fields="_id,email,registerDate,photoId"

exportData "ossDev" "T_User" $fields


#导出第二张表的sendTime为当月和上月的数据

echo 'export snsDev.T_Comment'

fields="_id,del,from,mbId,mbOwner,nwId,ownerId,sendTime"

exportData "snsDev" "T_Comment" $fields "sendTime"

----------------------------------------------------------------------------

保存为.sh文件后使用 chmod +x 将该脚本设置为可执行,然后使用crontab命令将脚本加到定时任务当中

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Which version is generally used for mongodb? Which version is generally used for mongodb? Apr 07, 2024 pm 05:48 PM

It is recommended to use the latest version of MongoDB (currently 5.0) as it provides the latest features and improvements. When selecting a version, you need to consider functional requirements, compatibility, stability, and community support. For example, the latest version has features such as transactions and aggregation pipeline optimization. Make sure the version is compatible with the application. For production environments, choose the long-term support version. The latest version has more active community support.

The difference between nodejs and vuejs The difference between nodejs and vuejs Apr 21, 2024 am 04:17 AM

Node.js is a server-side JavaScript runtime, while Vue.js is a client-side JavaScript framework for creating interactive user interfaces. Node.js is used for server-side development, such as back-end service API development and data processing, while Vue.js is used for client-side development, such as single-page applications and responsive user interfaces.

Experience sharing on implementing real-time recommendation system using MongoDB Experience sharing on implementing real-time recommendation system using MongoDB Nov 03, 2023 pm 04:37 PM

With the development of the Internet, people's lives are becoming more and more digital, and the demand for personalization is becoming stronger and stronger. In this era of information explosion, users are often faced with massive amounts of information and have no choice, so the importance of real-time recommendation systems has become increasingly prominent. This article will share the experience of using MongoDB to implement a real-time recommendation system, hoping to provide some inspiration and help to developers. 1. Introduction to MongoDB MongoDB is an open source NoSQL database known for its high performance, easy scalability and flexible data model. Compared to biography

Where is the database created by mongodb? Where is the database created by mongodb? Apr 07, 2024 pm 05:39 PM

The data of the MongoDB database is stored in the specified data directory, which can be located in the local file system, network file system or cloud storage. The specific location is as follows: Local file system: The default path is Linux/macOS:/data/db, Windows: C:\data\db. Network file system: The path depends on the file system. Cloud Storage: The path is determined by the cloud storage provider.

What are the advantages of mongodb database What are the advantages of mongodb database Apr 07, 2024 pm 05:21 PM

The MongoDB database is known for its flexibility, scalability, and high performance. Its advantages include: a document data model that allows data to be stored in a flexible and unstructured way. Horizontal scalability to multiple servers via sharding. Query flexibility, supporting complex queries and aggregation operations. Data replication and fault tolerance ensure data redundancy and high availability. JSON support for easy integration with front-end applications. High performance for fast response even when processing large amounts of data. Open source, customizable and free to use.

What does mongodb mean? What does mongodb mean? Apr 07, 2024 pm 05:57 PM

MongoDB is a document-oriented, distributed database system used to store and manage large amounts of structured and unstructured data. Its core concepts include document storage and distribution, and its main features include dynamic schema, indexing, aggregation, map-reduce and replication. It is widely used in content management systems, e-commerce platforms, social media websites, IoT applications, and mobile application development.

Where are the mongodb database files? Where are the mongodb database files? Apr 07, 2024 pm 05:42 PM

The MongoDB database file is located in the MongoDB data directory, which is /data/db by default, which contains .bson (document data), ns (collection information), journal (write operation records), wiredTiger (data when using the WiredTiger storage engine ) and config (database configuration information) and other files.

How to open mongodb How to open mongodb Apr 07, 2024 pm 06:15 PM

On Linux/macOS: Create the data directory and start the "mongod" service. On Windows: Create the data directory and start the MongoDB service from Service Manager. In Docker: Run the "docker run" command. On other platforms: Please consult the MongoDB documentation. Verification method: Run the "mongo" command to connect and view the server version.

See all articles