Home Database Mysql Tutorial Linux上从mongoDB中定时导出数据脚本

Linux上从mongoDB中定时导出数据脚本

Jun 07, 2016 pm 05:06 PM
mongodb

目标:编写一个shell脚本,以便能够将mongoDb中的数据导出成为csv格式的文件。要求:1. 为了避免导出巨量数据,仅需要过滤出当月

目标:编写一个shell脚本,,以便能够将mongoDb中的数据导出成为csv格式的文件。

要求:

1. 为了避免导出巨量数据,仅需要过滤出当月数据和上个月的数据即可。

2. 由于当天的数据是不完整的,所以需要排除当天的数据

以下是一个例子

--------------------------------------------------------------------------

#!/bin/bash

#创建保存目录

mypath=micAnalyDir/`date +%Y/%m%d`

mkdir -p $mypath


#计算过滤时间

baseDate=`date --date '1 months ago' +'%m/01/%Y'`

echo $baseDate

declare -i baseTime

baseTime=$((`date --date "$baseDate" +"%s"` * 1000))


#导出指定数据表中的数据

function exportData() {

   exportCmd="/kingdee/mongodb/bin/mongoexport --csv"

   if [ $# -gt 3 ]; then

#这里要注意baseTime和endTime不能加引号,否则会报错

        $exportCmd -d $1 -c $2 -f $3 -q \

 "{\"$4\":{\"\$gte\":{\"\$date\":$baseTime}, \

        \"\$lt\":{\"\$date\":$endTime}}}" -o $mypath/$2.csv

   else

        $exportCmd -d $1 -c $2 -f $3 -o $mypath/$2.csv

   fi

}


#导出第一张表的所有数据

echo export data from $baseDate

echo 'export ossDev.T_User'

fields="_id,email,registerDate,photoId"

exportData "ossDev" "T_User" $fields


#导出第二张表的sendTime为当月和上月的数据

echo 'export snsDev.T_Comment'

fields="_id,del,from,mbId,mbOwner,nwId,ownerId,sendTime"

exportData "snsDev" "T_Comment" $fields "sendTime"

----------------------------------------------------------------------------

保存为.sh文件后使用 chmod +x 将该脚本设置为可执行,然后使用crontab命令将脚本加到定时任务当中

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How to connect navicat to mongodb How to connect navicat to mongodb Apr 24, 2024 am 11:27 AM

To connect to MongoDB using Navicat, you need to: Install Navicat Create a MongoDB connection: a. Enter the connection name, host address and port b. Enter the authentication information (if required) Add an SSL certificate (if required) Verify the connection Save the connection

What is the use of net4.0 What is the use of net4.0 May 10, 2024 am 01:09 AM

.NET 4.0 is used to create a variety of applications and it provides application developers with rich features including: object-oriented programming, flexibility, powerful architecture, cloud computing integration, performance optimization, extensive libraries, security, Scalability, data access, and mobile development support.

Integration of Java functions and databases in serverless architecture Integration of Java functions and databases in serverless architecture Apr 28, 2024 am 08:57 AM

In a serverless architecture, Java functions can be integrated with the database to access and manipulate data in the database. Key steps include: creating Java functions, configuring environment variables, deploying functions, and testing functions. By following these steps, developers can build complex applications that seamlessly access data stored in databases.

How to configure MongoDB automatic expansion on Debian How to configure MongoDB automatic expansion on Debian Apr 02, 2025 am 07:36 AM

This article introduces how to configure MongoDB on Debian system to achieve automatic expansion. The main steps include setting up the MongoDB replica set and disk space monitoring. 1. MongoDB installation First, make sure that MongoDB is installed on the Debian system. Install using the following command: sudoaptupdatesudoaptinstall-ymongodb-org 2. Configuring MongoDB replica set MongoDB replica set ensures high availability and data redundancy, which is the basis for achieving automatic capacity expansion. Start MongoDB service: sudosystemctlstartmongodsudosys

How to ensure high availability of MongoDB on Debian How to ensure high availability of MongoDB on Debian Apr 02, 2025 am 07:21 AM

This article describes how to build a highly available MongoDB database on a Debian system. We will explore multiple ways to ensure data security and services continue to operate. Key strategy: ReplicaSet: ReplicaSet: Use replicasets to achieve data redundancy and automatic failover. When a master node fails, the replica set will automatically elect a new master node to ensure the continuous availability of the service. Data backup and recovery: Regularly use the mongodump command to backup the database and formulate effective recovery strategies to deal with the risk of data loss. Monitoring and Alarms: Deploy monitoring tools (such as Prometheus, Grafana) to monitor the running status of MongoDB in real time, and

Navicat's method to view MongoDB database password Navicat's method to view MongoDB database password Apr 08, 2025 pm 09:39 PM

It is impossible to view MongoDB password directly through Navicat because it is stored as hash values. How to retrieve lost passwords: 1. Reset passwords; 2. Check configuration files (may contain hash values); 3. Check codes (may hardcode passwords).

Major update of Pi Coin: Pi Bank is coming! Major update of Pi Coin: Pi Bank is coming! Mar 03, 2025 pm 06:18 PM

PiNetwork is about to launch PiBank, a revolutionary mobile banking platform! PiNetwork today released a major update on Elmahrosa (Face) PIMISRBank, referred to as PiBank, which perfectly integrates traditional banking services with PiNetwork cryptocurrency functions to realize the atomic exchange of fiat currencies and cryptocurrencies (supports the swap between fiat currencies such as the US dollar, euro, and Indonesian rupiah with cryptocurrencies such as PiCoin, USDT, and USDC). What is the charm of PiBank? Let's find out! PiBank's main functions: One-stop management of bank accounts and cryptocurrency assets. Support real-time transactions and adopt biospecies

How to open table connection in navicat How to open table connection in navicat Apr 24, 2024 am 09:39 AM

Steps to access table connections through Navicat: 1. Connect to the database; 2. Browse to the required database; 3. Right-click the table and select "Edit Table"; 4. View the table data.

See all articles