Home Database Mysql Tutorial How to Split a Large MySQL Dump into Smaller Files?

How to Split a Large MySQL Dump into Smaller Files?

Nov 24, 2024 pm 10:39 PM

How to Split a Large MySQL Dump into Smaller Files?

Splitting Large mysqldump Output into Smaller Files

It becomes necessary to transfer tables from one database to another, but the dump output from mysqldump is often too large for certain upload limits. This article explores a clever solution to split bulky mysqldump output into manageable, smaller files.

One potential approach suggested by shell scripting is to split the dump into separate files for each table. This can be achieved using the csplit command and specifying table structure markers as splitting points.

Consider this bash script that employs this technique:

#!/bin/bash

####
# Split MySQL dump SQL file into one file per table
# based on https://gist.github.com/jasny/1608062
####

#adjust this to your case:
START="/-- Table structure for table/"
# or 
#START="/DROP TABLE IF EXISTS/"


if [ $# -lt 1 ] || [[  == "--help" ]] || [[  == "-h" ]] ; then
        echo "USAGE: extract all tables:"
        echo "  DUMP_FILE"
        echo "extract one table:"
        echo "  DUMP_FILE [TABLE]"
        exit
fi

if [ $# -ge 2 ] ; then
        #extract one table 
        csplit -s -ftable  "/-- Table structure for table/" "%-- Table structure for table \`\`%" "/-- Table structure for table/" "%40103 SET TIME_ZONE=@OLD_TIME_ZONE%1"
else
        #extract all tables
        csplit -s -ftable  "$START" {*}
fi

[ $? -eq 0 ] || exit

mv table00 head

FILE=`ls -1 table* | tail -n 1`
if [ $# -ge 2 ] ; then
        # cut off all other tables
        mv $FILE foot
else
        # cut off the end of each file
        csplit -b '%d' -s -f$FILE $FILE "/40103 SET TIME_ZONE=@OLD_TIME_ZONE/" {*}
        mv ${FILE}1 foot
fi

for FILE in `ls -1 table*`; do
        NAME=`head -n1 $FILE | cut -d$'x60' -f2`
        cat head $FILE foot > "$NAME.sql"
done

rm head foot table*
Copy after login

This script uses csplit to split the dump file into individual table files, with each file named after the corresponding table. This allows for easy reassembly into a single file later on. To use this script, simply provide the path to the dump file as the first argument, and optionally specify a specific table to extract as the second argument.

The above is the detailed content of How to Split a Large MySQL Dump into Smaller Files?. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot Article Tags

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Reduce the use of MySQL memory in Docker Reduce the use of MySQL memory in Docker Mar 04, 2025 pm 03:52 PM

Reduce the use of MySQL memory in Docker

How do you alter a table in MySQL using the ALTER TABLE statement? How do you alter a table in MySQL using the ALTER TABLE statement? Mar 19, 2025 pm 03:51 PM

How do you alter a table in MySQL using the ALTER TABLE statement?

How to solve the problem of mysql cannot open shared library How to solve the problem of mysql cannot open shared library Mar 04, 2025 pm 04:01 PM

How to solve the problem of mysql cannot open shared library

What is SQLite? Comprehensive overview What is SQLite? Comprehensive overview Mar 04, 2025 pm 03:55 PM

What is SQLite? Comprehensive overview

Run MySQl in Linux (with/without podman container with phpmyadmin) Run MySQl in Linux (with/without podman container with phpmyadmin) Mar 04, 2025 pm 03:54 PM

Run MySQl in Linux (with/without podman container with phpmyadmin)

Running multiple MySQL versions on MacOS: A step-by-step guide Running multiple MySQL versions on MacOS: A step-by-step guide Mar 04, 2025 pm 03:49 PM

Running multiple MySQL versions on MacOS: A step-by-step guide

How do I secure MySQL against common vulnerabilities (SQL injection, brute-force attacks)? How do I secure MySQL against common vulnerabilities (SQL injection, brute-force attacks)? Mar 18, 2025 pm 12:00 PM

How do I secure MySQL against common vulnerabilities (SQL injection, brute-force attacks)?

How do I configure SSL/TLS encryption for MySQL connections? How do I configure SSL/TLS encryption for MySQL connections? Mar 18, 2025 pm 12:01 PM

How do I configure SSL/TLS encryption for MySQL connections?

See all articles