HowTo - Split A Gzip File In Linux

Question

I want to split a gz file (database dump) into smaller files and move to another server and restore the database dump there.

Answer

You can split a larger file into smaller pieces using the “split” command. The syntax of split command is as follows.

split [OPTION]... [INPUT [PREFIX]]

This command will output fixed sized pieces of input file to PREFIXaa, PREFIXab etc. You can split the file according to the size of the required split files (option –b) or according to the number of lines (-l).

For example, you can split the file into 512 MB files by using the following command.

# split –b 512m “file.gz” “file.gz.part-“

This will create 512MB files named file.gz.part-aa, files file.gz.part-ab etc.

As you are trying to split a database dump, it is important that the files should not get divided in between a single line. In order to avoid such problem, you can split the file according to the number of lines.

You can make use of zcat or “gunzip –c” to output the lines from the zip file without unzipping the actual file and then pipe the output to split command as follows.

zcat file.gz | split -l 2000000 – file.gz.part
or
gunzip –c file.gz | split -l 2000000 – file.gz.part

This should create files with 2000000 lines in each with the file.gz.partaa, files file.gz.partab etc.
You can then copy the split files into the other server. As this is a database dump file, there is no need to combine the files. You can separately import the split files into the database as follows.

# mysql –u username –p dbname < file.gz.partaa
# mysql –u username –p dbname < file.gz.partab

If you want to combine the split files to generate the single file, you can do it as,

cat file.gz.part* > file.gz

Bobbin Zachariah 8:28 pm

About Bobbin Zachariah

Founder of LinOxide, passionate lover of Linux and technology writer. Started his career in Linux / Opensource from 2000. Love traveling, blogging and listening music. Reach Bobbin Zachariah about me page and google plus page.

Author's All Posts
Like to become part of Linoxide Team and contribute tips? Contact us here.

Comments

Your email address will not be published. Required fields are marked *

All comments are subject to moderation.

1 Comment

  1. Please note that the "zcat file.gz | split -l 2000000 – file.gz.part" and
    "gunzip –c file.gz | split -l 2000000 – file.gz.part", mentioned in the article, will generate non-compressed split files (non-gzip). The correct syntax should be "| split -l 2000000 - file.part".

    You can also gzip and split at the same time: gzip -c file | split -b 1024m - file.gz.part