I had to send a large file (about 100MB) to a client for analysis.  They did not have an anonymous ftp server so I had to figure a way to e-mail the file to them and then have a way to put it back together.

I found the split command very useful.  I use Linux when I can but have installed Cygwin on my Windows PC to get the same functionality.

Here is a way to break the file into 5 megabyte chunks:

# split -b 8m veryLargeInputFile

This instance splits veryLargeInputFile 8MB segments named xaa xab xac…xap.

Now put the file back together  at the distant end:

# cat xaa xab xac xad xae xaf xag xah xai xaj > veryLargeInputFile

or

# cat * >veryLargeInputFile

Note: ensure xa* are the only files in the directory when using the wildcard
For ASCII files: Split lines — This example splits a document into 1000 line segments.

# split -l 1000 veryLargeTextFile

Use the same process to put the file back together again.

Note: For larger files, find a ftp server or make your filesize increments bigger.

Split options

-b ##   — replace ## with the number of bytes you want in a file

-C ##   — replace ## with the number of SIZE bytes of lines per output file

-l ##     — replace ## with the number of lines per file.

-d          — use numeric suffixes for output files instead of alphabetic

Tags: Utilities