Splitting Files in Unix Systems

Introduction

Unix systems have been popular for their efficiency and versatility in handling file operations. One of common operations is to split files into smaller chunks to make them more manageable. split command in Unix is used to achieve this task. This command allows user to split a large file into smaller files of specific sizes or based on specific delimiters. In this article, we will explore split command and its usage in Unix systems.

Split Command Syntax −

The basic syntax of split command is as follows −

split [OPTION]... [INPUT [PREFIX]]

The [OPTION] and [PREFIX] are optional arguments that can be used with command. [INPUT] argument specifies file to be split. If [PREFIX] argument is not provided, default prefix 'x' is used.

Options

The split command has several options that can be used to modify behavior of command. Here are some commonly used options −

• -b − This option is used to specify size of split files in bytes. For example, to split a file into 1MB chunks, use following command −

$ split -b 1m large_file.txt

• -l − This option is used to split a file based on number of lines. For example, to split a file into files with 100 lines each, use following command −

$ split -l 100 large_file.txt

• -a − This option is used to specify number of characters to be used in suffix of split files. For example, to use a suffix of three characters, use following command −

$ split -a 3 large_file.txt

• -d − This option is used to use numeric suffixes instead of alphabetic suffixes for split files. For example, to use numeric suffixes, use following command −

$ split -d large_file.txt

• -t − This option is used to specify delimiter to be used for splitting file. For example, to split a file based on occurrence of string 'END', use following command −

$ split -t 'END' large_file.txt

Examples

Let us now look at some examples to understand usage of split command in Unix systems.

Example 1: Splitting a File into Fixed Size Chunks

To split a file into fixed size chunks, use -b option followed by size of chunks in bytes. For example, to split a file named 'large_file.txt' into 1MB chunks, use following command −

$ split -b 1m large_file.txt

This command will split file into chunks of 1MB each and name them as 'xaa', 'xab', 'xac', and so on.

Example 2: Splitting a File into Fixed Number of Lines

To split a file into fixed number of lines, use -l option followed by number of lines in each split file. For example, to split a file named 'large_file.txt' into files with 100 lines each, use following command −

$ split -l 100 large_file.txt

This command will split file into files with 100 lines each and name them as 'xaa', 'xab', 'xac', and so on.

Example 3: Using a Custom Suffix for Split Files

To use a custom suffix for split files, use -a option followed by number of characters in suffix and -d option to use numeric suffixes. For example, to split a file named 'large_file.txt' into files with a suffix of three numeric characters, use following command −

$ split -a 3 -d large_file.txt

This command will split file into files with a suffix of three numeric characters, starting with '000' and incrementing for each file. files will be named as 'large_file.txt.000', 'large_file.txt.001', 'large_file.txt.002', and so on.

Example 4: Splitting a File Based on a Delimiter

To split a file based on a delimiter, use -t option followed by delimiter string. For example, to split a file named 'large_file.txt' based on occurrence of string 'END', use following command −

$ split -t 'END' large_file.txt

This command will split file into files based on occurrence of string 'END' and name them as 'xaa', 'xab', 'xac', and so on.

Other Usage of Split Command

The split command can also be used in combination with other Unix commands to perform more complex operations. Here are some examples of split command being used in combination with other commands.

Splitting and Compressing Files

To split a file into smaller chunks and compress them, use following command −

$ split -b 1m large_file.txt | gzip > large_file.tar.gz

This command will split file into 1MB chunks and then compress them using gzip. compressed chunks will be written to a file named 'large_file.tar.gz'.

Combining Split Files

To combine split files into a single file, use following command −

$ cat x* > combined_file.txt

This command will combine all split files with names starting with 'x' into a single file named 'combined_file.txt'.

Splitting and Encrypting Files

To split a file into smaller chunks and encrypt them, use following command −

$ split -b 1m large_file.txt | openssl enc -aes-256-cbc > encrypted_file.enc

This command will split file into 1MB chunks and then encrypt them using AES-256-CBC encryption algorithm provided by OpenSSL. encrypted chunks will be written to a file named 'encrypted_file.enc'.

Splitting and Transferring Files over Network

To split a file into smaller chunks and transfer them over network using SSH, use following command −

$ split -b 1m large_file.txt | ssh user@host 'cat > large_file.txt'

This command will split file into 1MB chunks and transfer them over network to a remote host using SSH. chunks will be combined on remote host and written to a file named 'large_file.txt'.

Conclusion

The split command in Unix systems is a useful tool for splitting large files into smaller chunks. It provides several options to customize splitting process based on file size, number of lines, suffix, and delimiter. split command can be used in various scenarios such as splitting log files, large data sets, and text files. By using examples provided in this article, users can easily split their files and manage them more efficiently.