Filtering Files Copied When Using rsync on Linux

The Linux command-line utility rsync is a powerful and flexible tool for synchronizing files and directories across different computers and locations. It is commonly used for backups, file transfers, and data replication. One of the main features of rsync is its ability to filter files based on various criteria such as file type, size, and modification time.

Rsync Overview

Rsync works by comparing the source and destination directories and copying only the files that have changed or been added to the source directory. This feature makes rsync ideal for transferring large amounts of data over networks, as it minimizes bandwidth usage. Additionally, rsync can remove files that no longer exist in the source directory, maintaining file consistency across multiple systems.

Using the Include Option

The --include option allows users to selectively copy only certain types of files. It works in conjunction with the --exclude option to create powerful filtering rules.

Basic Include/Exclude Example

To transfer only text files from a directory, use the include option followed by exclude to filter out everything else:

rsync --include="*.txt" --exclude="*" /source/ /destination/

Important: Order matters! Include and exclude options are applied sequentially, so specify what you want first, then exclude the rest.

Multiple File Types

To transfer multiple file types, use multiple include options:

rsync --include="*.txt" --include="*.log" --exclude="*" /source/ /destination/

Recursing through Subdirectories

To transfer files matching specific criteria in all subdirectories, use the -r (recursive) option with directory includes:

rsync -r --include="*/" --include="*.txt" --exclude="*" /source/ /destination/

The --include="*/" ensures directories are traversed. To prevent copying empty directories, add the --prune-empty-dirs option:

rsync -r --prune-empty-dirs --include="*/" --include="*.txt" --exclude="*" /source/ /destination/

Advanced Filtering Options

Rsync provides advanced filtering based on file attributes beyond just names and patterns.

Size-Based Filtering

Synchronize files larger than a specific size:

rsync -av --min-size=100M /source/ /destination/

Or exclude large files:

rsync -av --max-size=50M /source/ /destination/

Time-Based Filtering

Transfer files modified within a specific timeframe:

rsync -av --files-from=<(find /source -mtime -30 -type f -printf '%P<br>') /source/ /destination/

Common Filter Patterns

Practical Examples

Backup Only Documents

rsync -av --include="*.{doc,docx,pdf,txt}" --exclude="*" /home/user/ /backup/documents/

Sync Media Files Recursively

rsync -rv --prune-empty-dirs --include="*/" --include="*.{mp3,mp4,avi,mkv}" --exclude="*" /media/ /backup/media/

Filter File Method

For complex filtering rules, create a filter file:

echo "include *.txt" > filters.txt
echo "include *.log" >> filters.txt
echo "exclude *" >> filters.txt

rsync -av --filter="merge filters.txt" /source/ /destination/

Conclusion

Rsync's filtering capabilities make it an invaluable tool for selective file synchronization. By combining include/exclude patterns with recursive options and advanced filters, you can precisely control which files are transferred. Understanding the order of filter application and using appropriate options like --prune-empty-dirs ensures efficient and targeted file operations across Linux systems.