Rsync: File Transfer and Synchronization in Linux
Rsync is a powerful tool for transferring and synchronizing files between systems. It is commonly used for backups, file mirroring, and updating files between local directories or remote servers. Rsync reduces data transfer by copying only the differences between the source and destination files. Due to its speed, flexibility, and extensive options, it is a preferred choice for system administrators and developers.
Prerequisites
Ensure you have a Linux-based workstation with non-root sudo access and sufficient disk space for handling large file transfers.
Installing Rsync on Linux
Rsync is available in the official repositories of most Linux distributions and is often pre-installed. Use the commands below to install Rsync based on your Linux distribution.
Install Rsync using APT
$ sudo apt install rsync
Install Rsync using PKG
$ sudo pkg install rsync
Install Rsync using Pacman
$ sudo pacman -S rsync
Install Rsync using DNF
$ sudo dnf install rsync
Install Rsync using YUM
$ sudo yum install rsync
Rsync Command Syntax
Rsync uses the following syntax:
rsync [OPTION] SRC DEST
The Rsync command consists of the following components:
- [OPTION]: Specifies various flags or parameters used to control the Rsync operation.
- SRC: Denotes the source files and directories that need to be synchronized.
- For remote transfers over SSH, use
USER@HOST:SRC
as the source path. - DEST: Indicates the target location where files or directories will be synchronized.
- For remote synchronization over SSH, specify
USER@HOST:DEST
as the destination path.
Rsync Command Options
Below is a list of commonly used Rsync command options along with their descriptions:
Option | Long Form | Short Form | Description |
---|---|---|---|
Help | --help |
-h |
Displays the Rsync help documentation. |
Verbosity | --verbose |
-v |
Increases the verbosity level to show more details. |
--quiet |
-q |
Suppresses non-error messages for a cleaner output. | |
File Selection | --archive |
-a |
Enables archive mode to preserve symbolic links, permissions, and other attributes. |
--recursive |
-r |
Recursively transfers directories and their contents. | |
--relative |
-R |
Maintains relative paths for transferred files. | |
Updates | --update |
-u |
Skips files that are newer at the destination. |
--inplace |
Directly updates destination files in place instead of creating temporary files. | ||
Backups | --backup |
-b |
Creates backups of files before overwriting them. |
--backup-dir=DIR |
Stores backups in a specified directory. | ||
Symlinks | --links |
-l |
Copies symbolic links as symbolic links. |
--copy-links |
-L |
Replaces symbolic links with the actual files or directories they point to. | |
Permissions | --perms |
-p |
Preserves file permissions during transfer. |
Timestamps | --times |
-t |
Retains the original modification times of transferred files. |
Deletion | --delete |
Removes extra files from the destination that do not exist in the source. | |
--delete-excluded |
Deletes files at the destination that match exclusion patterns. | ||
Simulation | --dry-run |
-n |
Performs a trial run without making any actual changes. |
Remote Shell | --rsh=COMMAND |
-e |
Specifies a remote shell command for executing Rsync over a network. |
Size Control | --max-size=SIZE |
Skips transferring files that exceed the specified size. | |
--min-size=SIZE |
Ignores files that are smaller than the specified size. | ||
Partial Transfer | --partial |
Retains partially transferred files in case of interruptions. | |
File Filtering | --exclude=PATTERN |
Excludes files that match a specific pattern. | |
--include=PATTERN |
Ensures that files matching a pattern are not excluded. | ||
Output Format | --progress |
Displays real-time progress updates during the transfer. | |
-P |
Equivalent to using both --partial and --progress . |
||
--human-readable |
-h |
Formats output in an easily readable manner. | |
Compression | --compress |
-z |
Enables file compression during transfer. |
IP Version | --ipv4 |
-4 |
Forces Rsync to use IPv4. |
--ipv6 |
-6 |
Forces Rsync to use IPv6. |
Transferring Local Files Using Rsync
The following steps explain how to move files between directories using Rsync.
To copy file1
, file2
, and file3
into a directory called dest
, use the Rsync command. The last parameter in Rsync always refers to the destination, so be cautious to avoid unintentional overwrites.
rsync file1 file2 file3 dest
- file1 file2 file3: Files being copied.
- dest: Target directory for storing the copied files.
To confirm the transfer, use the ls
command:
ls dest/
Expected output:
./dest/file1 ./dest/file2 ./dest/file3
Copying Directories Locally
By default, Rsync does not automatically copy all contents inside a directory. To copy directories recursively while maintaining file attributes such as timestamps, permissions, and symbolic links, use the -a
(archive) flag.
Copy the previously created dest
directory:
rsync -a dest/ dest-copy
- -a: Enables archive mode for recursive copying while keeping file attributes intact.
- dest/: Source directory (a trailing
/
ensures only contents are copied, not the directory itself). - dest-copy: Destination directory.
Verify the copied directory using:
ls dest-copy/
Expected output:
./dest-copy/file1 ./dest-copy/file2 ./dest-copy/file3
Excluding Files or Directories in Rsync
Use the --exclude
option in Rsync to avoid copying certain files or directories.
The following command copies everything from the current directory while omitting file2
and any directory that starts with dest
:
rsync -a --exclude 'file2' --exclude 'dest*' . output
- -a: Enables archive mode.
- –exclude ‘file2’: Skips files named
file2
. - –exclude ‘dest*’: Excludes all files and directories that begin with
dest
. - .: Specifies the current directory as the source.
- output: Target directory.
Syncing Files to a Remote Server with Rsync
To synchronize your files and directories with a remote system using SSH, follow these steps.
Transfer Files to a Remote Machine
Use the Rsync command below to copy files and directories to a remote server:
rsync -hazvP --delete --stats . user@192.0.2.1:synced_from_local
- -h: Displays file sizes in a human-readable format.
- -a: Activates archive mode to retain file attributes.
- -z: Compresses files during transfer to optimize speed.
- -v: Enables verbose mode for detailed output.
- -P: Shows progress updates and resumes partial transfers if interrupted.
- –delete: Removes files from the destination that no longer exist in the source.
- –stats: Provides a summary of transfer details.
- .: Indicates the current directory as the source.
- user@192.0.2.1:synced_from_local: Specifies the remote machine as the target destination, structured as follows:
- user: Username for logging into the remote machine.
- 192.0.2.1: IP address of the remote system.
- synced_from_local: Destination directory on the remote machine.
Accessing the Remote Server
After syncing, connect to the remote machine via SSH to verify the transferred files:
ssh user@192.0.2.1
Checking Synced Files
Run the following command on the remote server to confirm that the files were successfully transferred:
ls synced_from_local/
Expected directory structure after synchronization:
synced_from_local/dest-copy/file1 synced_from_local/dest-copy/file2 synced_from_local/dest-copy/file3 synced_from_local/dest/file1 synced_from_local/dest/file2 synced_from_local/dest/file3 synced_from_local/file1 synced_from_local/file2 synced_from_local/file3 synced_from_local/output/file1 synced_from_local/output/file3
Caution When Using the --delete
Option
Be careful when utilizing the --delete
flag, as it will permanently remove files from the destination if they are not present in the source. To prevent unintended data loss, always conduct a test run before executing the command:
rsync -a --delete --dry-run SRC DEST
Conclusion
In this tutorial, you installed Rsync, learned how to synchronize files locally and remotely, and used essential features like --exclude
and recursive copying. Always double-check Rsync commands, particularly when using --delete
. For more details, run:
man rsync