Maximum performance at minimal cost. Start your server in seconds now!
    Vijona

    Rsync: File Transfer and Synchronization in Linux

    Rsync is a powerful tool for transferring and synchronizing files between systems. It is commonly used for backups, file mirroring, and updating files between local directories or remote servers. Rsync reduces data transfer by copying only the differences between the source and destination files. Due to its speed, flexibility, and extensive options, it is a preferred choice for system administrators and developers.

    Prerequisites

    Ensure you have a Linux-based workstation with non-root sudo access and sufficient disk space for handling large file transfers.

    Installing Rsync on Linux

    Rsync is available in the official repositories of most Linux distributions and is often pre-installed. Use the commands below to install Rsync based on your Linux distribution.

    Install Rsync using APT

    Copy Code 
    

    $ sudo apt install rsync

    Install Rsync using PKG

    Copy Code 
    

    $ sudo pkg install rsync

    Install Rsync using Pacman

    Copy Code 
    

    $ sudo pacman -S rsync

    Install Rsync using DNF

    Copy Code 
    

    $ sudo dnf install rsync

    Install Rsync using YUM

    Copy Code 
    

    $ sudo yum install rsync

    Rsync Command Syntax

    Rsync uses the following syntax:

    Copy Code 
    

    rsync [OPTION] SRC DEST

    The Rsync command consists of the following components:

    • [OPTION]: Specifies various flags or parameters used to control the Rsync operation.
    • SRC: Denotes the source files and directories that need to be synchronized.
    • For remote transfers over SSH, use USER@HOST:SRC as the source path.
    • DEST: Indicates the target location where files or directories will be synchronized.
    • For remote synchronization over SSH, specify USER@HOST:DEST as the destination path.

    Rsync Command Options

    Below is a list of commonly used Rsync command options along with their descriptions:

    Option Long Form Short Form Description
    Help --help -h Displays the Rsync help documentation.
    Verbosity --verbose -v Increases the verbosity level to show more details.
    --quiet -q Suppresses non-error messages for a cleaner output.
    File Selection --archive -a Enables archive mode to preserve symbolic links, permissions, and other attributes.
    --recursive -r Recursively transfers directories and their contents.
    --relative -R Maintains relative paths for transferred files.
    Updates --update -u Skips files that are newer at the destination.
    --inplace Directly updates destination files in place instead of creating temporary files.
    Backups --backup -b Creates backups of files before overwriting them.
    --backup-dir=DIR Stores backups in a specified directory.
    Symlinks --links -l Copies symbolic links as symbolic links.
    --copy-links -L Replaces symbolic links with the actual files or directories they point to.
    Permissions --perms -p Preserves file permissions during transfer.
    Timestamps --times -t Retains the original modification times of transferred files.
    Deletion --delete Removes extra files from the destination that do not exist in the source.
    --delete-excluded Deletes files at the destination that match exclusion patterns.
    Simulation --dry-run -n Performs a trial run without making any actual changes.
    Remote Shell --rsh=COMMAND -e Specifies a remote shell command for executing Rsync over a network.
    Size Control --max-size=SIZE Skips transferring files that exceed the specified size.
    --min-size=SIZE Ignores files that are smaller than the specified size.
    Partial Transfer --partial Retains partially transferred files in case of interruptions.
    File Filtering --exclude=PATTERN Excludes files that match a specific pattern.
    --include=PATTERN Ensures that files matching a pattern are not excluded.
    Output Format --progress Displays real-time progress updates during the transfer.
    -P Equivalent to using both --partial and --progress.
    --human-readable -h Formats output in an easily readable manner.
    Compression --compress -z Enables file compression during transfer.
    IP Version --ipv4 -4 Forces Rsync to use IPv4.
    --ipv6 -6 Forces Rsync to use IPv6.

    Transferring Local Files Using Rsync

    The following steps explain how to move files between directories using Rsync.

    To copy file1, file2, and file3 into a directory called dest, use the Rsync command. The last parameter in Rsync always refers to the destination, so be cautious to avoid unintentional overwrites.

    Copy Code 
    

    rsync file1 file2 file3 dest
    • file1 file2 file3: Files being copied.
    • dest: Target directory for storing the copied files.

    To confirm the transfer, use the ls command:

    Copy Code 
    

    ls dest/

    Expected output:

    ./dest/file1
./dest/file2
./dest/file3

    Copying Directories Locally

    By default, Rsync does not automatically copy all contents inside a directory. To copy directories recursively while maintaining file attributes such as timestamps, permissions, and symbolic links, use the -a (archive) flag.

    Copy the previously created dest directory:

    Copy Code 
    

    rsync -a dest/ dest-copy
    • -a: Enables archive mode for recursive copying while keeping file attributes intact.
    • dest/: Source directory (a trailing / ensures only contents are copied, not the directory itself).
    • dest-copy: Destination directory.

    Verify the copied directory using:

    Copy Code 
    

    ls dest-copy/

    Expected output:

    ./dest-copy/file1
./dest-copy/file2
./dest-copy/file3

    Excluding Files or Directories in Rsync

    Use the --exclude option in Rsync to avoid copying certain files or directories.

    The following command copies everything from the current directory while omitting file2 and any directory that starts with dest:

    Copy Code 
    

    rsync -a --exclude 'file2' --exclude 'dest*' . output
    • -a: Enables archive mode.
    • –exclude ‘file2’: Skips files named file2.
    • –exclude ‘dest*’: Excludes all files and directories that begin with dest.
    • .: Specifies the current directory as the source.
    • output: Target directory.

    Syncing Files to a Remote Server with Rsync

    To synchronize your files and directories with a remote system using SSH, follow these steps.

    Transfer Files to a Remote Machine

    Use the Rsync command below to copy files and directories to a remote server:

    Copy Code 
    

    rsync -hazvP --delete --stats . user@192.0.2.1:synced_from_local
    • -h: Displays file sizes in a human-readable format.
    • -a: Activates archive mode to retain file attributes.
    • -z: Compresses files during transfer to optimize speed.
    • -v: Enables verbose mode for detailed output.
    • -P: Shows progress updates and resumes partial transfers if interrupted.
    • –delete: Removes files from the destination that no longer exist in the source.
    • –stats: Provides a summary of transfer details.
    • .: Indicates the current directory as the source.
    • user@192.0.2.1:synced_from_local: Specifies the remote machine as the target destination, structured as follows:
      • user: Username for logging into the remote machine.
      • 192.0.2.1: IP address of the remote system.
      • synced_from_local: Destination directory on the remote machine.

    Accessing the Remote Server

    After syncing, connect to the remote machine via SSH to verify the transferred files:

    Copy Code 
    

    ssh user@192.0.2.1

    Checking Synced Files

    Run the following command on the remote server to confirm that the files were successfully transferred:

    Copy Code 
    

    ls synced_from_local/

    Expected directory structure after synchronization:

    synced_from_local/dest-copy/file1
synced_from_local/dest-copy/file2
synced_from_local/dest-copy/file3
synced_from_local/dest/file1
synced_from_local/dest/file2
synced_from_local/dest/file3
synced_from_local/file1
synced_from_local/file2
synced_from_local/file3
synced_from_local/output/file1
synced_from_local/output/file3

    Caution When Using the --delete Option

    Be careful when utilizing the --delete flag, as it will permanently remove files from the destination if they are not present in the source. To prevent unintended data loss, always conduct a test run before executing the command:

    Copy Code 
    

    rsync -a --delete --dry-run SRC DEST

    Conclusion

    In this tutorial, you installed Rsync, learned how to synchronize files locally and remotely, and used essential features like --exclude and recursive copying. Always double-check Rsync commands, particularly when using --delete. For more details, run:

    Copy Code 
    

    man rsync

    Source: vultr.com

    Create a Free Account

    Register now and get access to our Cloud Services.

    Try now

    Posts you might be interested in:

    Moderne Hosting Services mit Cloud Server, Managed Server und skalierbarem Cloud Hosting für professionelle IT-Infrastrukturen

    Real-Time Water Billing with PHP, Redis Pub/Sub & MySQL

    MySQL, Tutorial

    Linux file permissions with this comprehensive guide. Understand how to utilize chmod and chown commands to assign appropriate access rights, and gain insights into special permission bits like SUID, SGID, and the sticky bit to enhance your system’s security framework.

    Moderne Hosting Services mit Cloud Server, Managed Server und skalierbarem Cloud Hosting für professionelle IT-Infrastrukturen

    Create a PHP REST API with JSON on Ubuntu 20.04

    Tutorial, Ubuntu

    Linux file permissions with this comprehensive guide. Understand how to utilize chmod and chown commands to assign appropriate access rights, and gain insights into special permission bits like SUID, SGID, and the sticky bit to enhance your system’s security framework.