How to achieve efficient file copying in Linux system?

As the amount of data continues to increase, the efficiency of file copying has become an inevitable problem in server operation and maintenance. There are many ways to copy files under Linux systems, but many users may not know which method is suitable for their own scenarios. In this article, we will introduce you to several common Linux file copy methods, analyze their advantages and disadvantages, and help you find the solution that suits you best.

I have tested all commands and scripts in my own environment, so you can use them directly in your daily work.

Usually everyone prefers scp because it is one of the native commands for file copying. However, the other commands listed in this article are also very useful, and I suggest you try them.

File copying can be easily done using the following four methods.

scp: Copy files between two hosts on the network. It uses ssh for file transfer and uses the same authentication method with the same security.
rsync: is a fast and versatile file copying tool. It can be copied locally, between other hosts via a remote shell, or with a remote rsync daemon.
pscp: is a program that copies files to multiple hosts in parallel. It provides many features, such as configuring encryption-free transfers for scp, saving output to files, and timeout control.
prsync: is also a program that copies files to multiple hosts in parallel. It also provides many features, such as configuring password-free transfers for ssh, saving output to files, and timeout control.

Method 1: How to copy files/folders from local system to remote system using scp command on Linux?

scp The command allows us to copy files/folders from the local system to the remote system.

I will copy the output.txt file from the local system to the /opt/backup folder of the 2g.CentOS.com remote system.

# scp output.txt root@2g.CentOS.com:/opt/backup

output.txt                                                                                              100% 2468    2.4KB/s  00:00

Copy the two files output.txt and passwd-up.sh from the local system to the /opt/backup folder of the remote system 2g.CentOs.com.

# scp output.txt passwd-up.sh root@2g.CentOS.com:/opt/backup

output.txt 100% 2468 2.4KB/s 00:00
passwd-up.sh 100% 877 0.9KB/s 00:00

Copy the shell-script folder from the local system to the /opt/back folder of the remote system 2g.CentOs.com.

This will copy all the files in the shell-script folder to /opt/back.

# scp -r /home/daygeek/2g/shell-script/ root@:/opt/backup/

output.txt 100% 2468 2.4KB/s 00:00
ovh.sh      100% 76 0.1KB/s 00:00
passwd-up.sh 100% 877 0.9KB/s 00:00
passwd-up1.sh 100% 7 0.0KB/s 00:00
server-list.txt 100% 23 0.0KB/s 00:00

Method 2: How to copy files/folders to multiple remote systems using scp command and shell script on Linux?

If you want to copy the same file to multiple remote servers, you need to create a small shell script like the one below.

And, the server needs to be added to the server-list.txt file. Make sure that after successful addition, each server should be on its own line.

Ultimately, you want the script to look like this:

# file-copy.sh

#!/bin/sh
for server in `more server-list.txt`
do
  scp /home/daygeek/2g/shell-script/output.txt root@$server:/opt/backup
done

After completion, set executable permissions for the file-copy.sh file.

# chmod +x file-copy.sh

Finally run the script to complete the copy.

# ./file-copy.sh

output.txt 100% 2468 2.4KB/s 00:00
output.txt 100% 2468 2.4KB/s 00:00

Use the following script to copy multiple files to multiple remote servers.

# file-copy.sh

#!/bin/sh
for server in `more server-list.txt`
do
  scp /home/daygeek/2g/shell-script/output.txt passwd-up.sh root@$server:/opt/backup
done

The following results show that all two files are copied to both servers.

# ./file-cp.sh

output.txt 100% 2468 2.4KB/s 00:00
passwd-up.sh 100% 877 0.9KB/s 00:00
output.txt 100% 2468 2.4KB/s 00:00
passwd-up.sh 100% 877 0.9KB/s 00:00

Use the following script to recursively copy folders to multiple remote servers.

# file-copy.sh

#!/bin/sh
for server in `more server-list.txt`
do
  scp -r /home/daygeek/2g/shell-script/ root@$server:/opt/backup
done

Output of the above script.

# ./file-cp.sh

output.txt 100% 2468 2.4KB/s 00:00
ovh.sh      100% 76 0.1KB/s 00:00
passwd-up.sh 100% 877 0.9KB/s 00:00
passwd-up1.sh 100% 7 0.0KB/s 00:00
server-list.txt 100% 23 0.0KB/s 00:00

output.txt 100% 2468 2.4KB/s 00:00
ovh.sh      100% 76 0.1KB/s 00:00
passwd-up.sh 100% 877 0.9KB/s 00:00
passwd-up1.sh 100% 7 0.0KB/s 00:00
server-list.txt 100% 23 0.0KB/s 00:00

Method 3: How to use pscp command to copy files/folders to multiple remote systems on Linux?

The

pscp command allows us to directly copy files to multiple remote servers.

Use the pscp command below to copy a single file to the remote server.

# pscp.pssh -H 2g.CentOS.com /home/daygeek/2g/shell-script/output.txt /opt/backup

[1] 18:46:11 [SUCCESS] 2g.CentOS.com

Use the pscp command below to copy multiple files to the remote server.

# pscp.pssh -H 2g.CentOS.com /home/daygeek/2g/shell-script/output.txt ovh.sh /opt/backup

[1] 18:47:48 [SUCCESS] 2g.CentOS.com

Use the pscp command below to recursively copy the entire folder to the remote server.

# pscp.pssh -H 2g.CentOS.com -r /home/daygeek/2g/shell-script/ /opt/backup

[1] 18:48:46 [SUCCESS] 2g.CentOS.com

Use the pscp command below. Use the command below to copy a single file to multiple remote servers.

# pscp.pssh -h server-list.txt /home/daygeek/2g/shell-script/output.txt /opt/backup

[1] 18:49:48 [SUCCESS] 2g.CentOS.com
[2] 18:49:48 [SUCCESS] 2g.Debian.com

Use the pscp command below to copy multiple files to multiple remote servers.

# pscp.pssh -h server-list.txt /home/daygeek/2g/shell-script/output.txt passwd-up.sh /opt/backup

[1] 18:50:30 [SUCCESS] 2g.Debian.com
[2] 18:50:30 [SUCCESS] 2g.CentOS.com

**
Use the following command to recursively copy a folder to multiple remote servers. **

# pscp.pssh -h server-list.txt -r /home/daygeek/2g/shell-script/ /opt/backup

[1] 18:51:31 [SUCCESS] 2g.Debian.com
[2] 18:51:31 [SUCCESS] 2g.CentOS.com

In general, it is not difficult to achieve efficient file copying in Linux systems. As long as you select the corresponding tools or commands according to actual needs, you can improve the speed and efficiency of file transfer. In this article we introduce several common file copy methods and analyze their advantages and disadvantages. I hope this article can help Linux system administrators and allow everyone to better understand the principles and methods of file copying.

The above is the detailed content of How to achieve efficient file copying in Linux system?. For more information, please follow other related articles on the PHP Chinese website!