Many Linux commands like dd, cp, rsync, and mv are progressive but have no progress bars showing as they run. There is no built-in way to check the progress of ongoing operations except few commands that recently had commits ( dd command has status=progress from GNU coreutils version 8.24).
In this article, we’ll look at certain Linux tools and options which allow you to monitor the progress of data being transferred.
Read also: Pipe Viewer - A Tool to View Progress of Commands in Linux
1) PV Command
Pv is a command-line based tool that allows you to monitor the progress of data being transferred using pipe. The pv command gives you a visual view of the following information.
- Time elapsed
- Percentage completed using a progress bar.
- Current throughput rate.
- Total data transferred.
- ETA (Estimated Time).
Installing Pv
You can install pv on your distro using the following commands
Fedora, CentOS and RHEL
# yum install pv
Debian, Ubuntu and Linux Mint
# apt-get install pv
Gentoo Linux
# emerge --ask sys-apps/pv
FreeBSD Linux
# cd /usr/ports/sysutils/pv/ # make install clean
Pv Usage with Examples
The general syntax for pv is pv [Options] [File]
The -p option turns the progress bar on, -t option activates the total elapsed time pv has been running for, the -e option provides the ETA to completion based on the transfer rate and data size, the -r option shows the current rate of the transfer and finally the -b option shows the total amount of data transferred so far.
Monitor the progress of creating an archive using gzip
In this example, we use pv with gzip in the following example:
pv -tpre ~/Desktop/testfile.iso | gzip > ~/Desktop/testfile.gz
213MiB 0:00:14 [17.8MiB/s] [=========> ] 32% ETA 0:00:29
In this example, we provide the options and the file in question and then pipe it to gzip with a redirection to the file we want to archive.
Monitor the progress when copying a file with pv
$ pv ~/Desktop/testfolder/* > ~/Desktop/ 667MiB 0:00:13 [50.5MiB/s] [================================>] 100%
Using pv without any options will automatically run with the default p, t, e,r and b options.
Using pv and dialog commands to create a dialog progress bar
pv ~/Desktop/testfile.iso > ~/Desktop/destinationfile.iso | dialog --gauge "Progress" 10 70
2) Progress
Progress, formerly known as cv, is a tool that displays the percentage of copied data from coreutils basic commands like p, mv, dd, tar, gzip/gunzip, cat, etc. It can also provide additional information such as ETA and throughout.
Installing Progress
You can install progress on any distro with these steps.
sudo apt-get install libncurses5-dev (for Debian/Ubuntu) users.
# git clone https://github.com/Xfennec/progress.git # make # make install
Progress Usage with Examples
The general syntax for progress is watch progress -q. This lets you monitor all current and upcoming transfers in a terminal window. Now, any coreutils utility you invoke like cp and mv will be detected and indicated in the progress window.
Monitoring Downloads
progress -wc google-chrome
The -c option can be used to monitor a command name like google-chrome or firefox
3) Rsync
Rsync is used to sync files locally and remotely. Rsync can be used to copy files and also has the ability to show the progress without the use on an external application.
Installing Rsync
Fedora, CentOS and RHEL
# yum install rsync
Debian, Ubuntu and Linux Mint
# apt-get install rsync
Rsync Usage with Examples
You need to use rsync with the -P or --progress option to show progress during file transfer. The basic syntax for rsync is
rsync --progress source destination
Or
rsync -P source destination
To copy a zip file from the desktop to newfolder, enter:
$ rsync -P ~/Desktop/myfiles.zip ~/Desktop/newfolder 321,683,456 45% 3.17MB/s 0:01:56
To copy files from a /example/*.tar to a remote server called dummy@linoxide.com, enter:
rsync -av -P /example/*.tar dummy@linoxide.com:~
4) Advanced Copy
Advanced Copy, or advcpmv, is a mod for the cp and mv utilities which adds a progress bar and provides some information on ongoing transferred.
Installing Advanced Copy
You can install progress on any distro with these steps.
# wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.21.tar.xz # tar xvJf coreutils-8.21.tar.xz # cd coreutils-8.21/ # wget https://github.com/atdt/advcpmv/blob/master/advcpmv-0.5-8.21.patch # patch -p1 -i advcpmv-0.5-8.21.patch # ./configure # make
You might get this error, while running “./configure” command.
checking whether mknod can create fifo without root privileges... configure: error: in `/home/xxx/coreutils-8.21': configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check) See `config.log' for more details
Run the following command to fix that error and rerun the ./configure command again:
# export FORCE_UNSAFE_CONFIGURE=1
Replace the original cp and mv commands with these two new commands
# cp src/cp /usr/local/bin/cp # cp src/mv /usr/local/bin/mv
Advanced Copy Usage with Examples
The cp and mv commands stay same, the only change is adding the -g or -progress-bar option to the cp command. The -R option is used to copy directories recursively. Here is an example of the command:
$ cp -gR /home/linoxide /home/linoxide/Desktop
Or
# cp -R --progress-bar home/linoxide /home/linoxide/Desktop
We’ve seen a couple of tools and how they can be used to monitor progress on commands. The pv command is easy to set up and can monitor any command line activity. The progress utility is limited to only coreutils commands. Rsync has the inbuilt ability to monitor file transfers and advanced copy is a patch applied to only the cp and mv commands to show progress. Based on your trial, you can decide which of these is best for you.
