When you work on some processes, sometimes you need to visualize the progression of file operations. Some commands have a parameter to help you to see the progression but for those which don't have it, you need to use another method. To mitigate this problem, you can use a pipe viewer which indicates the progression of the different processes that you launch.
What is pipe viewer
Pipe viewer (pv) is a CLI tool which allows you to monitor the progress of data through a pipeline by giving information such as percentage completed, ETA, total data transferred, time elapsed, etc. If you have some processes, you can insert the pipe viewer as a normal pipe between the different processes to have a visual indication of the progression.
In this article, I will show you the Unix pipe viewer utility in Node.js proposed by Rocco Musolino on his Github page. It only uses two parameters:
-s, --size <size>: Assume the total amount of data to be transferred is SIZE. You can provide a size in bytes or using units (
b, kb, mb, gb, tb).
-N, --name <name>: Prefix the output information with NAME.
The Unix pv of Rocco can be installed through node.js as below:
# npm install -g pv /usr/bin/pv -> /usr/lib/node_modules/pv/bin/pv.js + firstname.lastname@example.org added 15 packages in 0.953s
Examples of pipe viewer usage
In this part, I will show you some different examples which can help you to use the pv tool
a) Display a dd copy progression
You can display the progression of a file copying when using the dd command
# dd if=ubuntu-gnome-17.04-desktop-amd64.iso | pv | dd of=dest/ubuntu17 0.00% | 0 ETA | 1.36GB Transferred | 95.71MB/s2964416+0 records in 2964416+0 records out 1517780992 bytes (1.5 GB) copied, 14.7174 s, 103 MB/s 100.00% | 0 ETA | 1.41GB Transferred | 96.5MB/s2964416+0 records in 2964416+0 records out 1517780992 bytes (1.5 GB) copied, 14.7301 s, 103 MB/s
b) A cat command progression
You can decide to display the progression of the cat command when displaying the content of message log file for example
# cat /var/log/messages | pv Oct 13 03:36:02 localhost journal: Runtime journal is using 6.1M (max allowed 49.4M, trying to leave 74.1M free of 488.4M available → current limit 49.4M). Oct 13 03:36:02 localhost kernel: Linux version 4.9.36-x86_64-linode85 (email@example.com) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 SMP Thu Jul 6 15:31:23 UTC 2017 ........ ........ 100.00% | 0 ETA | 114.83KB Transferred | 114.83KB/s
c) Display the progression while compressing
During some operations of compression, we can need to display the progression. It is possible as below
# tar -cf - /var/log/ | pv -N backlog | tar -C . -x tar: Removing leading `/' from member names backlog: 100.00% | 0 ETA | 660KB Transferred | 660KB/s
d) Visualize a live ssh network
You can connect to a host via ssh and displays the live transfer speed. In our case, we will direct all transferred data to
/dev/null with a local test
# yes | pv | ssh paul@localhost "cat > /dev/null" The authenticity of host 'localhost (::1)' can't be established. ECDSA key fingerprint is SHA256:lqes2c9Obi9lCSepwdT8uovcsmhcboUAsylF69MENDE. ECDSA key fingerprint is MD5:38:67:a1:25:96:7b:69:02:f6:b8:ff:5d:ce:03:fd:a0. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts. paul@localhost's password: 0.00% | 0 ETA | 800.84MB Transferred | 17.04MB/s
The Unix pipe viewer of Rocco Musolino doesn't display the percentage completed with the progress bar as the "normal" pipe viewer installed through the package. We can see that the pipe viewer is interesting when we face commands which don't allow to visualize the progression of the processes.