Machma - Enables to Run Multiple Commands Parallel in Linux

Posted on : May 24, 2017 , Last Updated on : May 26, 2017 By in LINUX HOWTO
| Reply More

machma run command parallelIn order to fully utilize computing power, jobs can be run in parallel. For example, re-sizing of images sequentially takes a lot of time, On the other hand re-sizing of multiple images in parallel makes better use of multi-core CPU and therefore is much faster. Machma makes it very easy to execute tasks in parallel and provides live feedback. In the case of errors or lines printed by the program, the messages are tagged with the job name.

An earlier article we have seen how we can execute commands parallel in a different host using pdsh tool. This article covers installation of machma - a tool for easy parallel execution of commands with live feedback in Ubuntu 16.10

Installation

Machma needs Go Lang , Install it using following commands in the terminal

 # wget https://storage.googleapis.com/golang/go1.7.1.linux-amd64.tar.gz
 # tar -zxvf  go1.7.1.linux-amd64.tar.gz -C /usr/local/
 # mkdir work
 # vi .profile
 export GOPATH=~/work
 export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin
 # go version
 go version go1.7.1 linux/amd64

Next install it using go get, y You also need git to get machma from github.

 # apt-get install git
 # go get github.com/fd0/machma
 # machma --help

Usage of Machma

machma by default reads newline-separated values and replaces all command-line arguments set to {} with the file name. The number of jobs is set to the number of cores for the CPU on which it is running.

Suppose you want to ping a large number of hosts. Write all the IP/domain you want to ping in a text file and pass the filtered output to machma. The -p option specifies the number of program. if you have 4 cores then specify the value of -p option to 4.

 # cat /tmp/ip_address.txt | machma -p 2 -- ping -c 2 -q {}

A slightly more advance example is the following, which does the same thing but reduces the output printed by ping.

 # cat /tmp/ip_address | machma -- sh -c 'ping -c 2 -q $0 > /dev/null && echo alive' {}

Re-size all images for the current directory recursively to 1200x1200 pixel at most. The mogrify command is a part of imagemagic that you need beforehand to execute the following command. You can install imagemagic using apt-get install imagemagic.

 # find . -iname '*.jpg' | machma --  mogrify -resize 1200x1200 -filter Lanczos {}

You can limit the time mogrify is allowed to run per image using --timeout When the timeout is reached the program gets canceled.

# find . -iname '*.jpg' | machma --timeout 5s --  mogrify -resize 1200x1200 -filter Lanczos {}

If the file name have spaces, use --null option that instructs machma to read items separated by null bytes from stdin.

# find . -iname '*.jpg' -print0 | machma --null --  mogrify -resize 1200x1200 -filter Lanczos {}

Conclusion

Machma is a nice tool for performing batch operation of the same command. It takes advantages of computing power to execute task in parallel. In this article, we have seen how to use it by pinging a group of remote host and also to resize images at the same time. You can use it in other scenarios where you need to execute a command repetitively.

Filed Under : LINUX HOWTO, OPEN SOURCE TOOLS

Free Linux Ebook to Download

Leave a Reply

All comments are subject to moderation.