How to Setup lsyncd for Synchronize Directories to Remote Computers

November 3, 2016 | By in LINUX HOWTO
| Reply More

If you want some directory of yours actively backed after every change, they are a lot of sync options to choose from. Software like csync, rsync, csync2, lsyncd and many others will all do the job, but in this article we will only review one of them. We will introduce features of lsyncd, a synchronization daemon that enables you to mirror your designated directory with to any other directory on the network or locally. To save network and disk bandwidth, it only mirror changes to your directory. So lets start.

Installing lsyncd

It is present in Ubuntu repositories so we can install it by simple apt command

sudo apt install lsyncd

For CentOS you first need to install EPEL repository because lsyncd is there.

yum install epel-release

And next we can install Live Syncing Daemon which is actual long name of the program

yum install lsyncd

It will not configure anything to sync, you need to do this by yourself and we will show how. Sample configuration files live in /usr/share/doc/lsyncd/examples/ directory in ubuntu and /usr/share/doc/lsyncd-2.1.5/examples/ in centos so lets check what is there

miki@ThinkPad-X220T:~$ ls /usr/share/doc/lsyncd*/examples/
lbash.lua lgforce.lua lpostcmd.lua lrsyncssh.lua
lecho.lua limagemagic.lua lrsync.lua

Those are sample config files for lsyncd. They are all written in lua same as lsyncd and they are interpreted when sync job is ran. Lets first create directories with which we work.

Setting up local syncing with lsyncd

First we create the directory which we will sync from, that is, the source directory:

sudo mkdir -p /linoxide/sync/

And then the directory which will be a backup of this one, that is, the target directory

sudo mkdir /backup

Next lets populate the source directory with some files

sudo touch /linoxide/sync/testfile{1..25}

We would want to have some logging, so lets make file for log and status

sudo mkdir /var/log/lsyncd
sudo touch /var/log/lsyncd/lsyncd.{log,status}

Next we need to make a dir in /etc directory for the configuration file

sudo mkdir /etc/lsyncd

There we create new configuration file from scratch

sudo nano /etc/lsyncd/lsyncd.conf.lua

In this config file you can paste the below code.

settings = {
        logfile = "/var/log/lsyncd/lsyncd.log",
        statusFile = "/var/log/lsyncd/lsyncd.status"
}

sync {
        default.rsync,
        source = "/linoxide/sync/",
        target = "/backup",
}

Then we can go to target directory and check of something is there

cd /backup/

ls

It should give you no output. When we restart the lsyncd service, it should immediately sync with the directory.

sudo systemctl restart lsyncd

miki@ThinkPad-X220T:/backup$ ls
testfile1 testfile14 testfile19 testfile23 testfile5
testfile10 testfile15 testfile2 testfile24 testfile6
testfile11 testfile16 testfile20 testfile25 testfile7
testfile12 testfile17 testfile21 testfile3 testfile8
testfile13 testfile18 testfile22 testfile4 testfile9

The sync is working. Lets then add some more files to our source dir, to se if it syncs  on the fly.

miki@ThinkPad-X220T:/backup$ sudo touch /linoxide/sync/new{1..9}

It doesn't sync every second for performance reasons, so we need to wait some time and then type ls in our /backup folder to see if new files are added

miki@ThinkPad-X220T:/backup$ ls
new1 new6 testfile10 testfile15 testfile2 testfile24 testfile6
new2 new7 testfile11 testfile16 testfile20 testfile25 testfile7
new3 new8 testfile12 testfile17 testfile21 testfile3 testfile8
new4 new9 testfile13 testfile18 testfile22 testfile4 testfile9
new5 testfile1 testfile14 testfile19 testfile23 testfile5

The files are there.

Syncing to remote location

If you want your target or source directory to be on remote machine, you first need to setup SSH keys for passwordless ssh login. If you didn't already create your ssh key, do so now by following command. Make sure you do it as root as lsyncd will also try to log in as root. Using normal user will not work so we use su

su

ssh-keygen -t rsa

You press enter to all prompts, and when it is finished, you copy the key to target machine by following command

ssh-copy-id root@159.203.161.112

Off course, change the IP address for your target machine. It will prompt you for the password on that machine for the last time, and from now on you will log in there without password. So  lets login to our remote server:

ssh root@159.203.161.112

There we will create a directory that will be our remote target:

mkdir /remotebackup

On our source machine that is running lsyncd we need to edit config file to make it look like this

settings = {

insist = true,
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status"
}

sync {
default.rsyncssh,
source = "/linoxide/sync/",
host = "159.203.161.112",
targetdir = "/remotebackup"

Notice new host line. You need your own server IP and don't copy above code as formatting might be wrong due to wordpress theme. Instead use the previous code and make manual changes. Next cd into /remotebackup on the server and check if it is empty. Then restart the lsyncd on your source machine with systemctl restart lsyncd command and look if it syncs on the server.

ls

syncing works

We see that it works.

Conclusion

We have set up local and remote syncing with live syncing daemon, but that is just scratch on the surface of what lsyncd can do. If you look at /usr/share/doc/lsyncd/examples/ directory for example configurations, you can see that this program can be used for example to continuously fix user permissions on all files in some directory and more. This is all for this article, thank you for reading and have a good day.

Filed Under : LINUX HOWTO, OPEN SOURCE TOOLS

Tagged With :

Free Linux Ebook to Download

Leave a Reply

Commenting Policy:
Promotion of your products ? Comment gets deleted.
All comments are subject to moderation.