How to Merge Directory Trees in Linux using cp Command

How to merge two directory trees with similar layout into a third directory?. Let us consider the following example to understand the problem.

Suppose two directories dir1 and dir2 have 3 sub-directories a, b and c in each of them. The directory layout is like below:

Layout of input directories

These directories a, b and c have some files in them. The output of tree command will illustrate better:

Layout of files

1) Using cp to create merge

Now we want to merge these two directories into a third directory, say "merged".
The simplest thing that you can do to achieve this is to copy recursively the directories like below:

Copy directories recursively to create new merge

1.1 Problem with cp command and alternative

The problem with this approach is that the files created inside merged directory are copy of original files, and not the original files themselves. But wait, (you might be asking yourself) what is the problem if the files are not original? So to answer your question, consider the situation where you have large number of bulky files. In that case, copying all the files might take hours.

Now let's get back and try the same with mv command instead of cp.

Attempt to merge with mv command

The directories are not merged. So we cannot use mv command to merge directories like this.
Now how can you keep the original files inside "merged" directory?

2) The solution

The cp command has a very useful option to draw us out of this situation. The -l or --link option to cp aommand creates the hard links instead of copying the files themselves. Let us try with that.

Before trying out the hard link option to cp command, let us print the inode number of the original files.
The tree command has option to print the inodes with --inodes option:

Display inodes of original files

Now we have the inodes listed here, we can proceed to creating the hard links with --link option to cp command:

Merge directories with hard links

2.1 Verify the files:

Now the files are copied, let us verify if the inodes match with original files:

Verify Inodes

2.2 Cleanup

As you can see that the files have same inodes as original files. Now the problem is solved and we have the original files inside merged directory. We can now cleanup by removing the directories dir1 and dir2.

Remove original directories
Read Also:

1 thought on “How to Merge Directory Trees in Linux using cp Command”... add one

  1. Thanks for this interesting post! In this example it is considered all individual files exist just once. e.g. "file1" in directory "a" exists only in directory "a". What about when a given file exists in more than two subtrees whereas the subtree names are identical? e.g. /dir1/x/file42.txt and /dir2/x/file42.txt . Consider both files have identical content and can therefore be considered identical. In this case either one of these files should be considered, optionally the first occurrence of the file should be considered. How would merging work best in that scenario? I am asking this, as I have regularly dumped photos form various sources (cameras, phones) into new subtrees whenever I have made a backup. The subtrees got a unique name composed of date and time info.

    Reply

Leave a Comment