I'm trying to copy a directory tree recursively preserving hardlinks to the file. Using gnu cp, this would work with the -l flag. cp would then recreate the directory structure, but wouldn't need to copy the contents of each file.
This is preliminary to a backup, first I want to make a cheap (hardlinked) copy of the previous backup and then rsync the source directory over this copy. Roughly:
cp -r -l yesterdays_backup todays_backup rsync -a source_dir todays_backupUnfortunately, OSX's cp doesn't support the -l flag, as far as I can tell, cpio doesn't support recursive copying. The other alternative is pax, but that leads to the entire directory structure being copied:
pax -rw backups/yesterdays_backup backups/todays_backuptransforms:
yesterdays_backup | \source_dir (...)to:
todays_backup | \backups \yesterdays_backup \source_dir(...)There should be an easy/obvious way to do this, but I'm currently stumped... Any alternatives to cpio and pax? I'd like to avoid having to install gnu cp.
I'm aware of Timemachine, but that won't properly back up encrypted directories incrementally.
18 Answers
It is easy enough to install cp from MacPorts, however, if you don't want to, or want to create a portable script, then you have three options:
rsync
rsync --archive --link-dest=../yesterdays_backup backups/yesterdays_backup\ backups/todays_backupcpio
mkdir backups/todays_backup
cd backups/yesterdays_backup
find . -print | cpio -p -al ../todays_backuppax
mkdir backups/todays_backup
cd backups/yesterdays_backup
pax -rwl . ../todays_backupIt's easy to install the coreutils package from MacPorts which contains the GNU cp command renamed to gcp.
But even better, newer versions of rsync, including the one in OS X 10.5 at least, support the --link-dest option which should eliminate the need for the initial cp -al. See here. It's good practice to use the -E option, too, to copy extended attributes, ACLs, etc.
The macOS Finder copy does it right, preserving hard links, even to a different Volume. But only if it is a simple 1-item copy without joining.
- You see if it works right away, because if it does, the amount "to be copied" is the net size like it is reported by du. If the copy is for some reason not hardlink-preserving, you see a size too big and can stop immediately.
- If you need to copy or join parts of a more complicated folder structure, move the source into a parent folder used for the copy. Moving the copied items later on the destination to the right places obviously preserves the hard links.
I believe what you want can also be achieved with ditto yesterdays_backup todays_backup. By default, ditto does a recursive copy that preserves hard links, ACLs, and extended attributes.
Although cpio does not support recursive copying directly, you can feed the exact list of files you want to copy into its standard input, and achieve the effect of a recursive copy using find piped into cpio. This is directly analogous to using GNU tar with the options -T - (which means read the list of files to back up from standard input).
I wrote a Perl script to do the equivalent of GNU cp -rl:
#!/usr/bin/perl
use strict;
use warnings;
use Cwd;
use File::Path qw(make_path);
use Getopt::Long;
use File::Copy;
sub copylink_dir($ $);
my $USAGE = <<USAGE;
Usage: cprl source_dir destination_dir
\tDoes equivalent of Linux cp -rl from source_dir to destination_dir
USAGE
my ($verbose, $force);
#---------------------------------------------------------------------------
GetOptions( "v|verbose" => \$verbose, "f|force" => \$force,
);
#---------------------------------------------------------------------------
if($#ARGV == -1){ die $USAGE; }
my $source_dir=shift || die $USAGE;
my $destination_dir=shift || die $USAGE;
unless ($source_dir =~ m'^/') { $source_dir = getcwd() . '/' . $source_dir ;
}
unless ($destination_dir =~ m'^/') { $destination_dir = getcwd() . '/' . $destination_dir ;
}
if($verbose){ print "copy-recursive-link $source_dir -> $destination_dir\n"; }
if ( -e $destination_dir ) { if ( ! -d $destination_dir ) { die "ERROR destination $destination_dir exists and is not a directory\n"; } if ( ! $force ) { die "destination dir $destination_dir exists - use -f to force over-write\n" ; } if ( $verbose ) { print "Destination dir $destination_dir exists - forcing over-write\n" ; }
}
copylink_dir($source_dir, $destination_dir);
sub copylink_dir($ $) { my $dir=shift; my $dest=shift; # create destination dir if necessary: unless (-e $dest) { make_path($dest) or die "Error creating destination directory: $!\n"; } if($verbose){ print "DIR: $dir -> $dest\n"; } opendir(DIR,"$dir") or die "Cannot open $dir\n"; my @files = readdir(DIR); closedir(DIR); foreach my $file (@files) { next if $file =~ /^\.\.?$/; my $filepath = "$dir/$file"; if (-d $filepath) { copylink_dir($filepath, "$dest/$file"); } else { # else this is a file or symlink or something if($verbose){ print "FILE: $dir/$file -> $dest/$file\n"; } link("$dir/$file", "$dest/$file"); } }
}+1 for Pauls's answer. ditto would be an option, but only if used in conjunction with find - it does preserve hard links, but only inside directories.
Whereas cp -r -L fails on Mac, cp -R -L works just fine.
