TransWikia.com

rsync copies the current directory when I don't pass an option to my script

Unix & Linux Asked by rm1948 on December 8, 2020

Below is a script on Ubuntu 18.04 used to copy files from one directory to another. All works fine except the files from the current directory (/media/rmerriam/Backup/rsync_backup) are copied to the destination. They shouldn’t be copied at all as they are never in the source path.

Manually executed (with and without -s makes no difference):

rmerriam@mysticlakelinux:/media/rmerriam/test/rsync_backup$ rsync -auv -s ../src/ ../dest
sending incremental file list

sent 59 bytes  received 12 bytes  142.00 bytes/sec
total size is 0  speedup is 0.00
rmerriam@mysticlakelinux:/media/rmerriam/test/rsync_backup$ ls ../dest

Executed script without -s:

===========================
./merge.sh
 
 ../src/
 ../dest
rsync -auv  ../src/ ../dest
sending incremental file list
./    <====== added why???
merge.sh
merge_all.sh
merge_backups.sh

sent 3,124 bytes  received 76 bytes  6,400.00 bytes/sec
total size is 2,835  speedup is 0.89
rmerriam@mysticlakelinux:/media/rmerriam/test/rsync_backup$ ls ../dest
merge_all.sh  merge_backups.sh  merge.sh

Executed script with -s option:

rmerriam@mysticlakelinux:/media/rmerriam/test/rsync_backup$ ./merge.sh ../src/ ../dest -s
===========================
./merge.sh
 -s
 ../src/
 ../dest
rsync -auv -s ../src/ ../dest
sending incremental file list
./

sent 66 bytes  received 19 bytes  170.00 bytes/sec
total size is 0  speedup is 0.00
rmerriam@mysticlakelinux:/media/rmerriam/test/rsync_backup$ ls ../dest

Result: manual execution and -s option didn’t move the $PWD files. Without the -s the script copied the files.

The script without the -s added the ./ directory as a source directory. Why?

The command lines are shown by the echo and they are identical. Why different behavior in a script?

Here’s the script:

#!/bin/bash

# base script for copying files to the common directory.
# only copies newer files

printf '===========================n'
printf "$0n $3n $1n $2n"

echo rsync -auv "$3"    "$1" "$2"
rsync -auv "$3"    "$1" "$2"

For reference : rsync version 3.1.2 protocol version 31

2 Answers

Went looking at issues on rsync and found "Empty quotes adds cwd to SRC directories". I have a $3 argument on the script for adding extra options. Often it is empty so this is the culprit. Adding the -s made the argument non-empty so avoided the issue.

This manual run confirms the issue:

rsync ""  -auv ../src/ ../dest
sending incremental file list
created directory ../dest
./
merge.sh
merge_all.sh
merge_backups.sh

sent 3,124 bytes  received 106 bytes  6,460.00 bytes/sec
total size is 2,835  speedup is 0.88

Correct answer by rm1948 on December 8, 2020

There's a bug in your script. Arguably there's also a bug in rsync, because it's silently doing something surprising when you pass it weird data, whereas it would be better for it to error out. But either way your script doesn't work.

The problematic line is:

rsync -auv "$3"    "$1" "$2"

When the third argument of the script ("$3") starts with a -, this is an option for rsync. But when the third argument does not start with a -, which includes the case where the script only has two arguments (or less), this is a non-option argument. Rsync accepts multiple source directories, so when "$3" doesn't start with a -, you're asking it to copy "$3" and "$1" to "$2". For example,

./merge.sh ../src/ ../dest

tells rsync to copy both the empty string and ../src/ to ../dest.

Most applications reject the empty string when they're looking for a file name. However, in so far as the empty string is accepted as a file name, it means the current directory. It's coherent with how relative paths are transformed into absolute paths: if a path is relative (doesn't start with /), then prepend the absolute path to the current working directory and a slash, and you get an equivalent absolute path. If you apply this to the empty string, which is a relative path since it doesn't start with a slash, you get the current working directory with a slash at the end, which is the current working directory.

Rsync accepts the empty string as meaning the current directory. So it's doing exactly what you're telling it.

Arguably, passing the empty string as an argument is more likely to be a mistake than something deliberate. The shortest widely acceptable way of specifying the current directory is . (a dot). So arguably rsync should print an error message saying that the argument is invalid and error out.

You want your script to accept an optional extra option, but the way you've implemented it, the extra option is mandatory. To make it optional, check whether the script has a third argument.

if [ $# -lt 2 ]; then
  echo >&2 "$0: too few arguments (SRC and DEST are required)"
  exit 120
elif [ $# -eq 2 ]; then
  rsync -auv "$1" "$2"
elif [ $# -eq 3 ]; then
  rsync -auv "$3" "$1" "$2"
else
  echo >&2 "$0: too many arguments (only one OPTIONS argument is permitted)"
  exit 120
fi

This is a weird interface though: why accept at most one option? You might as well accept an arbitrary number of options (why can't I pass -q --dry-run?). It would be both simpler for the user to understand and simpler to implement. Furthermore, the convention is to pass options before non-option arguments, not after. Once the interface is simplified to accept an arbitrary number of options, you can simplify the code to

rsync -auv "$@"

Answered by Gilles 'SO- stop being evil' on December 8, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP