I have created this mre:
import subprocess
m = 'rsync'
url = 'mirror.nsc.liu.se/CentOS/7.8.2003/'
flags = '-avHP --delete '
destf = ''
url_prefix = 'rsync://'
root = '/tmp'
dest = 'test'
cmd = m + ' ' + flags + ' ' + url_prefix+url + ' ' +destf + ' ' + root+'/'+dest
print(cmd)
print(cmd.split(' '))
proc=subprocess.run(cmd.split(' '), capture_output=True)
print(proc.stderr)
Which gives me this output:
$ python3 test.py
rsync -avHP --delete rsync://mirror.nsc.liu.se/CentOS/7.8.2003/ /tmp/test
['rsync', '-avHP', '--delete', '--bwlimit=800', 'rsync://mirror.nsc.liu.se/CentOS/7.8.2003/', '', '/tmp/test']
b'Unexpected local arg: nIf arg is a remote file/dir, prefix it with a colon (:).nrsync error: syntax or usage error (code 1) at main.c(1368) [Receiver=3.1.3]n'
I really don’t understand what’s wrong here. There are no newlines in the string as far as I know.
If I change the creation of the command line to this (remove destf
), the problem still remains:
cmd = '' + m + ' ' + flags + ' ' + url_prefix+url + ' ' + ' ' + root+'/'+dest
But with this (also remove one space) it works:
cmd = '' + m + ' ' + flags + ' ' + url_prefix+url + ' ' + root+'/'+dest
If I replace ' '
with ' '
it stops working again. But when I enter the string directly in a shell, it works even with double spaces.
So what is actually happening here? I could use my findings above to shoehorn in a quick fix, but it feels like that will bite me later. So what is it I’m actually missing here? The reason I want the destf
parameter is that it is set to -P
when I’m using wget
and could possibly be something else for another command.
2
Answers
Thanks to Barmar for the answer that solved it. While I was out on lunch I realized what the problem was.
When a program is executed, it looks in a variable traditionally called
argv
which is basically an array of strings. The shell is usually responsible for converting a string into a string array, so if you invoke this command:then, the shell will parse the string into:
But with the approach I used, it instead would end up in:
So there was nothing wrong with the string. The problem was bypassing the conversion done by the shell. But using
.split()
instead of.split(' ')
solved it.It’s not complaining about a newline in the string. The error message has multiple lines, and the newline is the separator between the lines.
The unexpected argument is the empty string
''
before'/tmp/test'
becausedestf
is empty.When you use
.split(' ')
, multiple spaces are not merged into a single delimiter. So when you dothe result is
rather than
If you want multiple spaces to be coalesced into a single delimiter, use
.split()
instead of.split(' ')