Question 417974 in Ubuntu Question posted in
The official Ubuntu documentation can be found here.

Ubuntu – Python script can't move file between mounted docker volumes while mv command could

I think I did configure docker volumes correctly, but I can’t understand why mv command success called from interactive bash shell.

FROM ubuntu:rolling

RUN apt update

RUN apt install -y python3 python3-venv

USER ubuntu

$ docker run --rm --interactive --tty --volume .:/home/ubuntu/data --volume $HOME/Downloads:/home/ubuntu/downloads --workdir /home/ubuntu/data vntk2:latest bash

>>> import pathlib
>>> pathlib.Path('/home/ubuntu/downloads/SoftwareEngineer.pdf').exists()
True
>>> pathlib.Path('/home/ubuntu/downloads/SoftwareEngineer.pdf').rename(pathlib.Path('/home/ubuntu/data/'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.12/pathlib.py", line 1365, in rename
    os.rename(self, target)
OSError: [Errno 18] Invalid cross-device link: '/home/ubuntu/downloads/SoftwareEngineer.pdf' -> '/home/ubuntu/data'
>>> 

ubuntu@d3d1c2e286c2:~/data$ mv /home/ubuntu/downloads/SoftwareEngineer.pdf /home/ubuntu/data/ 
ubuntu@d3d1c2e286c2:~/data$ ls -lah /home/ubuntu/data/
total 820K
drwxrwxr-x 8 ubuntu ubuntu 4.0K Dec  8 00:03  .
drwxr-x--- 1 ubuntu ubuntu 4.0K Dec  8 00:02  ..
-rw-rw-r-- 1 ubuntu ubuntu 765K Nov 15 12:14  SoftwareEngineer.pdf
ubuntu@d3d1c2e286c2:~/data$
Tags:

2

Answers


  1. 0

    Have your tried os.replace() or shutil.move()?

    Login or Signup to reply.
  2. 0

    There are two ways to move one file to another. One is to leave the underlying bits in place and change the directory entry. The other is to copy all of the bytes from the original file to a new file, and then delete the old file. Just renaming the existing file is much faster, but it will only work if the source and destination are on the same filesystem.

    In your case, the two directories /home/ubuntu/data and /home/ubuntu/downloads come from different volume mounts, so for purposes of this rule they aren’t they same filesystem. (If you ran mount(8) in a debugging shell, you’d see them as separate, for example.) That means you can’t just rename the existing file. That’s also what the Invalid cross-device link error means.

    In code, you’re calling Path.rename() which "is implemented in terms of os.rename()". That function notes:

    The operation may fail if src and dst are on different filesystems. Use shutil.move() to support moves to a different filesystem.

    shutil.move() is documented as understanding this: if it’s possible to use os.rename() it does, and if it’s not then it does the slow file copy. mv(1) can do the same thing, which is why that works. shutil.move() also accepts pathlib.Path objects, so it should work to change your code

    from pathlib import Path
import shutil

origin = Path('/home/ubuntu/downloads/SoftwareEngineer.pdf')
destination = Path('/home/ubuntu/data/')
shutil.move(origin, destination)  # not origin.rename()

    (Also note that, if the primary goal of your application is to deal with files that exist on the host system, Docker introduces a lot of challenges here – different path names, user ID problems, this particular filesystem issue. It tends to be much easier to avoid containers if you need to work directly on host files.)

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search