Ubuntu – Docker image looses user file permissions if running on another machine

Seems like I’ve found an interesting bug. I build several images with custom (non-root) user inside them. When I use these images in the host I made them it works okay. But when I pull them to another hosts and run, the user files switches their permissions to root! For example, even files in user home are owned by root:

docker@minikube:~$ docker run -it --rm -u dreamdocs  pasaopasen/docutable-test-perm:latest bash
bash: /home/dreamdocs/.bashrc: Permission denied
bash-5.2$ 

Dockerfile

# syntax=docker/dockerfile:1

ARG SYSTEM=pasaopasen/redos7.3:2023-08-06_14-01

ARG USER=dreamdocs
ARG USER_ID=2122
ARG GROUP=dreamdocs
ARG GROUP_ID=515

FROM ${SYSTEM} as os-base
ARG USER
ARG USER_ID
ARG GROUP
ARG GROUP_ID

RUN dnf update -y && 
    dnf autoremove -y && 
    dnf clean all

RUN groupadd -r ${GROUP} -g ${GROUP_ID} && 
    useradd -u ${USER_ID} -g ${GROUP} ${USER} 

WORKDIR /home/${USER}
USER ${USER}
RUN touch file

To reproduce

$ docker pull pasaopasen/docutable-test-perm:latest
$ docker run -it --rm -u root pasaopasen/docutable-test-perm:latest  ls -l 
#### On your machine it will be
# total 0
# -rw-r--r--. 1 root root 0 Aug  6 23:28 file
#### On build machine it is 
# total 0
# -rw-r--r-- 1 dreamdocs dreamdocs 0 Aug  6 23:28 file

I can build them again in same way on another host and they will work okay in all my hosts, but images from my build-host (Ubuntu 20 LTS) work okay only there.

Diagnostics

Build host info:

$ uname -a
Linux serv 5.15.0-76-generic #83~20.04.1-Ubuntu SMP Wed Jun 21 20:23:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

$ docker version
Client:
 Version:           20.10.25
 API version:       1.41
 Go version:        go1.18.1
 Git commit:        20.10.25-0ubuntu1~20.04.1
 Built:             Fri Jul 14 22:00:45 2023
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.25
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.1
  Git commit:       20.10.25-0ubuntu1~20.04.1
  Built:            Thu Jun 29 21:55:06 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.2
  GitCommit:        
 runc:
  Version:          1.1.7-0ubuntu1~20.04.1
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:  

My other hosts cannot run the image correctly but can build images which will work okay on them all:

• Fedora 36

$ uname -a
Linux fedora 6.2.15-100.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 11 16:51:53 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

$ docker version
Client: Docker Engine - Community
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:53:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:52:13 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• Fedora 37

$ uname -a
Linux fedora 6.4.6-100.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jul 24 20:38:53 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

docker version
Client: Docker Engine - Community
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:37:21 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.5
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.6
  Git commit:       a61e2b4
  Built:            Fri Jul 21 20:35:43 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• Minikube VM

docker@minikube:~$ uname -a
Linux minikube 6.2.15-100.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 11 16:51:53 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
docker@minikube:~$ docker version
Client: Docker Engine - Community
 Version:           23.0.2
 API version:       1.42
 Go version:        go1.19.7
 Git commit:        569dd73
 Built:             Mon Mar 27 16:16:18 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.2
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.7
  Git commit:       219f21b
  Built:            Mon Mar 27 16:16:18 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• Astra Linux (Debian 10 based)

dreamdocs@astra:~$ uname -a
Linux astra 5.15.0-70-generic #astra2 SMP Mon Apr 10 13:35:08 UTC 2023 x86_64 GNU/Linux
dreamdocs@astra:~$ docker version
Client:
 Version:           20.10.2+dfsg1
 API version:       1.41
 Go version:        go1.15.9
 Git commit:        2291f61
 Built:             Thu Apr  6 14:43:28 2023
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.2+dfsg1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.15.9
  Git commit:       8891c58
  Built:            Thu Apr  6 14:43:28 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.13~ds1
  GitCommit:        1.4.13~ds1-1~deb11u2astra.se2+ci2
 runc:
  Version:          1.1.4+ds1
  GitCommit:        1.1.4+ds1-1
 docker-init:
  Version:          0.18.0
  GitCommit:

Notes

I did not find anything about this strange behaviour. Also the main problem is that I should build the images exactly on that Ubuntu host and I don’t have freedom to perform packages updates for packages that I maybe should not update

Notes:

• the problem is not with images transfer, I pulled from DockerHub and made archives on the server both
• the problem exists only with images based on my RedOS container, what is the tar of the part of the RedOS Linux system loaded to docker; so, I think the problem depends on base image but I don’t know what can be wrong there
• some weeks ago the earlier versions of these images worked well; I cannot certainly say what exactly have been changed except I removed some virtualization packages from the base image

EDIT 1

I save an image to archive by command: docker image save img | gzip > archive.tar.gz and download it. But, as I said, it does not matter because the problem is same for the image from DockerHub too