Permission denied when acl says rwx – CentOS

Alas, chmod 777 /tmp in the console is not the answer. It is much more complicated than that. So I wasn’t silly at all.

How to reproduce?

In my case, I have 2 docker PHP containers writing to a number of case specific log files, one of them using apache, the other nginx. Both have a mapping /tmp:/tmp, so we can look at a file from the host or from inside a container.

From inside the containers, the owner is apache:apache if created in the apache container or nginx:nginx in the nginx container, but from the host’s perspective it is systemd-timesync:systemd-journal for the same file in both cases.

Of course, the apache container does not have a user nginx and vice versa (neither does the host). So if my PHP script wants to write to a file created by the other container, I have said permission problem.

Fix

The remedy is easy, if at creation time the owner is changed to nobody and permissions to 666.

If not, we cannot change the owner from the other container and get failed to open stream: Permission denied. See https://serverfault.com/questions/772227/chmod-not-working-correctly-in-docker (also see the discussion about changing permissions from host via system call below).

So I wrote a wrapper function str_to_file to add this manipulation:

function str_to_file($str, $filename= "/tmp/tmp.txt", $mode="a+") {
    if (!file_exists($filename)){
        touch($filename);           # create file
        chmod($filename, 0666);     # the owner can chmod
        chown($filename, 'nobody'); # the owner can chown
    } # if (!file_exists($filename))

    $mode = $mode == 'w'
        ? LOCK_EX
        :  FILE_APPEND | LOCK_EX;

    file_put_contents($filename, $str . PHP_EOL, $mode);
} # str_to_file

This works for both the apache and nginx containers.

Explanation

How did I get into this mess anyway?

It looks like either I didn’t mess it up when on CentOS, so in this case most probably it has nothing to do with my switch to Ubuntu. I cannot remember what I did but I don’t remember I had this kind of permission problem on CentOS.

Or CentOS handles files created in a container differently than Ubuntu. I am sure I never saw a user systemd-timesync:systemd-journal on CentOS. Unfortunately I cannot investigate without too much effort what user CentOS would substitute from the host point of view and which consequences this has.

Oh wait, I remember I do have another server running PHP and nginx on CentOS. Creating a file in /tmp from within the container reveals that the owner from both inside the container and from the host is nobody. So there is a significant difference between both Linux versions.

What’s more, this insight explains why I experienced this permission problem only after the switch from CentOS to Ubuntu, to which I was forced due to the fact that my provider does not offer CentOS anymore for obvious reasons.

By the way, I first tried the CentOS substitutes AlmaLinux and RockyLinux with bad results of different nature which finally forced me to use Ubuntu. This switch in turn revealed lots of problems, this one being the last, which cost me several weeks so far. I hope that this nightmare ends now.

Acting from cron, the apache version is and was used exclusively, so no problem here.

Testing from the browser lately, I switched by chance and for no particular reason from the apache version to the nginx version and vice versa, creating those described problems.

Actually I don’t know why the containers write on behalf of those users. The user of the container is root in both cases, as is standard with docker. I guess it is the browser engine which introduces these users.

Interestingly, when trying to change permissions and ownership via system call after creation, I failed, if I remember correctly, although the user then should be root and root should be capable of doing that.

It turns out that on apache the system user [system(‘whoami’)] is not root but apache, but the file owner is apache where as on nginx the system user is nobody and the file owner is nginx. So changing permissions with system should work on apache [system("chmod 0666 $filename"); system("chown nobody:nobody $filename");], but not on nginx. Alas, it does not work on apache either.

Quick check in apache container:

/tmp # whoami
root
/tmp # ls -la *6_0.ins
-rw-r--r--    1 apache   apache         494 Nov 14 18:25 tmp_test_t6_0.ins
/tmp # su apache chmod 0666 tmp_test_t6_0.ins
This account is not available

Sorry, I can’t understand this. And no idea about ACL.

apache vs. nginx

Why do I use both web server versions in the first place?

Well, the process I invoke processes random data which may take a long time, so chances are that nginx times out. This is a well-known nginx feature.

Despite all my studies and obvious instructions, I could not manage nginx to behave. Geez!

Finally, as an appropriate workaround, I introduced apache, which does not have this problem. For decent runtimes it makes no difference, of course.