I’m getting this error when pulling from a self-hosted git remote I just added.
$ git pull myremote master
fatal: cannot exec 'pack-objects': Permission denied
fatal: git upload-pack: unable to fork git-pack-objects
fatal: The remote end hung up unexpectedly
fatal: protocol error: bad pack header
Googling "cannot exec 'pack-objects': Permission denied
" with quotes got zero results, so this is not a duplicate. There are several questions about the fatal: protocol error: bad pack header
part, with answers like this one proposing adding a few memory-limiting lines to .gitconfig
. That didn’t solve my problem.
Based on the first line, I reckon some pack-objects
executable somewhere is missing its x
permission. But it’s not this one:
$ ls -l $(which git-upload-pack)
-rwxr-xr-x 1 root root 1559256 Apr 20 10:20 /usr/bin/git-upload-pack
For context, the remote is a VPS running Debian 10, while the client is an ancient Intel Celron desktop circa 2003, running Ubuntu 18.04. So resource limitation on the pulling machine could be playing a role.
EDIT:
All the git binaries are executable by all users:
$ ls -l /usr/bin/git*
-rwxr-xr-x 1 root root 2759388 Apr 20 10:20 /usr/bin/git
lrwxrwxrwx 1 root root 3 Apr 20 10:20 /usr/bin/git-receive-pack -> git
-rwxr-xr-x 1 root root 1546968 Apr 20 10:20 /usr/bin/git-shell
lrwxrwxrwx 1 root root 3 Apr 20 10:20 /usr/bin/git-upload-archive -> git
-rwxr-xr-x 1 root root 1559256 Apr 20 10:20 /usr/bin/git-upload-pack
And /usr/bin
is in my PATH:
$ echo $PATH
/home/keith/bin:/home/keith/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
And I can run them without sudo:
$ git-upload-pack
usage: git upload-pack [<options>] <dir>
--stateless-rpc quit after a single request/response exchange
--advertise-refs exit immediately after initial ref advertisement
--strict do not try <directory>/.git/ if <directory> is no Git directory
--timeout <n> interrupt transfer after <n> seconds of inactivity
EDIT 2:
The .git
dir appears to have correct permissions too:
$ ls -la .git{,/objects}
.git:
total 56
drwxr-xr-x 8 keith keith 4096 Sep 21 11:36 .
drwxr-xr-x 22 keith keith 4096 Sep 21 13:09 ..
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 branches
-rw-r--r-- 1 keith keith 290 Sep 21 11:36 config
-rw-r--r-- 1 keith keith 73 Jul 25 2017 description
-rw-r--r-- 1 keith keith 0 Sep 21 12:51 FETCH_HEAD
-rw-r--r-- 1 keith keith 23 Jul 25 2017 HEAD
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 hooks
-rw-rw-r-- 1 keith keith 1555 Sep 21 10:47 index
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 info
drwxr-xr-x 3 keith keith 4096 Jul 25 2017 logs
drwxr-xr-x 37 keith keith 4096 Jul 25 2017 objects
-rw-rw-r-- 1 keith keith 41 Apr 2 2018 ORIG_HEAD
-rw-rw-r-- 1 keith keith 46 Sep 21 10:47 packed-refs
drwxr-xr-x 5 keith keith 4096 Jul 25 2017 refs
.git/objects:
total 148
drwxr-xr-x 37 keith keith 4096 Jul 25 2017 .
drwxr-xr-x 8 keith keith 4096 Sep 21 11:36 ..
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 06
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 0a
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 17
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 22
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 29
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 2d
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 30
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 32
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 35
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 3c
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 3f
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 43
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 4f
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 50
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 53
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 5a
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 5f
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 6e
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 72
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 82
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 96
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 a5
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 a8
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 b4
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 b7
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 b9
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 bc
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 c0
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 c3
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 c9
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 cd
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 e7
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 ea
drwxr-xr-x 2 keith keith 4096 Jul 25 2017 info
drwxr-xr-x 2 keith keith 4096 Apr 2 2018 pack
EDIT 3:
I don’t have SELinux installed so I don’t know what context to check except linux permissions.
I looked at /usr/lib/git-core
. All the files (executables and symlinks) were root-owned but had rx permissions for all users. Most importantly:
$ ls -lhZ /usr/lib/git-core/{git-upload-pack,git-pack-objects}
lrwxrwxrwx 1 root root ? 3 Apr 20 10:20 /usr/lib/git-core/git-pack-objects -> git
-rwxr-xr-x 1 root root ? 1.5M Apr 20 10:20 /usr/lib/git-core/git-upload-pack
Except these:
$ ls -lhZ /usr/lib/git-core/git-sh-*
-rw-r--r-- 1 root root ? 2.3K Apr 20 10:20 /usr/lib/git-core/git-sh-i18n
-rwxr-xr-x 1 root root ? 1.5M Apr 20 10:20 /usr/lib/git-core/git-sh-i18n--envsubst
-rw-r--r-- 1 root root ? 16K Apr 20 10:20 /usr/lib/git-core/git-sh-prompt
-rw-r--r-- 1 root root ? 9.1K Apr 20 10:20 /usr/lib/git-core/git-sh-setup
EDIT 4:
Following torek‘s suggestion to check myremote
, I logged in there and checked the same suspects:
$ ls -lhZ /usr/bin/git*
-rwxr-xr-x 1 root root ? 2.7M Apr 19 18:19 /usr/bin/git
lrwxrwxrwx 1 root root ? 3 Apr 19 18:19 /usr/bin/git-receive-pack -> git
-rwxr-xr-x 1 root root ? 1.5M Apr 19 18:19 /usr/bin/git-shell
lrwxrwxrwx 1 root root ? 3 Apr 19 18:19 /usr/bin/git-upload-archive -> git
lrwxrwxrwx 1 root root ? 3 Apr 19 18:19 /usr/bin/git-upload-pack ->
$ ls -lhZ /usr/lib/git-core/{git-upload-pack,git-pack-objects,git}
-rwxr-xr-- 1 root root ? 2.7M Apr 19 18:19 /usr/lib/git-core/git
lrwxrwxrwx 1 root root ? 3 Apr 19 18:19 /usr/lib/git-core/git-pack-objects -> git
lrwxrwxrwx 1 root root ? 3 Apr 19 18:19 /usr/lib/git-core/git-upload-pack -> git
Now I see the culprit.
2
Answers
You probably want to take a look at your local
.git
repository, especially under.git/objects
. Check out the owers and permissions of the files and see if git has to right to create them.What most likely has happened is that you probably have run
git
once either withsudo
or being temporary logged as root usingsu
. As long as files already exists, it won’t be a problem because they will keep their original rights and owners, but if Git had to create non-yet-existent files such as the packed object subdirecty, if he did as root, it’s normal to be unable to exploit them as your regular identity afterwards.And this actually could happen with any command, not simply Git. Thus the error message, which only reports the error reason given by the system.
EDIT
Considering the comments below, you’ll probably want to check out the content of
/usr/libexec/git-core/
as well. This contains all executables that are actually associated to git commands and subcommands.Among them, lies
git-pack-objects
which is run bygit-upload-pack
. Both of them are different and distinct executables.It still doesn’t explain why the rights would have been lost here, but it remains a place to look at.
Good luck.
Edit 4 has the final clues: the problem was in fact on the server, which for some reason had the
/usr/lib/git-core/git
binary as mode 754 (rwxr-xr--
), instead of mode 755 (rwxr-xr-x
). This meant anyone not user root, and not in group root, was not allowed to run it, and a simplechmod
of that binary on the server would fix the problem.