Issue
After creating a private and public SSL certificate for a self-hosted GitLab server, and reconfiguring GitLab, I encounter the error:
An error occured during a connection to localhost:80. PR_END_OF_FILE_ERROR
When I try to locally visit the GitLab server.
Verification
To make sure that the issue is with adding the SSL certificates to GitLab, (and not with creating the SSL certificates, nor with Firefox), the following verifications are performed:
- The self-signed root CA is added to Firefox (and Ubuntu for that matter). This is implicitly verified by the next verification.
- I verified the SSL certificate is valid by running a dash app on port 80 and giving it (a different form of) the private and public key of the SSL certificate respectively, and visiting the https://localhost:80 url and seeing it is accessible.
Below is an image that shows the onion
domain is reachable over https for dash:
Details
I think the essence of the question boils down to finding the right settings for GitLab and Nginx. This is because verified the SSL certificates work for a Dash app running on https://localhost:80
I do not yet know how to add the public and private SSL certificates to GitLab, nor what the exact nginx settings should be. In particular,
- I am not sure on what port(s) to host GitLab (e.g. 80, 443, 444 and/or a combination of these ports). Specifically, I do not know what the values of
--publish "$GITLAB_PORT1" --publish $GITLAB_PORT2" etc.
should be, in:
sudo docker run --detach
--hostname "$GITLAB_SERVER"
--publish "$GITLAB_PORT_1"
--publish "$GITLAB_PORT_2"
--publish "$GITLAB_PORT_3"
--name "$GITLAB_NAME"
--restart always
--volume "$GITLAB_HOME"/config:/etc/gitlab
--volume "$GITLAB_HOME"/logs:/var/log/gitlab
--volume "$GITLAB_HOME"/data:/var/opt/gitlab
-e GITLAB_ROOT_EMAIL="$GITLAB_ROOT_EMAIL_GLOBAL"
-e GITLAB_ROOT_PASSWORD="$gitlab_pwd"
-e EXTERNAL_URL="https://127.0.0.1"
"$gitlab_package"
- I am not sure what the exact nginx settings should be in the
gitlab.rb
file, to allow the SSL certificates to function (such that GitLab reachable on https. I tried the followinggitlab.rb
settings:
## GitLab configuration settings
external_url 'https://localhost'
letsencrypt['enable'] = false
nginx['enable'] = true
nginx['redirect_http_to_https'] = true
nginx['ssl_certificate'] = "/etc/gitlab/ssl/localhost/public_key.pem"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/localhost/private_key.pem"
nginx['listen_port'] = 80
nginx['listen_https'] = true
In particular:
- I do not know whether I should add the root ca certificate to Gitlab.
- I do not know whether I should convert the public and private SSL key for GitLab from a
.pem
file (which they are when they are generated), into a.crt
and.key
file or not. - I do not know whether I should specify the listen port or not, and if yes, whether I should set it to 80 or 443.
- The https examples that I have seen used
nginx['listen_https'] = false
which confuses me, because I would expect nginx has to listen to https for the GitLab server to be reachable on https.
Question
On which port should I host GitLab and what should the accompanying nginx settings be in the gitlab.rb
file to make GitLab reachable at https://localhost:port_nr and https://some_onion.onion:port_nr?
Approach
Based on this video, I have created the following function that:
- is able to convert the private and public SSL certificates for GitLab, from
.pem
files into the.key
and.crt
file (for private and public respectively). - Copies those SSL certificates into the default location where GitLab looks for them.
- Then applies the above
gitlab.rb
settings. - and tells Gitlab to reconfigure.
- It has two boolean parameters:
include_root_ca_in_gitlab
to add the root ca to GitLab (or not), andconvert_to_crt_and_key_ext
to convert the.pem
private and public key into a.crt
(for the public key) and a.key
(for the private key) respectively.
The instructions to run this script are given in the MWE below.
add_private_and_public_ssl_certs_to_gitlab() {
local project_name="$1"
local domain_name="$2"
local ssl_private_key_filename="$3"
local ssl_public_key_filename="$4"
local ca_public_cert_filename="$5"
local convert_to_crt_and_key_ext="$6"
local include_root_ca_in_gitlab="$7"
local ssl_private_key_filepath="certificates/ssl_cert/$project_name/$ssl_private_key_filename"
local ssl_public_key_filepath="certificates/ssl_cert/$project_name/$ssl_public_key_filename"
# Assert local private and public certificate exist for service.
manual_assert_file_exists "$ssl_private_key_filepath"
manual_assert_file_exists "$ssl_public_key_filepath"
create_gitlab_ssl_directories
local ssl_public_key_in_gitlab_filepath
local ssl_private_key_in_gitlab_filepath
if [[ "$convert_to_crt_and_key_ext" == "true" ]]; then
# Convert public .pem into public .crt with:
openssl x509 -outform der -in "$ssl_public_key_filepath" -out "certificates/ssl_cert/$project_name/$domain_name.crt"
manual_assert_file_exists "certificates/ssl_cert/$project_name/$domain_name.crt"
# Convert private .pem into private .key with:
openssl pkey -in "$ssl_private_key_filepath" -out "certificates/ssl_cert/$project_name/$domain_name.key"
manual_assert_file_exists "certificates/ssl_cert/$project_name/$domain_name.key"
# TODO: verify the generated .key is valid with the old public .pem.
# TODO: verify the generated .key is valid with the new public .crt.
ssl_public_key_in_gitlab_filepath="/etc/gitlab/ssl/$domain_name.crt"
ssl_private_key_in_gitlab_filepath="/etc/gitlab/ssl/$domain_name.key"
# Copy your new certificates into the folder where GitLab looks by default
# for new SSL certificates.
sudo cp "certificates/ssl_cert/$project_name/$domain_name.crt" "$ssl_public_key_in_gitlab_filepath"
sudo cp "certificates/ssl_cert/$project_name/$domain_name.key" "$ssl_private_key_in_gitlab_filepath"
else
ssl_public_key_in_gitlab_filepath="/etc/gitlab/ssl/$domain_name/public_key.pem"
ssl_private_key_in_gitlab_filepath="/etc/gitlab/ssl/$domain_name/private_key.pem"
sudo mkdir -p "/etc/gitlab/ssl/$domain_name/"
sudo cp "$ssl_public_key_filepath" "$ssl_public_key_in_gitlab_filepath"
sudo cp "$ssl_private_key_filepath" "$ssl_private_key_in_gitlab_filepath"
fi
manual_assert_file_exists "$ssl_public_key_in_gitlab_filepath"
manual_assert_file_exists "$ssl_private_key_in_gitlab_filepath"
# The ~/gitlab/config/gitlab.rb file says:
##! Most root CA's are included by default
# nginx['ssl_client_certificate'] = "/etc/gitlab/ssl/ca.crt"
# So perhaps also include the self-signed root ca into that dir.
if [[ "$include_root_ca_in_gitlab" == "true" ]]; then
manual_assert_file_exists "certificates/root/$ca_public_cert_filename"
sudo cp "certificates/root/$ca_public_cert_filename" "/etc/gitlab/ssl/ca.crt"
manual_assert_file_exists "/etc/gitlab/ssl/ca.crt"
fi
add_lines_to_gitlab_rb "$domain_name" "$include_root_ca_in_gitlab" "$ssl_public_key_in_gitlab_filepath" "$ssl_private_key_in_gitlab_filepath"
reconfigure_gitlab_with_new_certs_and_settings
}
create_gitlab_ssl_directories() {
sudo rm -rf "/etc/gitlab/ssl/*"
sudo mkdir -p "/etc/gitlab/ssl"
sudo chmod 755 "/etc/gitlab/ssl"
}
reconfigure_gitlab_with_new_certs_and_settings() {
# Create a method to get the docker id.
local docker_container_id
docker_container_id=$(get_docker_container_id_of_gitlab_server)
sudo docker exec -i "$docker_container_id" bash -c "gitlab-ctl reconfigure"
}
add_lines_to_gitlab_rb() {
local domain_name="$1"
local include_root_ca_in_gitlab="$2"
local ssl_public_key_in_gitlab_filepath="$3"
local ssl_private_key_in_gitlab_filepath="$4"
# Create a copy of the basic gitlab.rb file.
rm "$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
cp "$GITLAB_RB_TEMPLATE_FILEPATH" "$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
# Verified you only have to add lines (instead of modify) into that basic gitlab.rb.
if [[ "$domain_name" == "localhost" ]]; then
echo """external_url 'https://localhost'""" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
else
echo "external_url '$domain_name'" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
fi
# shellcheck disable=SC2129
echo """letsencrypt['enable'] = false""" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
echo "nginx['enable'] = true" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
echo "nginx['redirect_http_to_https'] = true" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
echo "nginx['ssl_certificate'] = "$ssl_public_key_in_gitlab_filepath"" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
echo "nginx['ssl_certificate_key'] = "$ssl_private_key_in_gitlab_filepath"" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
#echo "nginx['ssl_dhparam'] = "/etc/gitlab/ssl/dhparams.pem"" >> "$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
echo "nginx['listen_port'] = 80" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
echo "nginx['listen_https'] = false" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
if [[ "$include_root_ca_in_gitlab" == "true" ]]; then
echo "nginx['ssl_client_certificate'] = "/etc/gitlab/ssl/ca.crt"" >>"$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
fi
# TODO: verify the external url is found correctly:
# sudo cat ~/gitlab/config/gitlab.rb | grep external_url
tail -15 "$GITLAB_RB_TEMPLATE_DIR""gitlab.rb"
sudo cp "$GITLAB_RB_TEMPLATE_DIR""gitlab.rb" ~/gitlab/config/gitlab.rb
}
get_docker_container_id_of_gitlab_server() {
local docker_container_id
docker_container_id=$(sudo docker ps -aqf "name=gitlab")
assert_is_non_empty_string "$docker_container_id"
echo "$docker_container_id"
}
MWE
To reproduce the specified behaviour quickly, I have created the following minimal working example (MWE). To allow you to keep your system clean, I’ve included a Qemu virtual machine setup as well.
Qemu
One can install qemu and set up an Ubuntu 22 machine with:
# Download the ubuntu-22.04.2-desktop-amd64.iso into this directory.
wget https://releases.ubuntu.com/jammy/ubuntu-22.04.2-desktop-amd64.iso
# Create the image in which the Ubuntu 22.04 VM will be created.
qemu-img create ubuntu22.img 30G
# Create the Ubuntu 22.10 Ubuntu image.
qemu-system-x86_64
--enable-kvm
-m 1024
-machine smm=off
-cdrom $PWD/ubuntu-22.04.2-desktop-amd64.iso
-boot order=d ubuntu22.img
Next one can install and run Ubuntu with:
qemu-system-x86_64
--enable-kvm
-m 4096
-machine smm=off
-boot order=d ubuntu22_server.img
-smp 4
-chardev qemu-vdagent,id=ch1,name=vdagent,clipboard=on
-device virtio-serial-pci
-device virtserialport,chardev=ch1,id=ch1,name=com.redhat.spice.0
GitLab
The GitLab instance can be installed with:
sudo apt install git -y
mkdir git
cd git
git clone https://github.com/TruCol/Self-host-GitLab-CI-for-GitHub.git
cd Sel*
git checkout https
./install_gitlab.sh
--server
--gitlab-email [email protected]
--gitlab-password
--labprereq
--external-url "https://0.0.0.0"
--gitlab-server "0.0.0.0"
SSL
The self-signed SSL certificates for the onion domains (and localhost) for GitLab can be created with:
git clone https://github.com/HiveMinds/SSL4Tor.git
cd SSL4Tor
./src/main.sh
--1-domain-1-service
--delete-onion-domain
--services 80:gitlab:443/22:ssh:22
--make-onion-domains
--ssl-password somepassword
--background-dash
--make-ssl-certs
--setup-ssh-server
--get-onion-domain
Add self-signed SSL cert into GitLab
Next, those SSL-certificates can be automatically added to GitLab, and GitLab is automatically reconfigured with:
./src/main.sh -1d1s --services 80:gitlab:443 --apply-certs
One can check whether or not https://localhost:443 is available or not.
2
Answers
TL;DR
Below is the code I used to successfully run a self-hosted instance of GitLab over https://localhost:443 using self-signed SSL certificates:
GitLab
The GitLab instance can be installed with:
SSL
The self-signed SSL certificates for the onion domains (and localhost) for GitLab can be created with:
Add self-signed SSL cert into GitLab
Next, those SSL-certificates can be automatically added to GitLab, and GitLab is automatically reconfigured with:
Next, one should be able to visit GitLab at https://localhost:443
Nginx Settings
The settings below were found to be working nginx settings:
Note the root ca certificate (neither public nor private cert/key) is not added/given to GitLab
Initial Errors
To explain what I did wrong, and how the accepted answer led me to a working configuration, the following content of that answer is parsed and discussed:
.pem files vs .crt and .key file.
I assume there indeed should not be a difference between the content of the
public.pem
andpublic.crt
file, nor betweenprivate.pem
andprivate.key
. I verified the two files were the same. However, when copying theprivate.key
into the docker, the content got scrambled, I do not exactly know how at this time. In the end, I used the.pem
files, because that allowed me to remove the step of converting the.pem
files into.crt
and.key
respectively.Nginx able to read SSL files?
Which port to make nginx listen to
This was the last bit of information I applied to get it working. I changed the port in the
nginx
settings in thegitlab.rb
file from port 80 to port 443 and then it worked.Make nginx listen to https
That was True, I did not do that. Making nginx listen to https.
External url name for GitLab
I was unaware of the difference between
0.0.0.0
and127.0.0.1
thank you for explaining that. Changing that has helped, I expect it will also be valuable when running GitLab as an onion service. I do not yet know how to run GitLab on multiple external urls (neither for thedocker run
command to start GitLab, nor for thenginx
settings.).How to access GitLab locally
Yes, after switching the nginx settings from port 80 to port 443, I was able to visit GitLab locally at https://0.0.0.0:443.
A bunch of thoughts:
About renaming .pem files to .crt and .key – for unix systems this doesn’t really matter, since they’re all just regular text files anyway, but for the sake of accuracy, I’d change the extensions.
Did you make sure that the cert files are readable by nginx inside the container? Check the permissions.
Listening on port 80 in your config should be fine, there’s probably a bit of configuration inside nginx website config file (usually in
/etc/nginx/sites-enabled/*
) that redirects https traffic from port 80 to 443 automatically.You should definitely enable https in your config. Maybe this will be enough to get your gitlab running on https.
The EXTERNAL_URL envvar inside your docker config is "https://127.0.0.1" – this usually only allows users from localhost to view the website. To allow traffic from "everywhere", change this to "https://0.0.0.0" (check gitlab docker documentation first)
Docker ports 80 and 443 must of course be exposed
You’re trying to access the website through
https
protocol, but on port 80. In most default configs, this will not work, as port 80 is reserved for http, and port 443 for https traffic.Try these links: