To install git on e2-medium (2 vCPUs, 4 GB memory) Debian GNU/Linux 10 (buster) instances, we run the following script defined in the startup-script-url as per the instructions seen here: GCP Startup Scripts on Linux.
After running sudo apt-get update or sudo apt update the behavior for the VMS change:
- Startup scripts don’t seem to execute after stopping, and resuming VMs
- Cannot SSH via web console via GCP GUI.
Any thoughts on why this is happening and how to fix this? We’ve temporarily worked around by installing git from source.
startupscript.sh
Defined as --metadata startup-script-url=gs://my-project/startupscript.sh
#!/bin/bash
if ! command -v git &> /dev/null
then
echo "============================================"
echo "GIT WAS NOT FOUND. PLEASE INSTALL"
sudo apt -y update
sudo apt install -y git-all
echo "============================================"
fi
echo "CONTINUE TO DO MORE HERE"
1. Initital Successful Run
CHECK STARTUP SCRIPT LOGS
- SSH into VM
- Run
sudo journalctl -xefu google-startup-scripts -f
The script has run successfully as per logs below:
Oct 29 14:48:48 instance-1 google_metadata_script_runner[521]: startup-script-url: Processing triggers for man-db (2.8.5-2) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Processing triggers for install-info (6.5.0.dfsg.1-4+b1) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Processing triggers for mime-support (3.62) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Processing triggers for libglib2.0-0:amd64 (2.58.3-2+deb10u3) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Processing triggers for libc-bin (2.28-10) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up glib-networking:amd64 (2.58.0-2+deb10u2) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up libsoup2.4-1:amd64 (2.64.2-2) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up libsoup-gnome2.4-1:amd64 (2.64.2-2) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up librest-0.7-0:amd64 (0.8.1-1) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up libgtk-3-0:amd64 (3.24.5-1) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up emacs-gtk (1:26.1+1-3.2+deb10u2) ...
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: update-alternatives: using /usr/bin/emacs-gtk to provide /usr/bin/emacs (emacs) in auto mode
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Install emacsen-common for emacs
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: emacsen-common: Handling install of emacsen flavor emacs
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Install git for emacs
Oct 29 14:49:09 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up git-el (1:2.20.1-2+deb10u3) ...
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: Install git for emacs
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: Install git for emacs
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up emacs (1:26.1+1-3.2+deb10u2) ...
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: Setting up git-all (1:2.20.1-2+deb10u3) ...
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.38.1+dfsg-1) ...
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: Processing triggers for libc-bin (2.28-10) ...
Oct 29 14:49:10 instance-1 sudo[1212]: pam_unix(sudo:session): session closed for user root
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: ============================================
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url: CONTINUE TO DO MORE HERE
Oct 29 14:49:10 instance-1 google_metadata_script_runner[521]: startup-script-url exit status 0
ALSO VALIDATE IN GOOGLE CLOUD LOGGING
Cloud Logging only displays up until the Processing triggers for man-db log, which is a bit weird, but it appears that all would have installed.
2. Power Off
sudo poweroff
3. Start VM from GUI
Startup script execution, and web ssh are broken.
CHECK STARTUP SCRIPT LOGS
- SSH into VM
- Run
sudo journalctl -xefu google-startup-scripts -f
No journal files were found.
ALSO VALIDATE IN GOOGLE CLOUD LOGGING
No logs can be seen other than the startup. Also no indication that startup-script-url metadata found (which typically occurs)
WEB SSH NO LONGER WORKS
Boot Logs
journalctl -b
No journal files were found
-- No entries --
sudo /var/log/dmesg
command not found
/var/log/boot.log
-bash: /var/log/boot.log: No such file or directory
2
Answers
GCP startup scripts run at container creation, not at Linux startup (those "subsequent startups" are a delusion). And the script by itself is non-sensical; just do
apt install -y git. There simply is no point to check if the package exists, whenaptitudeperforms just the same check (one can see in to logs when trying unnecessary packages). Be aware that everything this script does will run as userrootand be owned by userroot, which might require the one or otherchmodorchown. "VMs no longer work as expected" is nothing I could reproduce; that’s not an error description.Here when we are installing git-all it replaces the systemd init system with the older SysV init system, breaking
cloud-initand pretty much all the booting process.This is the reason why you’re unable to SSH into your instance.There are 2 possible fixes for this issue :
–no-install-recommends -y git-all this will make sure that recommends are not getting installed.