Question posted in 
I have a ruby on rails project and it’s hosted on Digital Ocean droplet using Nginx, Puma, and Capistrano.
I’m taking over this project and there isn’t a tracked repo of this project. So the only source code I have is on the remote server.
I had to restart the droplet server through D.O UI to access the console. After the restart, some of the pages were not loading (while some do). At this point, zero changes were made to the code base.
Looking at production.log, it spits out a common generic error for the pages that are not loading:
TypeError (no implicit conversion of nil into String):
...
Completed 500 Internal Server Error in 11ms (Views: 10.0ms | Allocations: 3709)
So I dug deeper. This is what I get in /var/log/nginx/error.log:
2023/05/03 18:59:02 [crit] 16157#16157: *1 connect() to unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock failed (2: No such file or directory) while connecting to upstream, client: 100.11.109.141, server: ms.explore.myapp.com, request: "GET /users/sign_up HTTP/2.0", upstream: "http://unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock:/users/sign_up", host: "ms.explore.myapp.com", referrer: "https://ms.explore.myapp.com/users/sign_up"
2023/05/03 18:59:03 [crit] 16157#16157: *1 connect() to unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock failed (2: No such file or directory) while connecting to upstream, client: 100.11.109.141, server: ms.explore.myapp.com, request: "GET /users/sign_up HTTP/2.0", upstream: "http://unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock:/users/sign_up", host: "ms.explore.myapp.com", referrer: "https://ms.explore.myapp.com/users/sign_up"
2023/05/03 18:59:03 [crit] 16157#16157: *1 connect() to unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock failed (2: No such file or directory) while connecting to upstream, client: 100.11.109.141, server: ms.explore.myapp.com, request: "GET /users/sign_up HTTP/2.0", upstream: "http://unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock:/users/sign_up", host: "ms.explore.myapp.com", referrer: "https://ms.explore.myapp.com/users/sign_up"
I’ve checked that /var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock exists.
This is /etc/nginx/sites-available/myapp:
upstream puma {
server unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock;
}
server {
listen 80;
server_name ms.explore.myapp.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl http2;
ssl_certificate /etc/letsencrypt/live/ms.explore.myapp.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ms.explore.myapp.com/privkey.pem;
server_name ms.explore.myapp.com;
root /var/www/vhosts/myapp/current/public;
add_header X-Frame-Options "DENY";
location ^~ /(assets|packs)/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
location = /robots.txt {
add_header Content-Type text/plain;
return 200 "User-agent: *nAllow: /n";
}
try_files $uri/index.html $uri @puma;
location @puma {
proxy_pass http://puma;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
client_max_body_size 25M;
rewrite ^/contacts/new$ /contact-us permanent;
rewrite ^/program/resources/course-navigation$ /program/course-navigation permanent;
}
Running ps aux | grep puma shows that puma is connected to the correct sock file:
deploy 16523 0.0 5.8 410660 120276 ? S 18:59 0:00 puma 3.12.1 (unix:///var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock)
deploy 16525 0.0 6.3 1161592 130524 ? Sl 18:59 0:00 puma: cluster worker 0: 16523
deploy 16536 0.0 6.4 1161592 131408 ? Sl 18:59 0:00 puma: cluster worker 1: 16523
deploy 19690 0.0 0.0 14860 1072 pts/1 S+ 19:16 0:00 grep --color=auto puma
What could be happening here? I’m not too familiar with Capistrano and it’s total functionality. I was suspecting that restarting the server could ignore Capistrano’s automated set up steps, which could be configuring puma service workers and nginx server wrong.
I’ve tried restarting puma and nginx with sudo service puma restart and sudo service nginx restart respectively. As well as tried loading backup provided by D.O. But none seem to help.
I’m not able to do a deployment process as I don’t have the source code locally.
Would greatly appreciate help from some server/ruby/rails experts! Thank you in advance!
EDIT
Here are more relevant information:
config/deploy.rb:
# config valid for current version and patch releases of Capistrano
lock "~> 3.17.0"
set :application, "myapp"
set :repo_url, "[email protected]:cra/myapp.git"
set :deploy_to, "/var/www/vhosts/myapp"
append :linked_files, "config/database.yml", "config/master.key", "config/features.yml"
append :linked_dirs, "log", "tmp/pids", "tmp/cache", "tmp/sockets", "public/system", "public/uploads", "vendor/bundle"
set :keep_releases, 3
set :keep_assets, 2
Here is also config/deploy/ms-instance.rb:
server "myapp-ms", user: "deploy", roles: %w(app db web)
set :puma_threads, [4, 16]
set :puma_workers, 2
set :pty, true
set :use_sudo, false
set :stage, :production
set :puma_bind, "unix://#{shared_path}/tmp/sockets/#{fetch(:application)}-puma.sock"
set :puma_state, "#{shared_path}/tmp/pids/puma.state"
set :puma_pid, "#{shared_path}/tmp/pids/puma.pid"
set :puma_access_log, "#{release_path}/log/puma.error.log"
set :puma_error_log, "#{release_path}/log/puma.access.log"
set :puma_preload_app, true
set :puma_worker_timeout, nil
set :puma_init_active_record, true # Change to false when not using ActiveRecord
EDIT: 05/04/2023
After restarting the server directly on Digital Ocean and also restarting nginx and puma, I no longer see the error I was receiving in /var/log/nginx/error.log. But I’m still getting Internal Error 500 for the pages I’ve been having trouble accessing.
I also made a double check to see if database is connected to the application and it is. I can see the database logs and there doesn’t seem to be any abnormalities.
2
Answers
This is on behalf of you said: you are having the below errors on some of the pages only:
If you can access some of the pages after deployment, there are significantly fewer chances of missing configurations in Puma or Nginx at the server.
Your
config/deploy.rbandconfig/deploy/ms-instance.rbfiles looks ok.This means there are code errors on these pages code. These cases are generating on production within some of the cases only which are not tested on development or test environments.
Here are my suggetions:
You must use some exception monitoring tools in your project like Sentry, Honeybadger etc.
These will capture all requests parameters, headers, line numbe etc complete details of raised exceptions/errors.
Here are gems you can directly use:
Or
You should debug these pages code for the errors arround the line number you are getting at production.log file.
This is on behalf of you said:
So I dug deeper. This is what I get in /var/log/nginx/error.log
2023/05/03 18:59:02 [crit] 16157#16157: *1 connect() to
Based on the error message in the Nginx error log, it seems like
Nginx is unable to connect to the Puma application server.
The error indicates that Nginx cannot find the Unix socket file that Puma uses to communicate with Nginx.
Here are some possible solutions you can try:
Restart Nginx and Puma: You can try restarting Nginx and Puma to see if that resolves the issue. You can use the following commands to restart them:
sudo service nginx restart,sudo service puma restartCheck File and Folder Permissions: Make sure that the file and folder permissions are set up correctly. Check that the user running Nginx and Puma has access to the Unix socket file and its parent folders. You can use the following command to check the ownership and permissions of the Unix socket file:
ls -l /var/www/vhosts/myapp/shared/tmp/sockets/myapp-puma.sock