I’ll try to be brief.
I have multiple rails apps running on a Digital Oceans rps. Currently 3 but I’ve had more. All deployed with Capistrano. Two or rails 7.0, one is 7.1.2. The server is Ubuntu 18.04 and I’ve had it for over 5 years.
I took a quick look at the logs the other day. I had some garbage in the 7.1.2 app. I just noticed that there was no production log. In config/production I had some old stuff for setting log level.
config.log_level = ENV.fetch("RAILS_LOG_LEVEL", "info")
I set it to :warn and redeployed, but it still didn’t work. After a lot of reading I now understand the a lot of the config options are not set on a deploy unless the version changed.
I also noted that my NGINX logs were massive and not being rotated. Fixed that but one app is in some kind of spam or denial of service attack. Every second or so I get in the nginx access log
47.76.35.19 - - [05/Jan/2024:16:22:56 -0600] "HEAD /payouts/score_sheet/new_score_sheet/new_score_sheet/new_score_sheet/score_sheet/new_score_sheet/new_score_sheet/payouts/payouts/new_score_sheet/payouts/score_sheet/
new_score_sheet/new_score_sheet/about/new_score_sheet/payouts/new_score_sheet/about/new_score_sheet/score_sheet/score_sheet/new_score_sheet/payouts/new_score_sheet/
new_score_sheet/new_score_sheet/new_score_sheet/payouts/payouts/new_score_sheet/score_sheet/new_score_sheet/about/score_sheet/payouts/about/score_sheet/new_score_sheet/
about/score_sheet/new_score_sheet/payouts/payouts/score_sheet/payouts/score_sheet/about/payouts/payouts/score_sheet/payouts/score_sheet/payouts/about/new_score_sheet
payouts/score_sheet/new_score_sheet/new_score_sheet/new_score_sheet/score_sheet/
new_score_sheet/score_sheet/about/new_score_sheet/new_score_sheet/payouts/about/
about/payouts/score_sheet/new_score_sheet/about/about/payouts/new_score_sheet/
new_score_sheet/new_score_sheet/payouts/score_sheet/new_score_sheet/
HTTP/1.1" 403 0 "-" "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3522.153 Safari/537.36"
They’re all paths that are available on the home page menu without login – just strung together. The paths just rotate the names on every access.
I looks like the only way I can restart the server is to reboot the linux box, but I’m almost afraid!
It uptime is :18:29:15 up 949 days, 5:32, 3 users, load average: 0.00, 0.01, 0.00
Not sure why I’m afraid – I reboot my staging server (local debian box) and everything comes up.
Just though I’d ask if there was something other than reboot. ‘rails restart’ won’t work, that’s for Passenger and I’m using Puma.
Also, if anyone know what kind of spam and whatever is attacking my server I’d be grateful. I denied the IP but it still goes in the log.
Thanks in advance
2
Answers
After a couple of days of going in circles I found my answer. Not how to restart the server, but why my production log was not working. Don't think I needed to restart anyhow -- probably just a puma restart.
I commented on a closed Rails issue logs are not generating in production.log... that give more details
After a lot of digging I finally realized that it stopped logging the day I upgraded to 7.1.2. I found the above post in Rails Issues and suggested they reopen it. Or at least point out the it won't log in production unless you remove the log to STDOUT block.
Now if I could stop the idiot who's filling up my nginx logs with about 1000k of character a day, I'd be happy.
Regarding restart: I’m not completely sure about your setup of Puma. One setup we have is where puma is a separate unix process and nginx is separate. Both have their own entry points through systemd files, connected through sockets. So incase you want to rotate nginx logs or restart nginx, you can do that without restarting the rails application because those are not connected on process id (pid) level. There might be (very small) downtime but because rails app (however large it might be) has not been restarted, nginx should come up under a second.
Regarding those logs: Don’t worry, anything that is exposed to public internet is hit by bots every now and then so this is normal. However, make sure to keep your gems and operating system upgraded so that you’re not having any serious vulnerability in your application that exposes it to an attack.