I’m no sys-admin, but I use Amazon to host a number of applications. I’m getting some strange readings from one of my EC2 instances, which coincide with poor performance and the site being offline. I understand the AWS EC2 will cut server performance down to safe levels if it exceeds its resources, which seems to be what is happening. SO my questions for any AWS experts out there… (see attached screenshot)
- how much RAM should a php database-driven website be using (the site has about 100 users a day at this stage)? 1gb (T2.Micro instance + EBS) doesn’t seem to cut it!
- Does this look like a memory-leak in some of our scripts?
- Does this look like malicious activity on the server?
- Is this caused by resource-heavy DB calls? (tens of thousands of records, but not a vast number of users, >100 users)
I have tried :
Relaunching the instance (a number of times) provides a temporary fix.
Upgrading from nano to micro instance (fixed it for about a week until we began to run into trouble again)
Thanks!
EDIT:
This is after being up for 4.5 hours
EDIT:
This is after being up for 19 hours
EDIT:
Running strace on cfn-hup results in thousands of entries like this:
2
Answers
It sounds like it can be many things. Try recreating the instance and migrating to the new instance.
If the CPU activity is still super high then memory leak, or resource-heavy DB call.
If that fixes, then it sounded like it might have been a virus.
Couple of things I can think of.
Try to enable slow query log in MySQL, and address the slow queries if found.
Try
sudo strace -p <httpd_Main_Process_ID> -o httpd_trace.txt
, then analyze the file and see if there’s something suspicious in the WebApp – This will give you everythinghttpd
process is doing.