Question 89749 in PostgreSQL Question posted in
The official documentation can be found here.

Debugging an Out of Memory Error in Node.js – Postgresql

I’m currently working on a Node.js project and my server keeps running out of memory. It has happened 4 times in the last 2 weeks, usually after about 10,000 requests. This project is live and has real users.

I am using

  • NodeJS 16
  • Google Cloud Platform’s App Engine (instances have 2048mb of memory)
  • Express as my server framework
  • TypeORM as database ORM (database is postgres hosted on separate GCP SQL instance)

I have installed the GCP profiling tools and have captured the app running out of memory, but I’m not quite sure how to use the results. It almost looks like there is a memory leak in the _handleDataRow function within the pg client library. I am currently using version 8.8.0 of the library (8.9.0 was just released a few weeks ago and doesn’t mention fixing any memory leaks in the release notes).

I’m a bit stuck with what I should do at this point.

Any suggestions or advice would be greatly appreciated! Thanks.

enter image description here

enter image description here

enter image description here

Update: I have also cross-posted to reddit and someone there helped me determine that issue is related to large queries with many joins. I was able to reproduce the issue, and will report back here once I am able to solve it.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    The problem was the package @google-cloud/debug-agent. There is an open issue about the package causing a memory leak. Many people have had this issue.


  2. 0

    When using App Engine, a great place to start looking for "why" a problem occurred in your app is through the Logs Explorer. Particularly, if you know the time-frame of when the issues started escalating or when the crash occurred.

    Although based on your Memory Usage graph, it’s a slow leak. So a top-to-bottom approach of your back-end is really necessary to try and pin-point the culprit. I would go through the whole stack and look for things like Globals that are set and not cleaned up, promises that are not being returned, large result-sets from the database that are bottle-necking the server, perhaps from a scheduled task.

    Looking at the 2pm – 2:45pm range on the right-hand of the graph, I would narrow the Logs Explorer down to that exact time-frame. Then I would look for the processes or endpoints that are being utilized most frequently in that time-frame as well as the ones that are taking the most memory to get a good starting point.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search