Question 359633 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – AWS Lambda: IOException: Too many open files

I deployed an AWS Lambda written in C#, and depending on the input I can see in the console the following error message (truncated):

{
  "Cause": "{n  "errorType": "IOException",n  "errorMessage": 
"Too many open files : [...]at System.IO.FileSystem.RemoveDirectoryRecursive(String fullPath)
 "Error": "IOException",
  "ExecutionArn": [...]

I have a (somewhat?) good sense of what the error refer to, but I would like to reproduce it at least locally.

When I execute it using AWS .NET Lambda test tool with the exact same input (JSON), I cannot trigger the exception. So I assumed that my Windows 10 system is more forgiving about the number of file descriptors than the one on AWS side. So added a call to _setmaxstdio in my Lambda code:

[DllImport("msvcrt.dll")]
public static extern int _setmaxstdio(int newMax);

And then in the main code I call:

int ret = _setmaxstdio(128);

I see a value of 128 being returned in the debugger, but again I cannot reproduce the issue locally using AWS .NET Lambda test tool. I picked 128 so that it is lower than the 1024 value described at:

How would one reproduce the exception locally ?

Tags:

2

Answers


  1. 0

    I am not sure if this will be more helpful, but the AWS Serverless Application Model (SAM) will let you run your C# lambda code locally against AWS resources. It also helps with deployment and additional infrastructure like S3 buckets or DyanmoDB tables. This blog is a good intro to the subject.

    Login or Signup to reply.
  2. 0

    File handle limits on Windows don’t work the same way as fd limits on Linux. Try running your dotnet code in a Linux machine or container locally to reproduce the issue in a more representative environment. You can artificially set low limits using "ulimit -f 128", for example. You can find out more about Microsoft’s official .NET container images and how to get started here: https://hub.docker.com/_/microsoft-dotnet-samples/

    That said, are you sure you’re closing all your file handles appropriately? Anything that implements IDisposable should be used within a "using" statement or block to avoid leaking file handles and creeping toward that limit.

    https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/using

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search