Question 97411 in Nginx Question posted in
The official documentation can be found here.

C#: Unable to write data to the transport connection: Broken Pipe when using RestClient to upload larger files – Nginx

I am using the RestSharp restClient to upload a file, and I am getting an error:

 System.Net.WebException: Error while copying content to a stream.
       ---> System.Net.Http.HttpRequestException: Error while copying content to a stream.
       ---> System.IO.IOException: Unable to write data to the transport connection: Broken pipe.
       ---> System.Net.Sockets.SocketException (32): Broken pipe

This error only occurs when trying to upload larger files. I don’t know the exact cutoff but I do not get this error for files up to ~29 MB, but for a file that is 37 MB I do get this error.

The client code is:

RestClient client = new("http://data-target-service:15012")
{
  Timeout = -1
};
RestRequest request = new(Method.Post);
request.AddFile("formFile", file.FullName); //file is of type fileInfo

IRestResponse response = await client.ExecuteAsync(request);

These are both services deployed in the same namespace on Kubernetes, hence the HTTP .The API being called is:

[HttpPost]
public async Task<IActionResult> UploadFile(IFormFile formFile)
{
  try
  {
    await HandleDataHere(formFile);
    return StatusCode(200);
  }
  catch (Exception e)
  {
    Console.Writeline(e.message);
    return StatusCode(500);
  }
}

The HandleDataHere method is never actually called, so the error is probably not there. I have read elsewhere that problems may occur when several calls share the same HTTP client, therefore I have made every call use its own client.

At this point I am not sure if it is a problem in the client, the API, or even if it is a configuration issue for the ingress (which is handled by NGINX). Any ideas would be very helpful.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    So, the solution was to simply add the [DisableRequestSizeLimit] attribute to the API, and it works.

    When running the API through PostMan the error message was different:

    {
    "type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
    "title": "One or more validation errors occurred.",
    "status": 400,
    "traceId": "00-55ecaadcbc6dddd7d113966bfebe4d0f-760b8ab8f5082583-00",
    "errors": {
        "": [
            "Failed to read the request form. Request body too large. The max request body size is 30000000 bytes."
        ]
    }
}

  2. 0

    Usually in my app! I give the user the ability to upload a large set of files of csv. and sizes maybe like => 500MB file, 5GB file … etc.

     [RequestFormLimits(ValueLengthLimit = int.MaxValue,
        MultipartBodyLengthLimit = long.MaxValue)]
    [DisableRequestSizeLimit]
    [HttpPost("upload-csv")]
    public async Task<IActionResult> ImportCsvFile(
        [FromForm] IFormFile file)
    {
        var configuration = new CsvConfiguration(CultureInfo.InvariantCulture)
        {
            HasHeaderRecord = false,
        };

        using (var reader = new StreamReader(file.FileName))
        {
            using var csv = new CsvReader(reader, configuration);
            var records = csv.GetRecords<CustomerEntity>();
        }

        //Do somework


        return Ok();
    }
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search