Azure Question posted in
The official documentation to get you started can be found here.

Streaming response in ASP.NET Core 6 Web API working locally but not on Azure web app hosted on windows

I run a service as an Azure Web App. The problem is the response stream is not received as they are sent by the app, but come in large chunks. My assumption is it’s due to response buffering. My ASP.NET Core 6 Web API is hosted on IIS on windows OS in an Azure web app.

Everything is working locally (using local IIS hosting), but not on hosted environment.

I have tried the following:

  1. Disabled response buffering in middleware.
var responseBodyFeature = context.Features.Get<IHttpResponseBodyFeature>();

if (responseBodyFeature != null)   
    responseBodyFeature.DisableBuffering();

  1. Set response headers by adding a web.config in my ASP.NET Core 6 Web API:

     <httpProtocol>
     <customHeaders>
         <!-- Disable response buffering -->
         <add name="Buffer" value="False" />
         <add name="X-Accel-Buffering" value="no" />
         <add name="Cache-Control" value="no-cache" />
     </customHeaders>
 </httpProtocol>

  2. I have checked there is no APIM or proxy in between which can cause response buffering.

    I have the following code in my ASP.NET Core 6 Web API:

     await foreach (var streamingResponse in response.Response.WithCancellation(canToken))
 {
     foreach (var choice in streamingResponse.Choices)
     {
         yield return await Task.Run(() => choice.Text);            
         await Response.Body.FlushAsync();
     }
 }

  3. Considered below as well, but I don’t want other non streaming endpoints to be impacted:

     .UseKestrel(options =>
             {
                 options.Limits.MaxResponseBufferSize = 2000
             });

Below is the client side code calling the ASP.NET Core 6 Web API:

        let config = {
            'method': "POST",
            'cached': false,
             headers: {
                'Content-Type': 'application/json',
                'Authorization': '<token>'
             },
             body: JSON.stringify(requestBody)
        };
        const response = await fetch(url, config);
        const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
        while (true) {
            const { value, done } = await reader.read();
            if (done) break;
            console.log(value);                
        }
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    The issue for me was that a middleware for logging was waiting for entire result to log. So my code for streaming was correct and just removing that middleware got it working. So first you should check for any such blocking middlewares!

    My code for streaming in Controller -

     await foreach (var streamingCompletions in azResponse.WithCancellation(cancellationToken))
    {
        var data = streamingCompletions.Choices.Select(x => x.Text);
        string myFinalString = string.Join("", data);
        var sseMessage = $"{System.Text.Json.JsonSerializer.Serialize(myFinalString)}";
        await Response.Body.WriteAsync(Encoding.UTF8.GetBytes(sseMessage));
        await Response.Body.FlushAsync();
    }

return new EmptyResult();

  2. 0

    You should be successful if you set the Transfer-Encoding response header to identity.

    You can do it in your web.config, or in code.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search