Question 381569 in Azure Question posted in
The official documentation to get you started can be found here.

Unable to send blob to Azure – missing HTTP header exception

I am trying to upload a parquet file to Azure, but it keeps giving me an error message saying that its missing a required HTTP header.

In the example I am providing I just tried some basic data in a memory stream and it is still throwing this exception.

RequestFailedException: An HTTP header that’s mandatory for this
request is not specified.
RequestId:0bd87df1-701f-0003-5ff4-944ccd000000
Time:2024-04-22T20:36:16.2840714Z Status: 400 (An HTTP header that’s
mandatory for this request is not specified.) ErrorCode:
MissingRequiredHeader

Content: {"error":{"code":"MissingRequiredHeader","message":"An HTTP
header that’s mandatory for this request is not
specified.nRequestId:…nTime:2024-04-22T20:36:16.2840714Z"}}

Headers: Connection: keep-alive x-ms-error-code: MissingRequiredHeader
x-ms-request-id: … x-ms-version:
2024-05-04 x-ms-client-request-id:
… Content-Length: 204 Content-Type:
application/json;charset=utf-8 Date: Mon, 22 Apr 2024 20:36:16 GMT
Server: Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0 Via: REDACTED

Code:

            string storageAccountName = "name";
            string storageAccountKey = "key";
            StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential(storageAccountName, storageAccountKey);

            BlobServiceClient blobServiceClient = new BlobServiceClient(
                new Uri("https://" + storageAccountName + ".dfs.core.windows.net"),
                sharedKeyCredential
            );

            BlobContainerClient desContainer = blobServiceClient.GetBlobContainerClient("testcontainer");

            BlobClient desBlob = desContainer.GetBlobClient("/testAPI/my.parquet");

            var myStr = "Hello!";
            var content = Encoding.UTF8.GetBytes(myStr);
            using (var ms = new MemoryStream(content))
                desBlob.Upload(ms);

I am not able to find what headers are missing and I am not sure how to add them as there does not seem a standard way to add headers besides; Content-Type

Packages:

  • Azure.Core – 1.39.0
  • Azure.Storage.Blobs – 12.20.0-beta.2
  • Azure.Storage.Common – 12.19.0-beta.2

I’ve tried reverting to older packages thinking maybe functionality was there before but changed in recent revisions.

Any help in the right direction would be helpful.

Tags:

2

Answers


  1. 0

    As mentioned in the comments, please change the endpoint suffix from dfs.core.windows.net to blob.core.windows.net. My guess is that when the REST API sees dfs.core.windows.net endpoint, it tries to use Data Lake REST API instead of Blob Storage REST API and because of that, you are getting the missing header error.

    Login or Signup to reply.
  2. 0

    Parquet files are binary, so you shouldn’t use UTF8 encoding.

    Remove UTF encoding from your code:

    string storageAccountName = "name";
string storageAccountKey = "key";
StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential(storageAccountName, storageAccountKey);

BlobServiceClient blobServiceClient = new BlobServiceClient(
    new Uri("https://" + storageAccountName + ".dfs.core.windows.net"),
    sharedKeyCredential
);

BlobContainerClient desContainer = blobServiceClient.GetBlobContainerClient("testcontainer");

BlobClient desBlob = desContainer.GetBlobClient("/testAPI/my.parquet");

var myStr = "Hello!";
var content = Encoding.ASCII.GetBytes(myStr);  // Using ASCII encoding

using (var ms = new MemoryStream(content))
{
    await desBlob.UploadAsync(ms);
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search