Generating azure storage SAS token for client in java

Arnob
March 24, 2023
288 views
0 votes
2 Answers

For uploading media my client will send a request to the server:

message GetMediaUploadCredRequest {
  media.MediaType.Enum media_type = 1;
  media.Extension.Enum extension = 2;
  string file_name = 3;
  bool is_private = 4;
  string uploaded_by = 5;
}

And the server will generate a SAS token (like presigned_url from AWS) and http_header will send back to the client.

message GetMediaUploadCredResponse {
  string upload_credential_url = 1;
  map<string, string> https_headers = 2;
  string media_id = 3;
}

client will then make a PUT request to the URL with the https_headers and the upload process will be complete.

Here is the implementation:

public class StorexServiceImpl extends StorexServiceGrpc.StorexServiceImplBase {
    private static final Logger log = LoggerFactory.getLogger(StorexServiceImpl.class);

    private static final String STORAGE_ACCOUNT_CONNECTION_STRING = "<connection-string>";
    
    private static final String CONTAINER_NAME = "<container-name>";



    @Override
    public void getMediaUploadCred(GetMediaUploadCredRequest request, StreamObserver<GetMediaUploadCredResponse> response) {
        try{
            MediaType.Enum mediaType = request.getMediaType();
            Extension.Enum extension = request.getExtension();
            
            String fileName = String.format("%s.%s", UUID.randomUUID(), extension.toString());

            BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(STORAGE_ACCOUNT_CONNECTION_STRING).buildClient();
            BlobContainerClient containerClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME);
            BlobClient blobClient = containerClient.getBlobClient(fileName);
            BlobHttpHeaders headers = new BlobHttpHeaders().setContentEncoding("gzip");

            BlobSasPermission permission = new BlobSasPermission().setWritePermission(true); // Set the permission to allow uploading
            OffsetDateTime expiryTime = OffsetDateTime.now().plusHours(1); // Set the expiration time to 1 hour from now

            BlobServiceSasSignatureValues sasValues = new BlobServiceSasSignatureValues(expiryTime, permission)
                    .setProtocol(SasProtocol.HTTPS_HTTP);

            String sasUrl = blobClient.generateSas(sasValues);

            Map<String, String> httpHeaders = new TreeMap<>();
            httpHeaders.put("Content-Type", getContentType(mediaType, extension));
            if(!request.getIsPrivate()) {
                httpHeaders.put("x-amz-acl", "public-read");
            }

            String blobId = blobClient.getBlobName(); // get the ID of the blob

            GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                    .setMediaId(blobId) // set the blob ID in the response
                    .setUploadCredentialUrl(sasUrl)
                    .putAllHttpsHeaders(httpHeaders)
                    .build();
            response.onNext(res);
            response.onCompleted();
        } catch (Exception e){}
        super.getMediaUploadCred(request, response);

    }

    private String getContentType(MediaType.Enum mediaType, Extension.Enum extension) {
        if(mediaType == MediaType.Enum.IMAGE) {
            return "image/" + extension.toString().toLowerCase();
        } else if(mediaType == MediaType.Enum.AUDIO) {
            return "audio/mp3";
        } else if(mediaType == MediaType.Enum.VIDEO) {
            return "audio/mp4";
        } else if(mediaType == MediaType.Enum.FILE) {
            return "application/" + extension.toString().toLowerCase();
        }
        return "binary/octet-stream";
    }
}

But for result I got this:

{
    "uploadCredentialUrl": "sv=2021-10-04&spr=https%2Chttp&se=2023-03-24T13%3A36%3A38Z&sr=b&sp=w&sig=JUXXe1Qi13VWipgFWzx70mTOsVqadQCjmIF%2BxRl14cs%3D",
    "httpsHeaders": {
        "Content-Type": "image/jpeg"
    },
    "mediaId": "0ae4a0c5-167d-4a32-9752-0ad0d2b67e66.JPEG"
}

The uploadCredentialUrl seems weird and not like an actual URL at all.

Update 1:

Found this post. So I changed my code a bit:

    public void getMediaUploadCred(GetMediaUploadCredRequest request, StreamObserver<GetMediaUploadCredResponse> response) {
        try{
            MediaType.Enum mediaType = request.getMediaType();
            Extension.Enum extension = request.getExtension();

            String fileName = String.format("%s.%s", UUID.randomUUID(), extension.toString());

            
            log.info("New Implementations Started");
            SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd");
            fmt.setTimeZone(TimeZone.getTimeZone("UTC"));
            Calendar cal = Calendar.getInstance();
            cal.setTime(new Date());
            cal.add(Calendar.DATE, -2);
            String start = fmt.format(cal.getTime());
            cal.add(Calendar.DATE, 4);
            String expiry =  fmt.format(cal.getTime());
            SecretKeySpec secretKey = new SecretKeySpec(Base64.getDecoder().decode(STORAGE_ACCOUNT_KEY), "HmacSHA256");
            Mac sha256HMAC = Mac.getInstance("HmacSHA256");
            sha256HMAC.init(secretKey);
            String  resource ="sc";
            String permissions ="rwdlac";
            String service = "b";
            String apiVersion="2019-07-07";
            String stringToSign = STORAGE_ACCOUNT_NAME + "n" +
                    permissions +"n" +  // signed permissions
                    service+"n" + // signed service
                    resource+"n" + // signed resource type
                    start + "n" + // signed start
                    expiry + "n" + // signed expiry
                    "n" +  // signed IP
                    "httpsn";
            log.info("string to sign: {}", stringToSign);

            String signature=Base64.getEncoder().encodeToString(sha256HMAC.doFinal(stringToSign.getBytes("UTF-8")));
            String sasToken = "sv=" + apiVersion +
                    "&ss=" + service+
                    "&srt=" + resource+
                    "&sp=" +permissions+
                    "&se=" + URLEncoder.encode(expiry, "UTF-8") +
                    "&st=" + URLEncoder.encode(start, "UTF-8") +
                    "&spr=https" +
                    "&sig=" + URLEncoder.encode(signature,"UTF-8");
            log.info("sas token: {}", sasToken);
            String resourceUrl = "https://" + STORAGE_ACCOUNT_NAME + ".blob.core.windows.net/" + CONTAINER_NAME + "?comp=block&" + sasToken;
            
            

            Map<String, String> httpHeaders = new TreeMap<>();
            httpHeaders.put("Content-Type", getContentType(mediaType, extension));
            if(!request.getIsPrivate()) {
                httpHeaders.put("x-amz-acl", "public-read");
            }

            GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                    .setMediaId("blob id") // set the blob ID in the response
                    .setUploadCredentialUrl(resourceUrl)
                    .putAllHttpsHeaders(httpHeaders)
                    .build();
            response.onNext(res);
            response.onCompleted();
        } catch (Exception e){}
    }

And I got a response:

{
    "uploadCredentialUrl": "https://<STORAGE_ACCOUNT>.blob.core.windows.net/<CONTAINER_NAME>?comp=block&sv=2019-07-07&ss=b&srt=sc&sp=rwdlac&se=2023-03-28&st=2023-03-24&spr=https&sig=RpdV8prUgjzFApNo6bkuuiRAPHnw1mqww5l42cwVwyY%3D",
    "httpsHeaders": {
        "Content-Type": "image/jpeg"
    },
    "mediaId": "blob id"
}

But when I tried this:

curl -X PUT -T ~/Downloads/gfx100s_sample_04_thum-1.jpg -H "x-ms-date: $(date -u)" "https://<STORAGE_ACCOUNT>.blob.core.windows.net/<CONTAINER_NAME>/myimage.jpg?comp=block&sv=2019-07-07&ss=b&srt=sc&sp=rwdlac&se=2023-03-28&st=2023-03-24&spr=https&sig=RpdV8prUgjzFApNo6bkuuiRAPHnw1mqww5l42cwVwyY%3D"

I got this:

<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:337c6e4d-301e-0019-7ebd-5f3baf000000
Time:2023-03-26T08:30:28.5705948Z</Message><AuthenticationErrorDetail>Signature did not match. String to sign used was storageapollodevappstro
rwdlac
b
sc
2023-03-24
2023-03-28

https
2019-07-07
</AuthenticationErrorDetail></Error>%

Update 2:

I have updated my code following jccampanero guideline:

    @Override
    public void getMediaUploadCred(GetMediaUploadCredRequest request, StreamObserver<GetMediaUploadCredResponse> response) {
        BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(STORAGE_ACCOUNT_CONNECTION_STRING).buildClient();
        BlobContainerClient containerClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME);
        BlobSasPermission permission = new BlobSasPermission().setReadPermission(true).setWritePermission(true);
        OffsetDateTime expiryTime = OffsetDateTime.now().plusMinutes(30);

        String blobName = containerClient.getBlobClient(request.getFileName()).getBlobName();
        String sasToken = generateSasToken(STORAGE_ACCOUNT_NAME, STORAGE_ACCOUNT_KEY, CONTAINER_NAME, blobName, permission, expiryTime);
        String url = String.format("https://%s.blob.core.windows.net/%s/%s?%s", STORAGE_ACCOUNT_NAME, CONTAINER_NAME, request.getFileName(), sasToken);
        GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                .setMediaId(blobName)
                .setUploadCredentialUrl(url)
                .build();
        response.onNext(res);
        response.onCompleted();
    }

    private static String generateSasToken(String accountName, String accountKey, String containerName, String blobName, BlobSasPermission permission, OffsetDateTime expiryTime) {
        String sasToken = null;
        try {
            String signedPermissions = permission.toString();
            String signedStart = OffsetDateTime.now().format(DateTimeFormatter.ISO_OFFSET_DATE_TIME);
            String signedExpiry = expiryTime.format(DateTimeFormatter.ISO_OFFSET_DATE_TIME);
            String canonicalizedResource = String.format("/blob/%s/%s/%s", accountName, containerName, blobName);
            String signedVersion = "2020-12-06";

            String stringToSign = signedPermissions + "n" +
                    signedStart + "n" +
                    signedExpiry + "n" +
                    canonicalizedResource + "n" +
                    "n" + // signedKeyObjectId
                    "n" + // signedKeyTenantId
                    "n" + // signedKeyStart
                    "n" + // signedKeyExpiry
                    "n" + // signedKeyService
                    "n" + // signedKeyVersion
                    "n" + // signedAuthorizedUserObjectId
                    "n" + // signedUnauthorizedUserObjectId
                    "n" + // signedCorrelationId
                    "n" + // signedIP
                    "httpsn" + // signedProtocol
                    signedVersion + "n" +
                    "n" + // signedResource
                    "n" + // signedSnapshotTime
                    "n" + // signedEncryptionScope
                    "n" + // rscc
                    "n" + // rscd
                    "n" + // rsce
                    "n" + // rscl
                    "n"; // rsct

            Mac mac = Mac.getInstance("HmacSHA256");
            mac.init(new SecretKeySpec(Base64.getDecoder().decode(accountKey), "HmacSHA256"));
            String signature = Base64.getEncoder().encodeToString(mac.doFinal(stringToSign.getBytes("UTF-8")));
            sasToken = String.format("sv=%s&st=%s&se=%s&sr=b&sp=%s&sig=%s",
                    signedVersion, signedStart, signedExpiry, permission.toString(), URLEncoder.encode(signature, "UTF-8"));
        } catch (Exception e) {
            e.printStackTrace();
        }
        return sasToken;
    }

And running the CURL command:

curl -X PUT 
  -T ~/Downloads/gfx100s_sample_04_thum-1.jpg 
  -H "x-ms-blob-type: BlockBlob" 
  -H "x-ms-meta-name: example" 
  "<URL-with sas token>"

gives me:

<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:124d92c4-101e-001e-189f-6157cc000000
Time:2023-03-28T18:02:50.6977457Z</Message><AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail></Error>%

But I followed the Signature fields from docs. So why am I still getting the error?

Update 3:

        BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(STORAGE_ACCOUNT_CONNECTION_STRING).buildClient();
        BlobContainerClient containerClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME);

        OffsetDateTime expiryTime = OffsetDateTime.now().plusMinutes(30);
        BlobClient blobClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME)
                .getBlobClient(request.getFileName());
        BlobSasPermission permission = new BlobSasPermission().setReadPermission(true).setWritePermission(true);
        BlobServiceSasSignatureValues sasValues = new BlobServiceSasSignatureValues(expiryTime, permission);
        String sasToken = blobClient.generateSas(sasValues);

        String blobName = containerClient.getBlobClient(request.getFileName()).getBlobName();
        log.info(sasToken);
        String url = String.format("https://%s.blob.core.windows.net/%s/%s?%s", STORAGE_ACCOUNT_NAME, CONTAINER_NAME, request.getFileName(), sasToken);
        GetMediaUploadCredResponse res = GetMediaUploadCredResponse.newBuilder()
                .setMediaId(blobName)
                .setUploadCredentialUrl(url)
                .build();

Error:

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Answers

Chosen as BEST ANSWER

@jccampanero helped quite a lot. So thanks!

Here is the actual piece of code that worked:

    public String generatePresignedUrl {
        BlobServiceClient blobServiceClient = new BlobServiceClientBuilder().connectionString(STORAGE_ACCOUNT_CONNECTION_STRING).buildClient();
        BlobContainerClient containerClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME);

        OffsetDateTime expiryTime = OffsetDateTime.now().plusMinutes(30);
        BlobClient blobClient = blobServiceClient.getBlobContainerClient(CONTAINER_NAME)
                .getBlobClient(request.getFileName());
        BlobSasPermission permission = new BlobSasPermission().setReadPermission(true).setWritePermission(true);
        BlobServiceSasSignatureValues sasValues = new BlobServiceSasSignatureValues(expiryTime, permission);
        String sasToken = blobClient.generateSas(sasValues);

        String blobName = containerClient.getBlobClient(request.getFileName()).getBlobName();
        log.info(sasToken);
        String url = String.format("https://%s.blob.core.windows.net/%s/%s?%s", STORAGE_ACCOUNT_NAME, CONTAINER_NAME, request.getFileName(), sasToken);
        return url;
    }

and the curl command:

curl -X PUT -T <file-location> -H "x-ms-blob-type: BlockBlob" "<resource-url with SAS token>"

Notice the x-ms-blob-type.

Resources that helped:

String to sign [if you want to do create the token manually]
Header

(Edit)

- jccampanero
- March 28, 2023 at 12:05 am
- 0 votes
0
With your first code snippet you are actually generating a valid SAS:
```
{
    "uploadCredentialUrl": "sv=2021-10-04&spr=https%2Chttp&se=2023-03-24T13%3A36%3A38Z&sr=b&sp=w&sig=JUXXe1Qi13VWipgFWzx70mTOsVqadQCjmIF%2BxRl14cs%3D",
    "httpsHeaders": {
        "Content-Type": "image/jpeg"
    },
    "mediaId": "0ae4a0c5-167d-4a32-9752-0ad0d2b67e66.JPEG"
}
```
You mentioned that uploadCredentialUrl doesn’t seem to be a valid URL and you are right because in fact you are receiving a SAS token.

A SAS token is composed by a series of query parameters that should be appended to the URL of the resource for which it should be applied.

You are creating a SAS for uploading a specific blob (you can create the SAS for different resources); it means that you need to provide the URL for the blob resource you want to create and append to that obtained SAS token.

Your curl command looks fine indeed, just be sure of using the right SAS token:
```
curl -X PUT -T ~/Downloads/gfx100s_sample_04_thum-1.jpg -H "x-ms-date: $(date -u)" "https://<storage account name>.blob.core.windows.net/<container name>/myimage.jpg?<received SAS token>"
```
Please, consider review the Put Blob REST API documentation for detailed information about the headers you can specify.

This related page offers guidance as well.

Finally, you are trying generating the signature yourself, and the process is well documented but it is common to receive errors related to signature mismatch: if possible, prefer the Azure SDK for the specific programming language instead.
Login or Signup to reply.

Please signup or login to give your own answer.

Click here to cancel reply.