Question 95633 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

My CheckSumSHA256 i create in javascript for S3 ManagedUpload is always wrong – Amazon web services

sha256 is the SHA256 Hash of a file.
I use then btoa(256) to base64 it.

The result is always
InvalidRequest: Value for x-amz-checksum-sha256 header is invalid.

The AWS Documentation says,

ChecksumSHA256 — (String) The base64-encoded, 256-bit SHA-256 digest
of the object. This will only be present if it was uploaded with the
object. With multipart uploads, this may not be a checksum value of
the object. For more information about how checksums are calculated
with multipart uploads, see Checking object integrity in the Amazon S3
User Guide. Checking object integrity – Amazon Simple Storage Service
Verify the integrity of objects uploaded and downloaded to Amazon S3.

let sha256conv = btoa(sha256);

const params = {
        Bucket: process.env.BUCKET,
        Key: path.basename(task.file),
        Body: fileData,
        ContentType: ContentType || 'application/octet-stream',
        CacheControl: "max-age=172800",
        ChecksumAlgorithm: 'sha256',
        ChecksumSHA256: sha256conv
    
    };

const upload = new AWS.S3.ManagedUpload({
        service: s3,
        params
    });

the sha256 is generated like this:

export async function getFileSha256(fileName, fileSize, onProgress) {
    return new Promise((resolve, reject) => {
        const hash = crypto.createHash('sha256');

// change to 'binary' if you want a binary hash.
        hash.setEncoding('hex');
        const bar1 = new cliProgress.SingleBar({}, cliProgress.Presets.shades_classic);
        bar1.start(fileSize, 0);

        const fd = fs.createReadStream(fileName);

        fd.on('data', async chunk => {
            let processedBytes = bar1.value + chunk.length;
            bar1.update(processedBytes);
            await onProgress(processedBytes)
        })

        fd.on('end', function () {
            bar1.stop();
            hash.end();
            return resolve(hash.read())
        });

// read all file and pipe it (write it) to the hash object
        fd.pipe(hash);
    });
}
Tags:

2

Answers


  1. 0 
        (async()=>{
      var message='helloworld';
      const msgbuffer = new TextEncoder().encode(message);
      const hashbuffer = await crypto.subtle.digest('SHA-256',msgbuffer);
      var b64=toBase64(hashbuffer);
      console.log(message,' --> ',b64);
    })();

    var sha256    = 'hash'  // your sha256
    var hashbuffer   = new TextEncoder().encode(sha256);
    var b64=toBase64(hashbuffer);
    console.log(sha256,' --> ',b64);

    function toBase64(buffer){
      var binary = '';
      var bytes = new Uint8Array(buffer);
      var len = bytes.byteLength;
      for (var i = 0; i < len; i++) {
          binary += String.fromCharCode( bytes[ i ] );
      }
      return window.btoa( binary );
    }

    hope this helps?

    Login or Signup to reply.
  2. 0

    SHA-256 takes bytes (not text) as input and produces bytes (not text) as output. Of course, dealing with raw bytes is not always convenient so checksum bytes are often somehow converted to text, and there’re multiple ways to do so.

    I’m no JavaScript expert but it appears that when calculating SHA-256 checksum you automatically encode it as Hex:

    hash.setEncoding('hex');

    Then you encode it further to Base64:

    let sha256conv = btoa(sha256);

    So the entire chain looks like: bytes -> Hex -> Base64.

    It might solve the issue if you change it to bytes -> Base64 by using hash.setEncoding('binary').

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search