Question 19343 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Connect to AWS S3 bucket from Nodejs app ( persistent connection required)

I have nodejs/express app from which I want to connect to AWS S3.

I do have a temporary approach to make connection,

environment file

aws_access_key_id=XXX
aws_secret_access_key=XXXX/
aws_session_token=xxxxxxxxxxxxxxxxxxxxxxxxxx

S3-connection-service.js

const AWS = require("aws-sdk");

AWS.config.update({
    accessKeyId: `${process.env.aws_access_key_id}`,
    secretAccessKey: `${process.env.aws_secret_access_key}`,
    sessionToken: `${process.env.aws_session_token}`,
    region: `${process.env.LOCAL_AWS_REGION}`
});

const S3 = new AWS.S3();

module.exports = {
    listBucketContent: (filePath) =>
        new Promise((resolve, reject) => {
            const params = { Bucket: bucketName, Prefix: filePath };
            S3.listObjects(params, (err, objects) => {
                if (err) {
                    reject(err);
                } else {
                    resolve(objects);
                }
            });
        }),
       ....
       ....
 }

controller.js

   const fetchFile = require("../../S3-connection-service.js");

   const AWSFolder = await fetchFile.listBucketContent(filePath);

Fine it’s works and I’m able to access S3 bucket files and play with it.

PROBLEM

The problem is connection is not persistent. Since, I use session_token, connection remains alive for sometime and again after sometime new tokens will be generated, I have to copy-paste them in env file and re-run the node app.

I really have no idea how can I make connection persistent ?

Where to store AWS confidential/secrets and how to use them to connect to S3 so connection remains alive ?

Tags:

2

Answers


  1. 0

    connect to S3 so connection remains alive ?

    You can’t make one request to S3 and keep it alive forever.

    These are your options:

    1. Add a try/catch statement inside your code to handle credentials expired error. Then, generate new credentials and re-initialize the S3 client.

    2. Instead of using a Role, use a User. (IAM Identities). User credentials can be valid forever. You won’t need to update the credentials in this case.

    3. Do not provide the credentials to AWS.config.update like you are doing right now. If you don’t provide the credentials, the AWS client will try to read them from your ~/.aws/credentials file automatically. If you create a script to update them every hour (ex: a cronjob), then your credentials will be up-to-date at all times.

    Login or Signup to reply.
  2. 0

    Just remove

    AWS.config.update({
    accessKeyId: `${process.env.aws_access_key_id}`,
    secretAccessKey: `${process.env.aws_secret_access_key}`,
    sessionToken: `${process.env.aws_session_token}`,
    region: `${process.env.LOCAL_AWS_REGION}`
});

    code block from lambda source in file S3-connection-service.js

    Attach a role to lambda function with proper permissions. You will have same functionally.

    For local development.

    You can set environment variable before testing your application.

    export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-west-2

    https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html

    If you are using any IDE you can set these environment variables on it.

    If you are testing from cli

    $ AWS_ACCESS_KEY_ID=EXAMPLE AWS_SECRET_ACCESS_KEY=EXAMPLEKEY AWS_DEFAULT_REGION=us-west-2 npm start

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search