Question 172185 in Javascript Question posted in
A very good W3school tutorial can be found here.

How can I access private S3 buckets from localhost?

I want to fetch images to display them on a local website for dev/test purposes before pushing the code to a more serious environment.

I am using the AWS JS SDK V3 on a Node web server and while 2 solutions come to mind, they both seem convoluted.

1 – Use GetObjectCommand on my object, download it, make a blob/base64 encode, create a URL for my blob, and use that as my img src attribute. I am not sure if downloading files on to the server to serve is best practice. In my mind since the S3 object already has a link to the image, it just makes sense if I could somehow embed that into the src attribute. Unfortunately the SDK doesn’t seem to have any way to do this, beyond;

2 – Use the AWS CLI to presign an S3 bucket URL and use it. This works, and seems like a cleaner solution, but it needs a set duration and I would like the link to never expire. Also, it’s not very practical as I would need to presign a high number of S3 links and that is simply not practical.

I’ve enabled GET HTTP requests from * origins as a CORS policy on my bucket for the time being, but I’m pretty sure that’s not relevant because while it does let me use the SDK’s GetObjectCommand functionality from an app running on localhost, the issue is I want something like img src=… but it seems I need to pass appropriate headers to have access to the content, and without knowing which (if?) part of the SDK provides this functionality, I’m kind of stuck. This is also not a valid solution at all as realistically the bucket’s CORS policy should only allow requests from one origin – the actual website’s domain – and not every origin. So even if this let me send requests successfully, it wouldn’t be what I wanted.

Related questions:

  • Failed GET requests to AWS S3 from website running on local host
    It appears I have the same problem as this guy, but we’re using totally different architectures and I have no idea how his solution might apply to me.
  • Trouble Loading an Image from an Amazon S3 Bucket (without using the URL)
    This guy seems to be trying to achieve the same goal as me, and went down the base64 route, but his question seems to be about a thing that is completely irrelevant to me
  • I need to view S3 images using react website
    This guy seems to be in the exact same boat as me and, while he unfortunately has no answers, he does have a comment from Chris telling him in classic SO fashion that while he wants to do X, he should be doing Y instead and suggesting using Cognito as opposed to hard coded IAM role access credentials to access an item which is a fair suggestion, but has absolutely nothing to do with ducati1212’s problem. He then proceeds to recommend using Cloudfront to serve the images which, again, seems like a fair suggestion and apparently worked but I am not at the point in my app where I have began using a CDN to serve images yet, so maybe I’ll read more into it later but for now it is completely irrelevant to me.
Tags:

2

Answers


  1. 0

    The answer is: don’t. If it’s a public bucket use the URL. If it’s a private bucket you’re gonna have to handle signed URL’s which last up to 1 week max.

    The public URL is now formed like: https://bucket-name.s3.amazonaws.com/key-name.

    So: if your bucket is example and the object key is tree/oak.png, use:

    https://example.s3.amazonaws.com/tree/oak.png

    Realistically and practically speaking, for website/web app media – just don’t use any providers bucket storage offering. Use plain old static web storage from your web server. For Nodejs, that’s the ‘fs’ module. For NGINX it’s the http core module. Just keep it simple in my opinion. I’ve used Google’s Cloud Storage before for web media and it started breaking (expiring) and it was just not worth the hassle. If you want accounting (media/image access records) you can roll your own logging solution.

    Login or Signup to reply.
  2. 0

    I’ve had the same problem before, i was creating an Website-HomePage-Printer,
    and run through some trouble to draw the images out of the URLS

    I reached into 2 ways to extract img from URL:

    But first:

    • Check if you have your AWS credentials, keys set up in your env
    • There is a extension on VSCode, relatively new that allows many kinds of data manipulation and functions.

    1- Through canvas element

    Embedding an image via data: URL:

    Another possible way to include images is via the data: URL. Data URLs allow you to completely define an image as a Base64 encoded string of characters directly in your code.

    const img = new Image(); // Create new img element
img.src =
  "data:image/gif;base64,R0lGODlhCwALAIAAAAAA3pn/ZiH5BAEAAAEALAAAAAALAAsAAAIUhA+hkcuO4lmNVindo7qyrIXiGBYAOw==";

    Data URLs are composed of four parts: a prefix (data:), a MIME type indicating the type of data, an optional base64 token if non-textual, and the data itself:

    data:[<mediatype>][;base64],<data>

    The mediatype is a MIME type string, such as ‘image/jpeg’ for a JPEG image file. If omitted, defaults to ”text/plain;charset=US-ASCII”

    Canvas MDN

    2-Through a use of an headless browser and puppeteer package

    • Puppeter Package
    • Headless Chromium (You don’t actually need this, but it wraps the content better, just make sure you are using an updated version of chromium and if it exists on your WSL if you use one)

    You can basically get any data you have access to and screenshot it for example.

    
'use strict';

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('http://example.com');
  await page.screenshot({path: 'example.png'});
  await browser.close();
})();

    Github Examples

    EXTRA

    If you are having problems with getting the object, my advice is to read this documentation GetObjectsAPI and check your ACL GetObjectAcl again, check your permissions on AWS, keys, etc etc.

    [Junior Dev btw, forgive me for any misinterpretations]

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search