Question 298094 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Upload all files from a local folder to S3

I have a python method to sends one file that exists in a local folder to S3. I want modify the method to send all files that exist in the folder to S3.

My method is:

import boto3
from boto3 import Session
from botocore.exceptions import NoCredentialsError


session_root = boto3.Session(region_name='eu-west-3', profile_name='my_profile')
s3_client = session_root.client('s3')

prefix_key = "S3_folder/"
local_path = "/home/local_folder/"
bucket_name = "bucket_S3"

def upload_to_aws(local_file, bucket, s3_file):
    try:
        s3_client.upload_file(local_file, bucket, s3_file)
        print("Upload Successful")
        return True
    except FileNotFoundError:
        print("The file was not found")
        return False
    except NoCredentialsError:
        print("Credentials not available")
        return False


uploaded_files_to_S3 = upload_to_aws(local_path, bucket_name, prefix_key)

To you have please an idea how can I modify it to take all files in my local_folder and send them to S3_folder ?

Thanks,

Tags:

2

Answers


  1. 0

    You can achieve this with command

    aws s3 cp `your local folder` `s3://bucket`

    Above command can also be run with python

    import subprocess
local_folder = "/path/to/your/local/folder"
s3_bucket = "s3://bucket"

command = ["aws", "s3", "cp", local_folder, s3_bucket]


 try:
   subprocess.run(command, check=True)
   print("Files copied to S3 successfully!")
 except subprocess.CalledProcessError as e:
   print("Error:", e)
    Login or Signup to reply.
  2. 0

    The upload_file() API call can only upload one file at a time.

    Therefore, you will need to loop through your directory and upload each file individually:

    import os
import boto3

prefix_key = "S3_folder/"
local_path = "/home/local_folder/"
bucket_name = "bucket_S3"

s3_client = boto3.client('s3')

for filename in os.listdir(local_path):

    print("Uploading:", filename)
    s3_client.upload_file(local_path + filename, bucket_name, prefix_key + filename)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search