Question 95393 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

How to upload s3 using boto3 – Amazon web services

I want to upload my logs to my bucket
I never been used python and boto3
This is my code

import os
import datetime as dt
import boto3

x = dt.datetime.now()
date = x.strftime("%Y%m%d")
bucket = 'mybucket'
dir_path = "/log"
s3 = boto3.client('s3')

def log():
    global dir_path
    for (dir_path, dir, files) in os.walk(dir_path):
        for file in files:
            if date in file:
                file_path = os.path.join(dir_path, file)
                print file_path

file_name = (log())
key = (log())
res = s3.upoad_file(file_name, bucket, key)

and this is result

log1
log2
log3
log4
Traceback *most recent call last):
File "test2.py", line 21, in <module>
  res = s3.upload_file(file_name, bucket, key)
File "home/user/.local/lib/python2.7/site-packages/boto3/s3/tranfer.py", line 273, in upload_file extra_args=ExtraArgs, callback=Callback)
File "home/user/.local/lib/python2.7/site-packages/boto3/s3/tranfer.py", line 273, in upload_file raise ValueError('Filename must be a string')
ValueError: Filename must be a string

I have 4 log files

please help me
how to fix it?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    It was simple. this is my final code thanks.

    import os
import datetime as dt
import boto3
import socket

x = dt.datetime.now()
date = x.strftime("%Y%m%d")
bucket = 'mybucket'
dir_path = "/log"
s3 = boto3.client('s3')

def log(in_path):
    for (dir_path, dir, files) in os.walk(in_path):
        for file in files:
            if date in file:
                yield os.path.join(dir_path, file)


for file_name in log(dir_path):
    key = socket.gethostname() + '/' + file_name
    res = s3.upload_file(file_name, bucket, key)

  2. 0

    Since you need to upload more than one file, and you
    stated that the upload one log works, you could
    do the following, which basically goes through the
    directory list as per your original intention, and
    then for each file item that satisfies that criteria
    (date in file), it returns the filepath to the
    calling loop.

    import os
import datetime as dt
import boto3

x = dt.datetime.now()
date = x.strftime("%Y%m%d")
bucket = 'mybucket'
dir_path = "/log"
s3 = boto3.client('s3')

def log(in_path):
    for (dir_path, dir, files) in os.walk(in_path):
        for file in files:
            if date in file:
                yield os.path.join(dir_path, file)


for file_name in log(dir_path):
    res = s3.upload_file(file_name, bucket, file_name)

    Please note that if you need to keep track of the results,
    then you could make a change like so:

    .
.
.

results = {}
for file_name in log(dir_path):
    results[file_name] = s3.upload_file(file_name, bucket, file_name)
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search