我装不上焦油。gz文件从我的本地目录到S3存储桶位置。我在运行下面的函数来上传任何csv文件时没有遇到问题,但我收到了错误:“Fileobj必须实现读取”错误。我正在使用Boto3和Python
tar_文件是本地驱动器上要上传到S3存储桶位置的文件
import csv
import glob
import os
import tarfile
from datetime import date
from typing import Optional, Set
from io import BytesIO
import psycopg2
import boto3
from constants import (
ARTIFACT_STORE,
DB_HOST,
DB_PASSWORD,
DB_USER,
EXCLUDED_TABLES,
NIPR_DB_NAME,
S3_ACCESS_KEY_ID,
S3_SECRET_ACCESS_KEY,
S3_ENDPOINT_URL,
BUCKET_NAME
)
def upload_s3_file():
tar_file = f"{ARTIFACT_STORE}/{date.today()}_cds.tar.gz"
s3 = boto3.client('s3',endpoint_url=S3_ENDPOINT_URL,aws_access_key_id=S3_ACCESS_KEY_ID,aws_secret_access_key=S3_SECRET_ACCESS_KEY)
with tarfile.open(tar_file,'r:gz') as tar:
s3.upload_fileobj(tar,BUCKET_NAME,tar_file)
当我在S3存储桶的csv生成文件上运行以下函数时,我没有问题:
s3 = boto3.client('s3',endpoint_url=S3_ENDPOINT_URL,aws_access_key_id=ACCESS_KEY,aws_secret_access_key=SECRET_KEY)
with open("test.csv", "rb") as f:
s3.upload_fileobj(f,BUCKET_NAME, "test")