How can I write Python script result to S3 instead of local drive?

Question

#Im working on below script to pull the information of AWS ec2 instances with details. #What it does now is that it pull the information that I am after and create a csv file in the same patch of the script. I am working on expanding this to make it a Lambda function to pull down the information from all AWS accounts and instead of creating CSV on local machine push the data (CSV) format to S3 location. #Now challenge ahead that I need help on how to modify the script to be able to write to S3 bucket instead of writing to the local drive?

import boto3
import csv
from pprint import pprint

session = boto3.Session(profile_name='#ProfileName')
ec2_re=session.resource(service_name="ec2",region_name="ap-southeast-2")
ec2_cli=session.client(service_name="ec2")

fo=open('EC2_Details.csv','w',newline='')
data_obj=csv.writer(fo) data_obj.writerow(["InstanceID","InstanceType","InstanceName","InstanceLunchTime","Instance_Private_IP","InstanceState","InstanceTags"])

cnt=1
response = ec2_cli.describe_instances()
for reservation in response["Reservations"]:
    for instance in reservation["Instances"]:
        data_obj.writerow([instance["InstanceId"], instance["InstanceType"], instance["KeyName"], instance["LaunchTime"], instance["PrivateIpAddress"], instance["State"]["Name"], instance["Tags"]])
        cnt+=1

fo.close()

Thank you

Answer 1

Trivial way, you upload the file you created on disk.

There are advanced way that avoid writing on disk, you can find them on SO.

import boto3
import csv
from pprint import pprint

#Creating Session With Boto3.
session = boto3.Session(
    profile_name="your profile",
)



session = boto3.Session(profile_name='#ProfileName')
ec2_re=session.resource(service_name="ec2",region_name="ap-southeast-2")
ec2_cli=session.client(service_name="ec2")

fo=open('EC2_Details.csv','w',newline='')
data_obj=csv.writer(fo) data_obj.writerow(["InstanceID","InstanceType","InstanceName","InstanceLunchTime","Instance_Private_IP","InstanceState","InstanceTags"])

cnt=1
response = ec2_cli.describe_instances()
for reservation in response["Reservations"]:
    for instance in reservation["Instances"]:
        data_obj.writerow([instance["InstanceId"], instance["InstanceType"], instance["KeyName"], instance["LaunchTime"], instance["PrivateIpAddress"], instance["State"]["Name"], instance["Tags"]])
        cnt+=1

fo.close()


#Creating S3 Resource From the Session.
s3 = session.resource('s3')
bucket = s3.Bucket("your-bucket-name")
bucket.upload_file('EC2_Details.csv')