Using Boto3 library multiple JSON files from AWS API bucket and converted/saved into a single CSV

Question

I need assistance with my Python script. So I used the Boto3 library to get a JSON file from the AWS API and converted it to a CSV. But what I need help with is getting multiple JSON files and converting and saving them to a single CSV file.

Can anyone help? View the code below.

So the first file is 'data/JSON_GetAll_page_1.txt', but I need to get:

data/JSON_GetAll_page_2.txt

data/JSON_GetAll_page_3.txt

data/JSON_GetAll_page_4.txt

data/JSON_GetAll_page_5.txt

then convert and save them all to a single CSV. View code block 5.

Note that I plan to use this script to get over 100 JSON files and convert them to a single CSV.

import boto3

client = boto3.resource('s3',
                        aws_access_key_id = '',
                        aws_secret_access_key = '',
                        region_name = ''
)

import os
data_path = "/saved_csvfile/"

import json

content_object = client.Object('bucket', 'data/JSON_GetAll_page_1.txt')
file_content = content_object.get()['Body'].read().decode('utf-8')
json_content = json.loads(file_content)
print(json_content['payload'])

import pandas as pd
df = pd.DataFrame(json_content['payload'])
print('Converting from JSON to CSV file ...')

for i in range(2,6):
    content_object = client.Object('bucket', 'data/JSON_GetAll_page_{i}.txt')
    file_content = content_object.get()['Body'].read().decode('utf-8')
    new_df = pd.DataFrame(json_content['payload'])

    df = pd.concat([df, new_df])
print('Successfully Converted')

df.to_csv('JSON_GetAll_page_1-5.csv',index=False)
df.to_csv(os.path.join(data_path, 'JSON_GetAll_page_1-5.csv'))
print('CSV Saved!')

I've achieved this in the past (see bottom block code below) but I'm unsure how to do this with this particular API AWS script.

import pandas as pd
import requests
import os

gis = GIS("home")
url = 'https://()JSON_GetAll_page_1.txt'
data_path = "/arcgis/home/data/"
res = requests.get(url)

df = pd.DataFrame(res.json()['payload'])
print('Converting from JSON to CSV file ...')

for i in range(2,3):
    # res = requests.get(f'https://()JSON_GetAll_page_{i}.txt')
    new_df = pd.DataFrame(res.json()['payload'])

    df = pd.concat([df, new_df])
print('Successfully Converted')

df.to_csv('JSON_GetAll_page_1.csv',index=False)
df.to_csv(os.path.join(data_path, 'JSON_GetAll_page_1.csv'))
print('CSV Saved!')```