'Is there a way to upsert data into an Aurora MySQL RDS Database?

I am trying to create an AWS Glue Job in python Glue 3.0 that will move data from s3 to an Aurora MySQL database in RDS. The json s3 file will either have new records, old records, or updated records so I want it to upsert.

Currently I am using write_dynamic_frame.from_catalog() which will essentially insert all the data into the database catalog however it doesn’t update based on a duplicate key so I will have duplicate records over time. I want to avoid using the pymysql library because I don't want my password stored in the script

If I was using Redshift I would use write_dynamic_frame.from_jdbc_conf() which lets me run a prequery and postquery but these options only work for Redshift not RDS I believe.

Is there any way to perform a similar action to upserts with my RDS database based on duplicate unique keys?

mysqlaws-glueamazon-auroraaws-glue-data-catalog


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions