'RDS Postgres to Kinesis Data stream for real time load of data through AWS glue

The requirement is to load data from RDS POSTGRES to RDS oracle on a real-time basis.

AWS Glue is the ETL tool. AWS Glue has a feature to take data from Kinesis in real-time for streaming.

I am struggling to bring data to Kinesis from Postgres.

Any suggestions??

postgresqlamazon-kinesis


Solution 1:[1]

I used this link to push data into kinesis--

https://aws.amazon.com/blogs/database/stream-changes-from-amazon-rds-for-postgresql-using-amazon-kinesis-data-streams-and-aws-lambda/

points to be noted-

  • used Ec2.
  • No need to explicitly install wal2JSON , it is already there.
  • it produces data in a certain JSON structure. Change it to the structure as required(target database)
  • I used glue 2.0.

Feel free to post here if any one face any issue. Will try to answer.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Alam

Related Questions