'AWS Glue Studio is not creating Table
So I've been using AWS Glue Studio to create ETL jobs. I currently set it up so that the target node was an S3 bucket, however I also want to create a table on the Data Catalog. However, once I run the ETL job it doesn't mark an error and it does correctly save the output as a Parquet file to my S3 bucket, however it does not create a table in the Data Catalog.
This is my code:
node hem-horarios-bpi hemhorariosbpi_node3 = glueContext.getSink(
path="s3://hem-data-datalake-staging/staging_general/staging_horarioHP/", connection_type="s3", updateBehavior="LOG", partitionKeys=[], enableUpdateCatalog=True, transformation_ctx="hemhorariosbpi_node3", )
hemhorariosbpi_node3.setCatalogInfo( catalogDatabase="hem-db-staging-tables", catalogTableName="hem-horarios-pbi" )
hemhorariosbpi_node3.setFormat("glueparquet")
hemhorariosbpi_node3.writeFrame(S3bucket_node1)
job.commit()
I have tried changing the role, but it won't do.
Solution 1:[1]
Try changing the updateBehaviour property from LOG to UPDATE_IN_DATABASE.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Robert Kossendey |
