'Access Denied issue in AWS Glue while performing a simple ETL task
I am facing an error wile trying run AWS Glue , i am trying to copy data from my table which was filled with the help of a crawler.
The Error is given below
An error occurred while calling o91.pyWriteDynamicFrame. Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 6YRCFCNCKW9ZK2PF; S3 Extended Request ID: 7v/5/dEhaxjIMMxfpCEu5vT6fwzmyV0kIphicPvUDYKY23rFYN1ALn2qo/N3CcIUEhSrOGKklW4=; Proxy: null)
my script is given below :
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
args = getResolvedOptions(sys.argv, ["JOB_NAME"])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args["JOB_NAME"], args)
# Script generated for node AWS Glue Data Catalog
AWSGlueDataCatalog_node1647583966899 = glueContext.create_dynamic_frame.from_catalog(
database="s3-databse-mockdata",
table_name="mock_data_csv",
transformation_ctx="AWSGlueDataCatalog_node1647583966899",
)
# Script generated for node Amazon S3
AmazonS3_node1647583976365 = glueContext.getSink(
path="s3://destination-001/",
connection_type="s3",
updateBehavior="UPDATE_IN_DATABASE",
partitionKeys=[],
enableUpdateCatalog=True,
transformation_ctx="AmazonS3_node1647583976365",
)
AmazonS3_node1647583976365.setCatalogInfo(
catalogDatabase="s3-databse-mockdata", catalogTableName="dest-table"
)
AmazonS3_node1647583976365.setFormat("csv")
AmazonS3_node1647583976365.writeFrame(AWSGlueDataCatalog_node1647583966899)
job.commit()
I am unable to find what is the problem
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
