ETL Connector not loading in aws

Question

I am using bigquery connector for aws glue in my glue job. It was working fine few days ago but now suddenly it is giving me below error:

LAUNCH ERROR | Glue ETL Marketplace - failed to download connector.Please refer logs for details.

Below is the full error that i am getting on cloudwatch

2021-11-08T11:33:02.045+05:00   Traceback (most recent call last): File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main

2021-11-08T11:33:02.070+05:00   "__main__", mod_spec) File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/tmp/aws_glue_custom_connector_python/docker/unpack_docker_image.py", line 361, in <module>

2021-11-08T11:33:02.070+05:00   main() File "/tmp/aws_glue_custom_connector_python/docker/unpack_docker_image.py", line 351, in main

2021-11-08T11:33:02.070+05:00   res += download_jars_per_connection(conn, region, endpoint, proxy) File "/tmp/aws_glue_custom_connector_python/docker/unpack_docker_image.py", line 304, in download_jars_per_connection

2021-11-08T11:33:02.070+05:00   download_and_unpack_docker_layer(ecr_url, layer["digest"], dir_prefix, http_header) File "/tmp/aws_glue_custom_connector_python/docker/unpack_docker_image.py", line 168, in download_and_unpack_docker_layer

2021-11-08T11:33:02.070+05:00   layer = send_get_request(layer_url, header) File "/tmp/aws_glue_custom_connector_python/docker/unpack_docker_image.py", line 80, in send_get_request

2021-11-08T11:33:02.070+05:00   

2021-11-08T11:33:02.070+05:00   response.raise_for_status() File "/home/spark/.local/lib/python3.7/site-packages/requests/models.py", line 765, in raise_for_status

2021-11-08T11:33:02.071+05:00   raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 400 Client Error: Bad Request

2021-11-08T11:33:02.119+05:00   Glue ETL Marketplace - failed to download connector, activation script exited with code 

Answer 1

When Glue job tried using connector, it has to download the connector in the form of container. The containers for connectors are available in amazon public ECR repo. To pull the container from the AWS public repo, we have to add "AmazonEC2ContainerRegistryFullAccess" policy to your IAM role. We can limit the access to read only as well.

Answer 2

This could very likely be a permissions issue. I was running into it and temporarily gave looser permissions, which seemed to resolve it.

Answer 3

I have this issue with the GCP BigQuery connector. Some jobs are able to run the connector, some aren't. All have the same permissions and settings. There seems to be an issue after requesting the ECR Authorization Token where the request times out.