'Redshift COPY command gets stuck because of locks
I'm trying to use a python script using the redshift_connector library to perform multiple COPY commands from S3 and possibly DELETE commands - all on the same redshift table. I'm using serverless redshift.
The commands on redshift gets stuck because of locks - they work from time to time but mostly I find the following locks in redshift:
- ShareRowExclusiveLock
- AccessShareLock
Both are locks on the table I work with, and the only way I found to end the process is by calling the locks_to_release stored procedure
Here is how my code looks like:
with redshift_connector.connect(...) as cnn:
with cnn.cursor() as cursor:
cursor.execute("DELETE FROM my_table WHERE ...'")
cursor.execute("COPY my_table FROM 's3://my_bucket/my_folder/' IAM_ROLE 'my-role' FORMAT AS PARQUET")
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
