'Compress multiple files with zip in AWS Glue

I'm trying to compress the multiple csv files from a folder in the S3 bucket using AWS Glue. I have the script below.

currentdate = datetime.now().strftime("%Y%m%d")
directory = pathlib.Path("s3://mys3bucket/folder1/"+currentdate+"/")
zippy = "s3://mys3bucket/folder1/"+currentdate+"-abc.zip"

with ZipFile(zippy, "w", ZIP_DEFLATED) as archive:
     for file_path in directory.rglob("*csv"):
            archive.write(file_path, arcname=file_path.relative_to(directory))

It will end in the error below.

FileNotFoundError: [Errno 2] No such file or directory: 's3://mys3bucket/folder1/20220420-abc.zip'

Can anyone advise how to solve this error?

apache-sparkpysparkzip


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions