'Create Gzip Compressed JSON file and upload on S3 bucket using NodeJS

I wanted to upload GZip compressed JSON file on S3 bucket.I am struggling with this, can someone help me on this.As, I am trying to use zlib npm module for Gzip compression of JSON file but coudn't get a method to achieve this.

Below is my upload method to upload Gzip compressed JSON file on S3 :

    var uploadEntitlementDataOnS3 = function(next, event,
    jsonFileContent, filePath, results) {
    console.log("uploadEntitlementDataOnS3 function 
    started",jsonFileContent);
        var bufferObject = new 
    Buffer.from(JSON.stringify(jsonFileContent));
        var s3 = new AWS.S3();
        var params = {
            Bucket: configurationHolder.config.bucketName,
            Key: filePath,
            Body: bufferObject,
            CacheControl: 'no-cache',
            ContentType: "application/json",
            ContentEncoding: 'gzip'
        }
        s3.putObject(params, function(err, data) {
            if (err) {
                console.log(err, err.stack);
                next(err);
            } else {
                next(null, filePath);
            }
        });
    };

Thanks

node.jsgzip


Solution 1:[1]

Also, I am using below code snippet for gzip compression , please let me know if I am using wrong method for this :

    var bufferObject = new Buffer.from(JSON.stringify(jsonFileContent));
    var zlib = require('zlib');
    zlib.gunzip(bufferObject, function(err, zipped) {
        if(err) {
          console.log("error",err);
          next(err);
        }
        else {
          console.log("zipped",zipped);
        }
      })

Solution 2:[2]

Using zlib.gzip, you can create .gz archive with the above code. If you want to create .zip archive you can use the AdmZip.

 // The following code uploads data as zipped text file to S3
      var AdmZip = require("adm-zip");
      const AWS = require('aws-sdk');
      const s3 = new AWS.S3({
          accessKeyId: 'your access key id',
          secretAccessKey: 'your secret access key'
      });
    
      async function uploadFileAsZipToS3(bucket, key, data) {
          var zip = new AdmZip();
          zip.addFile("test.txt", Buffer.from(data, "utf8"), "entry comment goes here");
    
          var buf = zip.toBuffer();
          const params = {
              Bucket: bucket,
              Key: key,
              Body: buf,
              ContentType: 'application/zip',
          };
    
          const result = await s3.putObject(params).promise();
      }

uploadFileAsZipToS3('your bucket name','test.zip','content for the text file goes here')

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Ankit Uniyal
Solution 2

Related Questions