'how to read all the files in a s3 folder

The s3 folder contains large number of files(eg 1 million file), each file contains only one json record compressed with gz format. how to read all data/files one by one.

we have tried

const AWS = require('aws-sdk');
var s3 = new AWS.S3();
var LineStream = require('byline').LineStream;
    var params = {
        Bucket: 'xxxx',
        params: 'xxxxx'
    };

    s3.headObject(params, function (err, data) {
            
        var stream = s3.getObject(params).createReadStream();
        stream.on('data', (data) => {
            console.log('Served by Amazon S3: ' + data);
        });

    });

any other approach is also fine

thanks in advance Sundar



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source