Upload generated PNG to amazon S3 from node on lambda

Question

I'm downloading 2 png images from their respective URLs then using pixelmatch to compare them and generate a 3rd image.

I'm then trying to upload the third image to an S3 bucket but I'm struggling. Code follows.

promise1 and promise2 are the two calls to get png1 and png2 respectively. The dimensions1 array used below is created in promise1. PNG is the pngjs package.

const promArr = [promise1, promise2];

Promise.all(promArr).then(() => {
  const diff = new PNG({ width: dimensions1[0], height: dimensions1[1] });
  const pix = pixelmatch(png1.data, png2.data, diff.data, dimensions1[0], dimensions1[1], { threshold: 0.1 });
  const size = dimensions1[0] * dimensions1[1];
  diff.pack().pipe(fs.createWriteStream(`/tmp/${targetHash}.png`));
  const percentage = ((pix / size) * 100);

  const fileBuffer = fs.readFileSync(`/tmp/${targetHash}.png`);

  const s3 = new AWS.S3();
  s3.putObject({
    ACL: 'public-read',
    Key: targetFilename,
    Body: fileBuffer,
    Bucket: targetBucket,
    ContentType: 'image/png',
  }, (err) => {
    if (err) {
      console.warn(err);
      cb(err);
    } else {
      cb(null, {
        percentage,
        hash: `${targetFilename}`,
        key: `${targetFilename}`,
        bucket: targetBucket,
        url: `${event.stageVariables.endpoint}${targetFilename}`,
      });
    }
    return;
  });
})
.catch((err) => {
  console.warn(`error: ${err}`);
  cb(err);
});

This returns a URL to the image which Preview on mac reports to be empty. This suggests to me that I have an async issue.

I have also tried passing diff.pack() as the body and got the error: Cannot determine length of [object Object]. I found a discussion of this error here.

I feel really close to a solution but I'm not quite getting there. Can anyone help? Thank you

Answer 1

You could convert the stream to a buffer as mentioned in this article.

If you are using node >= 17.5.0:

const streamToBuffer = (stream) => Buffer.concat(await stream.toArray());

If you are using node < 17.5.0:

const streamToBuffer = (stream) =>
  new Promise((resolve, reject) => {
    const chunks = [];
    stream.on("data", (chunk) => chunks.push(chunk));
    stream.once("end", () => resolve(Buffer.concat(chunks)));
    stream.once("error", reject);
  });

And then implement it in your code:

const width = dimensions1[0];
const height = dimensions1[1];

const diff = new PNG({ width, height });

pixelmatch(
  png1.data,
  png2.data,
  diff.data,
  width,
  height,
  { threshold: 0.1 }
);

const diffBuffer = await streamToBuffer(diff.pack());

await s3Client
  .send(
    new PutObjectCommand({
      Bucket: /* Bucket */,
      Key: /* Key */,
      Body: diffBuffer,
      ContentType: "image/png",
    })
  )