node process restart upload to s3 without configuration

Question

I have an api GETendpoint that uploads a file of 1GB (1080000108B precisely) to my s3 bucket, this is the code :

var express = require('express'),
    HttpStatus = require('http-status-codes'),
    morgan = require('morgan'),
    packageConfig = require('./package.json'),    
    fs = require('fs'),    
    AWS = require('aws-sdk')
    //zlib = require('zlib')

express()
    .use(morgan('combined'))
    .get('/api/v1', function (req, res) {

        var s3 = new AWS.S3({
            accessKeyId:'my access key id',
            secretAccessKey:'my secret key access'
        });

        var rs = fs.createReadStream('./big.file')//.pipe(zlib.createGzip());

        s3.upload({
            Body:rs,
            Key:Date.now().toString(),
            Bucket:'my bucket'
        },function(err,data){
            console.log(err,data)
            res.status(HttpStatus.OK).send(err + data)
        }).on('httpUploadProgress', function(evt) {
            console.log('Progress:', evt.loaded, '/', evt.total); 
          })
        }).listen(process.env.PORT || 5000, function () {
        var address = this.address();
        var service = packageConfig.name + ' version: ' + packageConfig.version + ' ';
        console.log('%s Listening on %d', service, address.port);
    });

I open a first terminal A and run this server

node myserver.js

Then, I open a second terminal B and call the endpoint with the following command :

curl localhost:5000/api/v1

Immediatly, the upload progress start to be displayed in the terminal A :

Progress: 5242880 / 1080000108
Progress: 10485760 / 1080000108
Progress: 15728640 / 1080000108
...
...
Progress 513802240 / 1080000108

But when the progress reached 513802240 / 1080000108, my terminal B (where the curl request was made) displays the following message :

curl: (52) Empty reply from server

At this moment, I didn't find anything "strange" (I imagined a timeout error), but what intrigues me, it's that my terminal A (the server), received the same request again and continue the upload process from where it was left, and that, until the upload completes :

Progress: 513802240 / 1080000108
::ffff:127.0.0.1 - - [12/Apr/2020:08:50:26 +0000] "GET /api/v1 HTTP/1.1" - - "-" "curl/7.58.0"
Progress: 519045120 / 1080000108
Progress: 524288000 / 1080000108
...
Progress: 1074790400 / 1080000108
Progress: 1080000108 / 1080000108
null { Location:
   'https://my bucket name.s3.eu-west-3.amazonaws.com/1586681306481',
  Bucket: 'my bucket name',
  Key: '1586681306481',
  ETag: '"218ad7075293afb6d725aa60ddcb0142-206"' }

So here are my following questions :

• who send the same request to my server as my curl terminal is closed and have received the response:

  curl: (52) Empty reply from server

• even if I understand the first point, how the server knows that it has to restart (if it really restarts?) from the previous fail progress, why does it not restart from beggining as it's maybe another request from a different user coming

• optionnal point : I know that I can accelerate my upload with a gzip compressed file, but I first wanted to test without any optimization and then I encoutered this strange behaviour which I want to understand first before applying the gzip optimization

Thanks in advance

Answer 1

After a log of google search and console.log (I am a total newbie to node.js and s3), I have finally understand what's happening :

At a certain point (2 minutes is the default value), the server closes the current connection, so curl receives an empty response and stop, but the server continues to upload the file as it has not received any stop "order".

Answer 2

Here you have to set httpOptios for timeout:

var s3_data = new AWS.S3({
  httpOptions: {timeout: 0}, 
});