'Piping Node/Express Request to Response Hangs w/ NGINX

Update: I'm still having this issue, but it's not Node that's the problem. It's my NGINX proxy. When I make the request directly to Node, it works. When it goes through my NGINX proxy, I have the issue.

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # tried removing
        proxy_set_header Host $host; # tried removing
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1; # tried removing
        proxy_set_header Upgrade $http_upgrade; # tried removing
        proxy_set_header Connection "upgrade"; # tried removing
        proxy_read_timeout 900; # tried removing
    }

All other settings are default.

I'm not sure what I'm doing wrong here, but I'm trying to pipe request input to some transformers etc and stream a response and it seems to hang once the request body reaches a certain size. I took out all of my transformers and custom code, yet the issue is still there. Seems that this is the root of it:

app.post('/', function(req, res, next) {
  req.pipe(res);
});

This hangs once I post more than a few megabytes to it.

It works okay for small requests. But one thing I noticed is that all of the requests after the first one according to the Express console output execute in a couple of milliseconds. Much faster than the original (and smaller) request. According to the time it takes to make the call, these times are not accurate, so it seems something is wrong with how I'm "ending" the response, or something like that.

[nodemon] restarting due to changes...
[nodemon] starting `node ./bin/www npm start`
POST / 200 11.606 ms - - (small request)
POST / 200 1.729 ms - - (bigger request)
POST / 200 2.054 ms - - (bigger request)
POST / 200 1.689 ms - - (bigger request)

Interestingly enough, these work fine. Console time measurements look valid too:

app.post('/', function(req, res, next) {
  req.pipe(fs.createWriteStream("/tmp/x"))
    .on("finish", () => {
      fs.createReadStream("/tmp/x").pipe(res);
    })
});
app.post('/', function(req, res, next) {
  req.pipe(transform((d, c) => c(null, "")))
    .pipe(res);
});

This does not:

app.post('/', function(req, res, next) {
  req.pipe(transform((d, c) => c(null, d)))
    .pipe(res);
});

I'm sure this is a noob mistake but I can't seem to figure out what I'm doing wrong.

Full code:

const createError = require('http-errors');
const express = require('express');
const fs = require('fs');
const stream = require('stream');
const logger = require('morgan');
const fetch = require('node-fetch');
const cors = require('cors');

const app = express();

app.use(logger('dev'));
app.use(cors());

app.post('/', function(req, res, next) {
  req.pipe(res);
});

/*
app.post('/', function(req, res, next) {
  req.pipe(transform((d, c) => c(null, d)))
    .pipe(res);
});

app.post('/', function(req, res, next) {
  req.pipe(transform((d, c) => c(null, "")))
    .pipe(res);
});

app.post('/', function(req, res, next) {
  req.pipe(fs.createWriteStream("/tmp/x"))
    .on("finish", () => {
      fs.createReadStream("/tmp/x").pipe(res);
    })
});
*/
    
app.use(function(req, res, next) {
  next(createError(404));
});

app.use(function(err, req, res, next) {
  res.locals.message = err.message;
  res.locals.error = req.app.get('env') === 'development' ? err : {};
  res.status(err.status || 500);
  res.send(err.message);
  console.log(err);
});

module.exports = app;


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source