Node.js spawn: Keep StdOut and StdErr in the original order

Question

Trying to run windows batch script from node.js v12.6.0 and capture its output in real time and in correct order.

But order of stdout and stderr is often (not always, but in 80% cases) mixed during the tests.

How to keep stdout and stderr in the original order?

---

This is javascript snippet I use for testing:

const spawn = require('child_process').spawn;

// Using 'inherit' to fix:
// "ERROR: Input redirection is not supported, exiting the process immediately".

const options = {
    stdio: [
        'inherit', // StdIn.
        'pipe',    // StdOut.
        'pipe'     // StdErr.
    ],
};

const child = spawn('exectest.cmd', options);

let mergedOut = '';

child.stdout.setEncoding('utf8');
child.stdout.on('data', (chunk) => {
    process.stdout.write(chunk);
    mergedOut += chunk;
});

child.stderr.setEncoding('utf8');
child.stderr.on('data', (chunk) => {
    process.stderr.write(chunk);
    mergedOut += chunk;
});

child.on('close', (code, signal) => {
    console.log('-'.repeat(30));
    console.log(mergedOut);
});

I have tried to redirect stderr to stdout with:

child.stderr.pipe(child.stdout);

And remove stderr listener (handle only stdout):

child.stderr.on

To avoid race conditions, but stderr is not displaying in console, nor been added to 'mergedOut'.

---

This is contents of the batch file I trying to run (exectest.cmd):

@Echo Off
ChCp 65001 >Nul

Echo 1 (stdout) ...
Echo 2 (stderr) ... 1>&2
Echo 3 (stderr) ... 1>&2
Echo 4 (stdout) ...
Echo 5 (stdout) ...
Echo 6 (stderr) ... 1>&2

---

Current output:

1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
6 (stderr) ...
4 (stdout) ...
5 (stdout) ...
------------------------------
1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
6 (stderr) ...
4 (stdout) ...
5 (stdout) ...

Expected output:

1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
4 (stdout) ...
5 (stdout) ...
6 (stderr) ...
------------------------------
1 (stdout) ...
2 (stderr) ...
3 (stderr) ...
4 (stdout) ...
5 (stdout) ...
6 (stderr) ...

---

EDIT 1:

With:

    stdio: [
        'inherit', // StdIn.
        'inherit', // StdOut.
        'inherit'  // StdErr.
    ],

Output order is reliably correct, so node.js itself seems to 'know' how to do this properly.

But in this case, I do not know how to capture the output:

child.stdout

is 'null', and attempts to listen to process stdout of node.js itself:

process.stdout.on('data' ...)

in any configuration gives 'Error: read ENOTCONN'.

---

EDIT 2:

If merge streams and listen to it this way:

const mergedStream = child.stdout.wrap(child.stderr);

mergedStream.on('data' ...);

We have one listener to both stdout and stderr, but ordering is still broken.

---

EDIT 3:

Is is possible to keep the correct order if you want to capture output OR display it.

To display proper output without capturing it, see 'EDIT 1'.

To capture proper output without displaying it real-time, just use:

child.stdout.on('data', (chunk) => {
    mergedOut += chunk;
});

child.stderr.on('data', (chunk) => {
    mergedOut += chunk;
});

But as soon as you try to reference to 'process.stdout / process.stderr' inside of the:

child.stdout.on('data' ...)

callback, ordering will break.

For example:

child.stdout.on('data', (chunk) => {
    process.stdout; // <-- This line will break everything.
                    // You do not even need to .write() to it!
});

Or if you try:

child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);

to avoid referencing to 'process' inside of the 'data' callback, ordering will break too.

Answer 1

Unfortunately, you can't guarantee ordering.

These are different pipes with independent buffering and behavior. The shell itself will sometimes get STDERR/STDOUT messages out of the intended order. This is typical of other applications as well, even outside of Node.js.

Answer 2

If you don't need to distinguish between STDOUT and STDERR, you can join them at the source (inside the batch file). The batch file then returns all of STDERR and STDOUT only as STDOUT

@Echo Off
ChCp 65001 >Nul
(
Echo 1 [stdout] ...
Echo 2 [stderr] ... 1>&2
Echo 3 [stderr] ... 1>&2
Echo 4 [stdout] ...
Echo 5 [stdout] ...
Echo 6 [stderr] ... 1>&2
) 2>&1