Change in max-old-space-size between nodejs 16 and nodejs 14?

Question

We are using --max-old-space-size=8192 to run our complete E2E jest 26 tests with npm test.

node --max-old-space-size=8192 node_modules/jest/bin/jest --runInBand --coverage --detectOpenHandles --logHeapUsage --no-cache

We upgraded to node 16.14.2 and suddenly the tests stopps at exactly 4G with OOM under windows as well as Ubuntu 20.04.4 LTS.

The same behavior with node 17.8.0

I switched back to node 14.18.1 and see the following performance graph with process explorer.

With node 16 I get OOM at 4G in the beginning of the E2E test.

<--- Last few GCs --->

[14184:00000277700CA440]  1097059 ms: Mark-sweep (reduce) 2799.4 (3161.8) -> 2798.8 (3123.2) MB, 1520.8 / 0.4 ms  (average mu = 0.099, current mu = 0.064) last resort GC in old space requested
[14184:00000277700CA440]  1098475 ms: Mark-sweep (reduce) 2798.8 (3123.2) -> 2798.7 (3116.2) MB, 1416.0 / 1.6 ms  (average mu = 0.053, current mu = 0.000) last resort GC in old space requested

I switched between the node versions with nvm-windows.

The packages were all installed with npm from node16. They run perfectly on node 14.

Tried other several space related v8-options but no positive effect on node 16 and 17.

Didn't want to open an issue of github/node yet as it cannot be isolated easily.

Any suggestions?

Update:

My first deep finding in node 16 V8 is that --huge-max-old-generation-size is now true be default.

This limits the memory to 4G.

See also https://github.com/v8/v8/commit/b2f75b008d14fd1e1ef8579c9c4d2bc7d374efd3.

And Heap::MaxOldGenerationSize And Heap::HeapSizeFromPhysicalMemory

The max-old-space is limited down to 4G there as far as I understood. (At least when huge-old-space is on)

Now setting --no-huge-max-old-generation-size --max-old-space-size=8192 still has no effect and OOM at 4G again.

Update 2:

I tracked the v8 heap statistics and see just before the OOM at 4G following infos from v8.getHeapSpaceStatistics() and v8.getHeapStatistics()

total_heap_size               : 3184 MB
total_heap_size_executable    : 127 MB
total_physical_size           : 3184 MB
total_available_size          : 9162 MB
used_heap_size                : 2817 MB
heap_size_limit               : 12048 MB
malloced_memory               : 2 MB
peak_malloced_memory          : 44 MB
does_zap_garbage              : 0 MB
number_of_native_contexts     : 0 MB
number_of_detached_contexts   : 0 MB
read_only_space               : size : 0 MB, used: 0 MB, avail: 0 MB, phy: 0 MB
old_space                     : size : 2425 MB, used: 2111 MB, avail: 268 MB, phy: 2425 MB
code_space                    : size : 127 MB, used: 110 MB, avail: 8 MB, phy: 127 MB
map_space                     : size : 44 MB, used: 39 MB, avail: 4 MB, phy: 44 MB
large_object_space            : size : 555 MB, used: 541 MB, avail: 0 MB, phy: 555 MB
code_large_object_space       : size : 0 MB, used: 0 MB, avail: 0 MB, phy: 0 MB
new_large_object_space        : size : 0 MB, used: 0 MB, avail: 15 MB, phy: 0 MB
new_space                     : size : 32 MB, used: 13 MB, avail: 2 MB, phy: 32 MB

<--- Last few GCs --->

[7940:000001B87F118E70]   546939 ms: Mark-sweep (reduce) 2774.1 (3123.5) -> 2773.6 (3084.7) MB, 498.6 / 0.3 ms  (average mu = 0.080, current mu = 0.044) last resort GC in old space requested
[7940:000001B87F118E70]   547453 ms: Mark-sweep (reduce) 2773.6 (3084.7) -> 2773.4 (3077.2) MB, 513.2 / 0.3 ms  (average mu = 0.040, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

Update 3:

Upgraded to jest 27.5.1 and no difference. node 14 is fine but node 16/17 got stuck at 4G while their heap statistics report huge amount of available space.