'twitter.com Fetch NodeJS 404 Reply

I'm always getting a 404 reply when querying twitter.com (not the API, the homepage), I've used the exact same query than Edge is using.

const http2 = require('node:http2');
const fs = require('node:fs');
const client = http2.connect('https://twitter.com', {});
client.on('error', (err) => console.error(err));

const req = client.request({ 
    ':path': '/',
    ':method': 'GET',
    ':authority': `twitter.com`,
    accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US',
    'cache-control': 'no-cache',
    dnt: '1',
    pragma: 'no-cache',
    referer: 'https://twitter.com/',
    "sec-ch-ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"101\", \"Microsoft Edge\";v=\"101\"",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": "\"Windows\"",
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "same-origin",
    "sec-fetch-user": "?1",
    "sec-gpc": "1",
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36 Edg/101.0.1210.32"
    

});

req.on('response', (headers, flags) => {
  for (const name in headers) {
    console.log(`${name}: ${headers[name]}`);
  }
});

req.setEncoding('utf8');
let data = '';
req.on('data', (chunk) => { data += chunk; });
req.on('end', () => {
  //console.log(`\n${data}`);
  client.close();
});
req.end();

Sorry to ask but I'm stuck on this for hours and I simply want to retrieve the homepage of twitter.com (or other websites) but I keep getting this 404 while the browser returns 200.

If I do a simple wget https://twitter.com then it's all good, no 404 message, so it has to do with NodeJS way I'm doing I guess, I do not wish to use a NPM package but simply understand what is the problem here.

Thank you



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source