'Read mth row to nth row from a buffer data in node js

I am getting a large file data in chunks from frontend. The format of this data is base64.

As the file is too large I need to read or convert this base64 format to any other format if required and send only 1000 records at a time as a response to the frontend. So, first we need to save this base64 data somewhere and whenever a request comes from frontend i.e, the request could be asking for 1001 - 2000 records or 45001 - 46000 records, I need to send back those particular rows as a response to frontend in json format.

How can I fetch a particular rows like mth - nth?

This is my router file, where I am using multer

 const express = require("express");
 const router = express.Router();
 var multer  = require('multer')();
 const uploadDocument = require("./RA/UploadDocument")

 router.post("/uploadDocument", multer.single('slicedFile'), uploadDocument)

Below is the uploadDocument file where I have concatenated the chunks received from frontend

 const XLSX = require('xlsx') 
 var buffer = [];
      buffer.push(req.file.buffer);
    if(req.body.isLastBlob==='true'){
        let buf = Buffer.concat(buffer);
        var base64 = (buf.toString('base64'));
        var pre="data:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;base64,";
        base64 = pre+base64;
        buffer = [];  

        var docdata = base64.split(',').pop();
        var bufdata = Buffer.from(docdata, 'base64');
          let wb= XLSX.read(bufdata, {type: "buffer", sheetRows: 100});
          const wsname = wb.SheetNames[0];
          const ws = wb.Sheets[wsname];
          const exceldata = XLSX.utils.sheet_to_json(ws);
        console.log(exceldata)
    }

Consider this is my excel file

PlatformName    SerialNo    ICC_Serial_No
IS20    4520863450  9018229990
IS20    9965713930  7648298200
IS20    7365206700  4787377990
IS20    8417755210  2335387030
IS20    6499775990  811401940
IS20    3329719760  2241961760
IS20    540691670   4210369020
IS20    4805607930  6071220080
IS20    3537648760  5137843290

When there are upto 50000 records of such rows it is giving me proper output similar to below for console.log(exceldata)

 [
  {
    'PlatformName': 'IS20',
    'SerialNo': 4520863450,
    'ICC_Serial_No': 9018229990
  },
  {
    'PlatformName': 'IS20',
    'SerialNo': 4520863450,
    'ICC_Serial_No': 9018229990
  },
  {
    'PlatformName': 'IS20',
    'SerialNo': 4520863450,
    'ICC_Serial_No': 9018229990
    },
    ...... more 49080 rows
  ]

But when I upload huge file which have around 1000000 records it is giving me output similar to

 [
  {
    '0': 'IS20',
    '1': 4520863450,
      __EMPTY: 'SAF',
  },
  {
    'IS20': '9018229990',
    '9018229990': 4520863450,
    __EMPTY: 'SAF',
  },
  {
    __EMPTY: 'SAF',
     __EMPTY_2: 'Subscription fees',
    'IS20': 9018229990
    }
  ]

How can I fetch a particular rows like mth - nth and send it back to frontend as a response every time it is asked for passing the m and n in the request? Please help



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source