'Read mth row to nth row from a buffer data in node js
I am getting a large file data in chunks from frontend. The format of this data is base64.
As the file is too large I need to read or convert this base64 format to any other format if required and send only 1000 records at a time as a response to the frontend. So, first we need to save this base64 data somewhere and whenever a request comes from frontend i.e, the request could be asking for 1001 - 2000 records or 45001 - 46000 records, I need to send back those particular rows as a response to frontend in json format.
How can I fetch a particular rows like mth - nth?
This is my router file, where I am using multer
const express = require("express");
const router = express.Router();
var multer = require('multer')();
const uploadDocument = require("./RA/UploadDocument")
router.post("/uploadDocument", multer.single('slicedFile'), uploadDocument)
Below is the uploadDocument file where I have concatenated the chunks received from frontend
const XLSX = require('xlsx')
var buffer = [];
buffer.push(req.file.buffer);
if(req.body.isLastBlob==='true'){
let buf = Buffer.concat(buffer);
var base64 = (buf.toString('base64'));
var pre="data:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;base64,";
base64 = pre+base64;
buffer = [];
var docdata = base64.split(',').pop();
var bufdata = Buffer.from(docdata, 'base64');
let wb= XLSX.read(bufdata, {type: "buffer", sheetRows: 100});
const wsname = wb.SheetNames[0];
const ws = wb.Sheets[wsname];
const exceldata = XLSX.utils.sheet_to_json(ws);
console.log(exceldata)
}
Consider this is my excel file
PlatformName SerialNo ICC_Serial_No
IS20 4520863450 9018229990
IS20 9965713930 7648298200
IS20 7365206700 4787377990
IS20 8417755210 2335387030
IS20 6499775990 811401940
IS20 3329719760 2241961760
IS20 540691670 4210369020
IS20 4805607930 6071220080
IS20 3537648760 5137843290
When there are upto 50000 records of such rows it is giving me proper output similar to below for console.log(exceldata)
[
{
'PlatformName': 'IS20',
'SerialNo': 4520863450,
'ICC_Serial_No': 9018229990
},
{
'PlatformName': 'IS20',
'SerialNo': 4520863450,
'ICC_Serial_No': 9018229990
},
{
'PlatformName': 'IS20',
'SerialNo': 4520863450,
'ICC_Serial_No': 9018229990
},
...... more 49080 rows
]
But when I upload huge file which have around 1000000 records it is giving me output similar to
[
{
'0': 'IS20',
'1': 4520863450,
__EMPTY: 'SAF',
},
{
'IS20': '9018229990',
'9018229990': 4520863450,
__EMPTY: 'SAF',
},
{
__EMPTY: 'SAF',
__EMPTY_2: 'Subscription fees',
'IS20': 9018229990
}
]
How can I fetch a particular rows like mth - nth and send it back to frontend as a response every time it is asked for passing the m and n in the request? Please help
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
