'How to do deferred jobs in Node API (Express)

I have an API in Node with many endpoints. It works well but there is one that can have large requests run up to 1 hour of processing which can often break. I'm thinking of only sending back a url on a request where you can check the status of the request and then download it once it's ready. What would be the best way to handle a queue of jobs in Node for this?

Sample code below for the current endpoint

const router = express.Router();
const schema = joi.object().keys // This is a schema to validate the json input

router.post('/', async (req, res) => {
  let conn = await connect(); // Util method that connects to a Q/KDB server
  let request = req.body;

  joi.validate(request, schema, (err, _result) => {
    if (err) {
      res.status(400).send({ error: err['details'][0]['message'] });
    }
    else {
      let qRequest = buildRequest(request); // Util function to build request
      // Connect to Q/KDB server with node-q package and process request
      conn.k('api.process', qRequest, function(err, resp) {
        if (err) {
          log // Write to log
          res.status(400).send({ error: err['details'][0]['message']
        }
        else {
          res.status(200).send(resp);
        }
      });
    }
  });

});

EDIT: I have found that I basically just have to build a job queue with job ids corresponding to them. The package Bull seems to be good but I don't want to add another dependency such as Redis.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source