'Dynamic filename in Winston dailyrotate for Promtail/Loki/Grafana

My NodeJS application writes logs with Winston. These logs then will be picked up by Promtail, to be saved to S3 by Loki and then processed in a dashboard in Grafana.

I want to create logs in Winston with dailyrotation of 30m. I want the logs to first be stored in my folder "/home/gad-web/gad-logs" when they are still being appended. And when they are rotated I want to move them to "/home/gad-web/gad-logs-rotated". Promtail will be looking at this specific folder.

I want to use dynamic filenames for different logs being written out, so that I can easily assign static labels to each file separetly using Promtail, rather than having to process each log line and assign a dynamic label to each line of log in one large file.

my file logger.mjs looks like this (formats, levels and other irrelevant data is left out):

const logDir = '/home/gad-web/gad-logs'
const logDirRotated = '/home/gad-web/gad-logs-rotated'

let winstonGdprProofFormat = winston.format.combine(...)

let winstonDailyRotateFileTransport = new winston.transports.DailyRotateFile({
  frequency: '30m',
  format: winstonGdprProofFormat,
  filename: `${logDir}/all-gdpr-proof-%DATE%.log`,
  datePattern: 'YYYY-MM-DD HH-mm',
})

// Move the file to another location after it is rotated, so it can be picked up by Promtail
winstonDailyRotateFileTransport.on('rotate', function (oldFilenamePath, newFilenamePath) {
  let pathToMoveTo = `${logDirRotated}/${path.basename(oldFilenamePath)}`
  fs.rename(oldFilenamePath, pathToMoveTo, function (err) {
    if (err) throw err
  })
})

let winstonTransports = []
if (process.env.environment !== 'local') {
  winstonTransports.push(winstonConsoleTransport)
  winstonTransports.push(winstonDailyRotateFileTransport)
} else {
  winstonTransports.push(winstonConsoleWithColorsTransport)
}

const logger = winston.createLogger({
  level: process.env.environment !== 'local' ? 'info' : 'debug',
  levels: winstonLevels,
  transports: winstonTransports,
})

export function log (obj) {
  let { level, requestId, method, uri, msg, time, data } = obj

  if (!level) {
    level = 'info'
  }

  logger.log({
    level: level,
    requestId: requestId,
    method: method,
    uri: uri,
    msg: msg,
    time: time,
    data: data,
  })
}

It is being called in files that write logs like this:

import { log } from '../config/logger.mjs'
...
function writeRequestLog (start, request, requestId) {
  let end = new Date().getTime()
  let diff = end - start
  log({ level: 'info', requestId: requestId, method: request.method, uri: request.path, msg: null, time: `${diff}ms`, data: JSON.stringify(request.query) })
}

Since the file is imported directly, it is immediately executed, and the winstonDailyRotateFileTransport is created using ${logDir}/all-gdpr-proof-%DATE%.log as the filename. How do I go around this instantiating this with a filename, so that I get daily rotated log files of 30minutes for a bunch of dynamically created different files?

I tried creating a Class in JS, but I quickly got into trouble because of the .on('rotate', ...) defined for the winstonDailyRotateFileTransport, and i'm also not sure of other implications creating a class for this might have (since this logger will be used a lot of times in my code)



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source