Reading metrics in .csv format into prometheus from S3

Question

I have several hundred IOT devices uploading performance metrics in .csv format into an s3 bucket. I need to get those metrics into my already existing prometheus/alertmanager monitoring solution. I'm attempting to use an existing exporter called mtail (https://github.com/google/mtail), but for the life of me I can't figure out how to use it to parse a CSV file. The documentation says to use awk language, but the way to set a custom delimeter in awk is with -f, but that's not relevant to mtail programs (or is it?).

If there's a better way to get .csv-formatted metrics from S3 into prometheus I'm open to suggestions.

Answer 1

I think VictoriaMetrics might be a good option for You https://docs.victoriametrics.com/?highlight=Import#how-to-import-csv-data

Also You might consider its agent for scraping on iot devices.