How to enable file_sd_configs for prometheus operator using additional_scrape_configs?

Question

I have got prometheus operator along with Thanos working. But now I would like to add prometheus file_sd_configs to the additional_scrape_configs. I have multiple jobs and they all refer to similar files for the targets, want to control them with file_sd_Configs.

---
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
  namespace: prometheus-vvs-metrics
  labels:
    app: prometheus
spec:
  podMetadata:
    labels:
      app: prometheus
  image: quay.io/prometheus/prometheus:v2.22.1
  nodeSelector:
    kubernetes.io/os: linux
  replicas: 2
  resources:
    requests:
      memory: 400Mi
  securityContext:
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: prometheus
  version: v2.22.1
  serviceMonitorSelector: {}
  additionalScrapeConfigs:
    name: additional-scrape-configs
    key: new-file.yaml
  thanos:
    version: v0.25.0
    baseImage: quay.io/thanos/thanos
    objectStorageConfig:
      key: thanos-storage-config.yaml
      name: thanos-storage-config

scrape configs are created a secret with file named new-file.yaml and secret name being additional-scrape-configs

- job_name: 'http-rules'
  metrics_path: /probe
  honor_labels: true
  params:
    module: [http_2xx]
  file_sd_configs:
    - files :
      - ./target.yml
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: <host-name>:9115

- job_name: 'ssh-rules'
  metrics_path: /probe
  honor_labels: true
  params:
    module: [ssh_banner] # Probe tcp port 22
  file_sd_configs:
    - files :
      - ./target.yml
  relabel_configs:
  # Ensure port is 22, pass as URL parameter
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: <host-name>:9115

Any help is appreciated.

Answer 1

I was able to change the configs to on prometheus to have configmaps in it. Modified the manifest as below.

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
  namespace: prometheus-vvs-metrics
  labels:
    app: prometheus
spec:
  configMaps:
   - prometheus-cm
  podMetadata:
    labels:
      app: prometheus
  image: quay.io/prometheus/prometheus:v2.22.1
  nodeSelector:
    kubernetes.io/os: linux
  replicas: 2
  resources:
    requests:
      memory: 400Mi
  securityContext:
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: prometheus
  version: v2.22.1
  serviceMonitorSelector: {}
  additionalScrapeConfigs:
    name: additional-scrape-configs
    key: new-file.yaml
  thanos:
    version: v0.25.0
    baseImage: quay.io/thanos/thanos
    objectStorageConfig:
      key: thanos-storage-config.yaml
      name: thanos-storage-config

I have provided all my targets in prometheus-cm ConfigMap and created it. now the prometheus pods have a mount point at /etc/prometheus/configmaps/promethues-cm/targets.yaml.

I changed my new-file.yaml to have those paths reflected.

- job_name: 'http-rules'
  metrics_path: /probe
  honor_labels: true
  params:
    module: [http_2xx]
  file_sd_configs:
    - files :
      - /etc/prometheus/configmaps/promethues-cm/target.yml
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: <host-name>:9115

- job_name: 'ssh-rules'
  metrics_path: /probe
  honor_labels: true
  params:
    module: [ssh_banner] # Probe tcp port 22
  file_sd_configs:
    - files :
      - /etc/prometheus/configmaps/promethues-cm/target.yml
  relabel_configs:
  # Ensure port is 22, pass as URL parameter
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: <host-name>:9115

That worked like a charm.