'Can't read ingress-nginx controller pod's logs after configuring filebeat

currently we are working on persist ingress-nginx logs,we implemented filebeat sidecar put forlogs to logstash,but we cannot read acces.log and error.log this path{/var/log/nginx/access.log} now we required read logs for this path, please give solution for this issue

ingress-nginx deployed manifest files like this

filebeat configmap

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-configmap
  namespace: ingress-nginx
data:
  filebeat.yml: |
    filebeat:
      config:
        modules:
          path: /usr/share/filebeat/modules.d/*.yml
          reload:
            enabled: true
      modules:
      - module: nginx
        access:
          var.paths: ["/var/log/nginx/access.log*"]
        error:
          var.paths: ["/var/log/nginx/error.log*"]
    output:
      logstash:
        hosts: ["logstash-logstash-headless:9600"]
        loadbalance: true

deployment file

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.10.1
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.41.2
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/component: controller
  revisionHistoryLimit: 10
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    spec:
      dnsPolicy: ClusterFirst
      containers:
        - name: controller
          image: k8s.gcr.io/ingress-nginx/controller:v0.41.2@sha256:1f4f402b9c14f3ae92b11ada1dfe9893a88f0faeb0b2f4b903e2c67a0c3bf0de
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
            - --annotations-prefix=nginx.ingress.kubernetes.io
            - --enable-ssl-passthrough
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          livenessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 5
          readinessProbe:
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
            - name: prometheus
              containerPort: 10254
              protocol: TCP
          volumeMounts:
            - name: webhook-cert
              mountPath: /usr/local/certificates/
              readOnly: true
            - name: nginx-logs
              mountPath: var/log/nginx
          resources:
            requests:
              cpu: 100m
              memory: 90Mi
        - name: filebeat-nginx
          image: docker.elastic.co/beats/filebeat:7.13.0
          volumeMounts:
            - name: nginx-logs
              mountPath: var/log/nginx
            - name: filebeat-config
              mountPath: /usr/share/filebeat/filebeat.yml
              subPath: filebeat.yml      
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
        - name: nginx-logs
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission
        - name: filebeat-config
          configMap:
            name: filebeat-configmap
            items:
              - key: filebeat.yml
                path: filebeat.yml 


Solution 1:[1]

It looks like there is a typo in the file path in the following code:

            - name: nginx-logs
              mountPath: var/log/nginx
          resources:
            requests:
              cpu: 100m
              memory: 90Mi
        - name: filebeat-nginx
          image: docker.elastic.co/beats/filebeat:7.13.0
          volumeMounts:
            - name: nginx-logs
              mountPath: var/log/nginx

The file paths need to be changed from var/log/nginx to /var/log/nginx (forward slash in the beginning of the path)

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Kiran Indukuri