K8s: How do i exposing my app though ingress

Question

I'm trying to expose my application in K8s.

I have set up an Ingress Controller which gives the following properties:

kubectl get svc,pods --namespace ingress
NAME                                  TYPE           CLUSTER-IP       EXTERNAL-IP           PORT(S)                      AGE
service/ingress-nginx-nginx-ingress   LoadBalancer   10.254.234.220   111.111.111.111       80:32097/TCP,443:32755/TCP   16d

NAME                                               READY   STATUS    RESTARTS   AGE
pod/ingress-nginx-nginx-ingress-68848c49f8-54lx4   1/1     Running   0          12d
pod/ingress-nginx-nginx-ingress-68848c49f8-5dx97   1/1     Running   0          12d
pod/ingress-nginx-nginx-ingress-68848c49f8-89grn   1/1     Running   0          12d

So when i go to https://111.111.111.111 (not the real address) in my browser, I get:

This is the ingress for my app in values.yaml`

ingress:
  name: externalIngress
  enabled: true
  type: LoadBalancer
  ingressClassName: "nginx"
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
  hosts:
    - host:
      paths:
      - path: "/"
        backend:
          serviceName: my-app
          servicePort: 80

using the template:

{{- if .Values.ingress.enabled -}}
{{- $fullName := include "my-app.fullname" . -}}
{{- $svcPort := .Values.service.port -}}
{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
  name: {{ $fullName }}
  labels:
    {{- include "my-app.labels" . | nindent 4 }}
  {{- with .Values.ingress.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  {{- if .Values.ingress.tls }}
  tls:
    {{- range .Values.ingress.tls }}
    - hosts:
        {{- range .hosts }}
        - {{ . | quote }}
        {{- end }}
      secretName: {{ .secretName }}
    {{- end }}
  {{- end }}
  rules:
    {{- range .Values.ingress.hosts }}
    - host: {{ .host | quote }}
      http:
        paths:
          {{- range .paths }}
          - path: {{ .path }}
            backend:
              serviceName: {{ $fullName }}
              servicePort: {{ $svcPort }}
          {{- end }}
    {{- end }}
  {{- end }}

I have a service set up for the ingress:

kubectl get service --namespace=ingress -o yaml
apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    annotations:
      meta.helm.sh/release-name: ingress-nginx
      meta.helm.sh/release-namespace: ingress
    creationTimestamp: "2022-04-29T14:51:36Z"
    labels:
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: ingress-nginx-nginx-ingress
      helm.sh/chart: nginx-ingress-0.10.4
    name: ingress-nginx-nginx-ingress
    namespace: ingress
    resourceVersion: "4711523"
    selfLink: /api/v1/namespaces/ingress/services/ingress-nginx-nginx-ingress
    uid: a8183382-151f-499b-b06f-0a189e302226
  spec:
    clusterIP: 10.254.234.220
    externalTrafficPolicy: Local
    healthCheckNodePort: 31343
    ports:
    - name: http
      nodePort: 32097
      port: 80
      protocol: TCP
      targetPort: 80
    - name: https
      nodePort: 32755
      port: 443
      protocol: TCP
      targetPort: 443
    selector:
      app: ingress-nginx-nginx-ingress
    sessionAffinity: None
    type: LoadBalancer
  status:
    loadBalancer:
      ingress:
      - ip: 111.111.111.111
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

I have a service set up for my app

kubectl get service --namespace=my-app -o yaml
apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    annotations:
      meta.helm.sh/release-name: my-app
      meta.helm.sh/release-namespace: my-app
    creationTimestamp: "2022-05-12T12:11:47Z"
    labels:
      app.kubernetes.io/instance: my-app
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: my-app
      app.kubernetes.io/version: 1.16.0
      helm.sh/chart: my-app-0.1.0
    name: my-app
    namespace: my-app
    resourceVersion: "7395487"
    selfLink: /api/v1/namespaces/my-app/services/my-app
    uid: d72661e1-be92-42f5-a030-65bdf4da06c8
  spec:
    clusterIP: 10.254.153.184
    ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    selector:
      app.kubernetes.io/instance: my-app
      app.kubernetes.io/name: my-app
    sessionAffinity: None
    type: ClusterIP
  status:
    loadBalancer: {}
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

And the following ingress:

kubectl get ingress --namespace=my-app -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class: nginx
      meta.helm.sh/release-name: my-app
      meta.helm.sh/release-namespace: my-app
      nginx.ingress.kubernetes.io/rewrite-target: /$2
    creationTimestamp: "2022-05-16T11:38:13Z"
    generation: 3
    labels:
      app.kubernetes.io/instance: my-app
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: my-app
      app.kubernetes.io/version: 1.16.0
      helm.sh/chart: my-app-0.1.0
    name: my-app
    namespace: my-app
    resourceVersion: "8501216"
    selfLink: /apis/extensions/v1beta1/namespaces/my-app/ingresses/my-app
    uid: 27fa844e-7672-47ff-94b2-b9c18492cb5d
  spec:
    rules:
    - http:
        paths:
        - backend:
            serviceName: my-app
            servicePort: 80
          path: /
  status:
    loadBalancer: {}
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

A clusterRole for the ingress with the following permissions:

Based on this, I'd expect to be able to go to http://111.111.111.111/ and be routed to my-app but I just get the 404.

The ingress logs give:

 controller.go:3050[] Using the DEPRECATED annotatio │
│ n 'kubernetes.io/ingress.class'. The 'ingressClassName' field will be ignored.

and

event.go:285[] Event(v1.ObjectReference{Kind:"Ingre │
│ ss", Namespace:"my-app", Name:"my-app", UID:"27fa844e-7672-47ff-94b2-b9c18492cb5d", APIVersion:"networking.k8s.io/v1beta1", ResourceVersio │
│ n:"8499566", FieldPath:""}): type: 'Warning' reason: 'Rejected' my-app/my-app was rejected: with error: spec.rules[0].host: Required value

What am I doing wrong!?

Answer 1

Its usually easier to share the yaml AFTER the templating :)

In order for this to work you need-

( in the ingress namespace)

1. A way to get traffic from the outside world to your cluster, such as an aws load balancer - this one is infra specific
2. svc to map traffic to the ingress controller (could be of type LoadBalancer in supported cloud deployments)
3. nginx-ingress-controller running
4. a kubernetes cluster role + binding that gives the ingress controller permission to see ingresses and services in every namespace

(in your app namespace)

5. A pod with your app running in it
6. A svc that matches your pod
7. An ingress that matches your svc

Since you didn't mention a service, my guess is thats your issue- trying to send traffic directly to the pod instead of through a service. If my guess is wrong, the logs from the nginx ingress controller should be more clear. You probably don't need to run 3 of those especially to start btw ;)