Secure mTLS communication within Istio-knative services + external requests

Question

We are converting existing k8s services to use istio & knative. The services receive requests from external users as well as from within the cluster. We are trying to setup Istio AuthorizationPolicy to achieve the below requirements:

1. Certain paths (like docs/healthchecks) should not require any special header or anything and must be accessible from anywhere
2. Health & metric collection paths required to be accessed by knative must be accisible only by knative controllers
3. Any request coming from outside the cluster (through knative-serving/knative-ingress-gateway basically) must contain a key header matching a pre-shared key
4. Any request coming from any service within the cluster can access all the paths

Below is a sample of what I am trying. I am able to get the first 3 requirements working but not the last one...

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: my-svc
  namespace: my-ns
spec:
  selector:
    matchLabels:
      serving.knative.dev/service: my-svc
  action: "ALLOW"
  rules:
    - to:
        - operation:
            methods:
              - "GET"
            paths:
              - "/docs"
              - "/openapi.json"
              - "/redoc"
              - "/rest/v1/healthz"

    - to:
        - operation:
            methods:
              - "GET"
            paths:
              - "/healthz*"
              - "/metrics*"
      when:
        - key: "request.headers[User-Agent]"
          values:
            - "Knative-Activator-Probe"
            - "Go-http-client/1.1"

    - to:
        - operation:
            paths:
              - "/rest/v1/myapp*"
      when:
        - key: "request.headers[my-key]"
          values:
            - "asjhfhjgdhjsfgjhdgsfjh"

    - from:
        - source:
            namespaces:
              - "*"

We have made no changes to the mTLS configuration provided by default by istio-knative setup, so assume that the mtls mode is currently PERMISSIVE.

Details of tech stack involved

• AWS EKS - Version 1.21
• Knative Serving - Version 1.1 (with Istio 1.11.5)

Answer 1

I'm not an Istio expert, but you might be able to express the last policy based on either the ingress gateway (have one which is listening only on a ClusterIP address), or based on the SourceIP being within the cluster. For the latter, I'd want to test that Istio is using the actual SourceIP and not substituting in the Forwarded header's IP address (a different reasonable configuration).