unable to give static ip to nlb

Question

I have hard time getting this working with NLB using ingress controller : https://kubernetes.github.io/ingress-nginx/deploy/#network-load-balancer-nlb

Even subnets are not taking effect here , its not passing my configurations in the API that creates the NLB:

================================
kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-eip-allocations: "eipalloc- 
    07e3afcd4b7b5d644,eipalloc-0d9cb0154be5ab55d,eipalloc-0e4e5ec3df81aa3ea"
    service.beta.kubernetes.io/aws-load-balancer-subnets: "subnet- 
    061f4a497621a7179,subnet-001c2e5df9cc93960"
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: http
  - name: https
    port: 443
    targetPort: https

Answer 1

So, as it turned out - these annotations will be supported only since Kubernetes 1.16, which is "coming soon" on AWS. Currently supported version is 1.15, which just ignores those annotations...

Considering that you are using AWS-specific annotations here (service.beta.kubernetes.io/aws-load-balancer-eip-allocations) - I assume that this is exactly the reason why it does not work on your case.

As a workaround, I would advice:

1. Create custom post-deployment script that re-configures newly-created LoadBalancer, after each Kubernetes Service Update.
2. Switch to use something more conventional, like ELB with your Container, and AutoScaling groups (that's what we did.)
3. Setup your own Kubernetes Controller (super-hard thingie, which will become completely obsolete and will just be basically a lost of time, as soon as 1.16 is officially out). See this how-to
4. Wait...

Official statement: https://docs.aws.amazon.com/eks/latest/userguide/update-cluster.html#1-16-prequisites

Full list of annotations (when they will be "supported" ofc): https://github.com/kubernetes/kubernetes/blob/v1.16.0/staging/src/k8s.io/legacy-cloud-providers/aws/aws.go#L208-L211

Stay tuned! :(

Answer 2

The number of eip allocations must match the number of subnets in the subnet annotation.

service.beta.kubernetes.io/aws-load-balancer-eip-allocations: eipalloc-xyz, eipalloc-zzz

service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-xxxx, mySubnet

You have 3 allocations but only 2 subnets.

In addition, the annotation

service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing" is missing.

By default this will use scheme "internal".

I assume since you are allocating elastic IP addresses that you might want "internet-facing".

Also, you are using annotations that are meant for "AWS Load Balancer Controller" but you are using an "AWS cloud provider load balancer controller"

The external value for aws-load-balancer-type is what causes the AWS Load Balancer Controller, rather than the AWS cloud provider load balancer controller, to create the Network Load Balancer. docs

You are using service.beta.kubernetes.io/aws-load-balancer-type: nlb which means that none of the links provided earlier in this answer pertain to your Load Balancer. nlb type is an "AWS cloud provider load balancer controller" not an "AWS Load Balancer Controller"

For "AWS cloud provider load balancer controller" all the docs reference is this.