Kubectl commands to EKS, from EC2 in a private networks, are timing out

Question

This EKS cluster has a private endpoint only. My end goal is to deploy Helm charts on the EKS. I connect to an EC2 machine via SSM and I have already installed Helm and Kubectl on that machine. The trouble is that in a private network, the AWS APIs can't be called. So, instead of calling aws eks update-kubeconfig --region region-code --name cluster-name I have created the kubeconfig such as below.

apiVersion: v1
clusters:
- cluster:
    server: 1111111111111111.gr7.eu-west-1.eks.amazonaws.com
    certificate-authority-data: JTiBDRVJU111111111
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: aws
  name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws
      args:
        - "eks"
        - "get-token"
        - "--cluster-name"
        - "this-is-my-cluster"
        # - "--role-arn"
        # - "role-arn"
      # env:
        # - name: AWS_PROFILE
        #   value: "aws-profile"

Getting the following error:

I0127 21:24:26.336266    3849 loader.go:372] Config loaded from file:  /tmp/.kube/config-eks-demo
I0127 21:24:26.337081    3849 round_trippers.go:435] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.21.2 (linux/amd64) kubernetes/d2965f0" 'http://1111111111111111.gr7.eu-west-1.eks.amazonaws.com/api?timeout=32s'
I0127 21:24:56.338147    3849 round_trippers.go:454] GET http://1111111111111111.gr7.eu-west-1.eks.amazonaws.com/api?timeout=32s  in 30001 milliseconds
I0127 21:24:56.338171    3849 round_trippers.go:460] Response Headers:
I0127 21:24:56.338238    3849 cached_discovery.go:121] skipped caching discovery info due to Get "http://1111111111111111.gr7.eu-west-1.eks.amazonaws.com/api?timeout=32s": dial tcp 10.1.1.193:80: i/o timeout

There is connectivity in the VPC, there are no issues with NACLs, security groups, port 80.

Answer 1

That looks like this open EKS issue: https://github.com/aws/containers-roadmap/issues/298

If that’s the case, upvote it so that the product team can prioritize it. If you have Enterprise support your TAM can help there as well.