'Calico Felix-Typha connection cancelled on ARM64 EKS
I'm attempting to get Calico installed on an Graviton EKS cluster using the manifests listed here: https://docs.aws.amazon.com/eks/latest/userguide/calico.html
In order to successfully run on ARM64, I'm using a tigera ImageSet with the sha256 manifest of the master-arm64 tags for calico and tigera containers (when they exist). ref: https://projectcalico.docs.tigera.io/maintenance/image-options/imageset
apiVersion: operator.tigera.io/v1
kind: ImageSet
metadata:
name: calico-master
spec:
images:
- image: "calico/apiserver"
digest: "sha256:1a2bc0bad25eb95e77353d59e6ad9edc9d56aa9caebdcfbd027e8ddb7eb956b1"
- image: "calico/cni"
digest: "sha256:a257ee22e3d9e74d2b4c6362045147002104cea6101d3aaefa74661b91fea89b"
- image: "calico/kube-controllers"
digest: "sha256:fd101df470937e14033f602e5817e31e46933c6088a8bdc6fc80e43a1c9e011b"
- image: "calico/node"
digest: "sha256:8694683b9bd0d13caef2e67f1486ded0e843c810f1eb9d4c021a5ffdedd4af8d"
- image: "calico/typha"
digest: "sha256:174b0c47db4297623500cc044826bc259af28974cf5e0df4f84244e824cfda52"
- image: "calico/pod2daemon-flexvol"
digest: "sha256:a276db19af1cba49b7a032ee259e0e0f198575d8af27c9cadfebfe4d63bf15bf"
- image: "calico/windows-upgrade"
digest: "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
- image: "tigera/operator"
digest: "sha256:01327468115202b72519fbe99344b2cc64cca37302d12840827190d97c2ba9cb"
- image: "tigera/key-cert-provisioner"
digest: "sha256:b8b4f0ae606626e029c77dc1c30199f8484f797d2bc52d8e484efc5b938725ad"
Tigera and Calico seem to launch fine, but my calico-node daemonset remains 0/1 forever due to the Felix-Typha connection:
kubectl -n calico-system get all
The logs for Typha (calico-typha deployment):
2022-03-03 15:27:34.692 [INFO][7] sync_server.go 368: Accepted from 192.168.40.212:49482 port=5473
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 393: New connection connID=0x3e1 port=5473
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 558: Per-connection goroutine started client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 636: Failed to read from client client=192.168.40.212:49482 connID=0x3e1 error=gob: name not registered for interface: "github.com/projectcalico/calico/typha/pkg/syncproto.MsgClientHello" thread="read"
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 629: Read goroutine finished client=192.168.40.212:49482 connID=0x3e1 thread="read"
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 666: Asked to stop by context. client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [WARNING][7] sync_server.go 675: Failed to read client hello. client=192.168.40.212:49482 connID=0x3e1 error=context canceled
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 545: Client connection shutting down. client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 554: Client connection shut down. client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 421: Connection handler finished error=context canceled
The logs for Felix (calico-node daemonset):
2022-03-03 15:33:40.427 [INFO][13853] status-reporter/startup.go 425: Early log level set to info
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHAK8SSERVICENAME=calico-typha
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHAK8SNAMESPACE=calico-system
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHAKEYFILE=/felix-certs/key.key
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHACERTFILE=/felix-certs/cert.crt
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHACAFILE=/typha-ca/caBundle
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHACN=typha-server
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/discovery.go 163: Found ready Typha addresses. addrs=[]string{"192.168.193.138:5473", "192.168.36.92:5473"}
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/discovery.go 166: Chose Typha to connect to. choice="192.168.36.92:5473"
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/startsyncerclient.go 56: Connecting to Typha. addr="192.168.36.92:5473"
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/sync_client.go 71: requiringTLS=true
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/sync_client.go 200: Starting Typha client
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/sync_client.go 71: requiringTLS=true
2022-03-03 15:33:40.448 [INFO][13853] status-reporter/tlsutils.go 39: Make certificate verifier requiredCN="typha-server" requiredURISAN="" roots=&x509.CertPool{byName:map[string][]int{"0,1*0(\x06\x03U\x04\x03\f!tigera-operator-signer@1646270114":[]int{0}}, lazyCerts:[]x509.lazyCert{x509.lazyCert{rawSubject:[]uint8{0x30, 0x2c, 0x31, 0x2a, 0x30, 0x28, 0x6, 0x3, 0x55, 0x4, 0x3, 0xc, 0x21, 0x74, 0x69, 0x67, 0x65, 0x72, 0x61, 0x2d, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x2d, 0x73, 0x69, 0x67, 0x6e, 0x65, 0x72, 0x40, 0x31, 0x36, 0x34, 0x36, 0x32, 0x37, 0x30, 0x31, 0x31, 0x34}, getCert:(func() (*x509.Certificate, error))(0x6d99f0)}}, haveSum:map[x509.sum224]bool{x509.sum224{0xc0, 0x54, 0x82, 0x63, 0xb1, 0xf5, 0xe0, 0xda, 0x83, 0x69, 0x3f, 0x40, 0x66, 0xf7, 0x5a, 0x72, 0x3a, 0x4e, 0x4a, 0xe6, 0x1a, 0xfe, 0xb0, 0xa5, 0x5d, 0xd1, 0x2e, 0xdf}:true}}
2022-03-03 15:33:40.448 [INFO][13853] status-reporter/sync_client.go 252: Connecting to Typha. address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.455 [INFO][13853] status-reporter/tlsutils.go 46: Verify certificate chain signing address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.461 [INFO][13853] status-reporter/sync_client.go 267: Connected to Typha. address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.461 [INFO][13853] status-reporter/sync_client.go 301: Started Typha client main loop address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.462 [ERROR][13853] status-reporter/sync_client.go 293: Failed to read from server address="192.168.36.92:5473" connID=0x0 error=EOF type="node-status"
2022-03-03 15:33:40.462 [INFO][13853] status-reporter/sync_client.go 166: Typha client Context asked us to exit connID=0x0 type="node-status"
2022-03-03 15:33:40.462 [FATAL][13853] status-reporter/startsyncerclient.go 77: Connection to Typha failed
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|