Istio Multicluster primary-remote on Azure AKS

Question

I am trying to create multi cluster istio primary-remote.

First created two clusters AZURE AKS. Used AzureCNI for Network Configuaration and following are the settings of the cluster.

First cluster

vnet istioclusterone - 10.10.0.0/20
subnet default - 10.10.0.0/20
k8s service address range 10.100.0.0/16
DNS service ip - 10.100.0.10
Docker Bridge address - 172.17.0.1/16
DNS-prefix - app-cluster-dns

Second cluster

vnet istioclusterone - 10.11.0.0/20
subnet default - 10.11.0.0/20
k8s service address range 10.101.0.0/16
DNS service ip - 10.101.0.10
Docker Bridge address - 172.18.0.1/16
DNS-prefix - processing-cluster-dns

Other than this gone with default settings.

Next Followed below articles to setup multi Istio cluster.

Before you begin

Primary-remote

last step in second article to setup cluster2 as remote is failed.

Found below errors in logs of istio-ingressgateway pod.

2022-04-11T07:51:00.352057Z     warning envoy config    StreamAggregatedResources gRPC config stream closed since 431s ago: 14, connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying to verify candidate authority certificate \"cluster.local\")"
2022-04-11T07:51:08.514428Z     warning envoy config    StreamAggregatedResources gRPC config stream closed since 439s ago: 14, connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying to verify candidate authority certificate \"cluster.local\")"
2022-04-11T07:51:12.462140Z     warning envoy config    StreamAggregatedResources gRPC config stream closed since 443s ago: 14, connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying to verify candidate authority certificate \"cluster.local\")"
2022-04-11T07:51:39.950935Z     warning envoy config    StreamAggregatedResources gRPC config stream closed since 471s ago: 14, connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying to verify candidate authority certificate \"cluster.local\")" 

Has anyone tried this scenario please share your insights.

Thanks.

Update:

Have used custom certs for both the clusters previous error was solved.

then created a gateway in both the clusters.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: cluster-aware-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 443
      name: tls
      protocol: TLS
    tls:
      mode: AUTO_PASSTHROUGH
    hosts:
    - "*.local"

Now getting new error. check below logs of pod istio-ingressgateway-575ccb4d79 of cluster2.

2022-04-13T09:14:04.650502Z     warning envoy config    StreamAggregatedResources gRPC config stream closed since 60s ago: 14, connection error: desc = "transport: Error while dialing dial tcp <publicIPofEastWestgateway>:15012: i/o timeout"
2022-04-13T09:14:27.026016Z     warning envoy config    StreamAggregatedResources gRPC config stream closed since 83s ago: 14, connection error: desc = "transport: Error while dialing dial tcp <publicIPofEastWestgateway:15012: i/o timeout"

what I undertood here, I have an eastwestgateway installed in cluster1 as in the documentation linkToDoc

cluster2 is trying to access cluster1. using publicIp of eastwest-gateway on port 15012 which is failing.

checked security groups port is opened. Tried telnet from a test pod from within the cluster to check. its failing.

can anyone help me here.

Thanks

Answer 1

It looks like a firewall issue. not sure if it'll help, but try opening the ports 15012 and 15443 on the remote cluster's outbound, to the eastwestgateway elb ip (primary cluster)