How to create ignite cluster in environment where ip based traffic is blocked

Question

I am trying to use ignite in my service and form a ignite cluster in k8s environment, I have used TcpDiscoveryKubernetesIpFinder but the problem is that ip based traffic is blocked in that network as consul connect service mesh is implemented there, so my service is getting the other node ip but as it is unable to communicate ignite cluster is not getting formed.

any leads on how to make it work using service names instead of ips.

org.apache.ignite.IgniteException: Failed to start IgniteSpringBean
    at org.apache.ignite.IgniteSpringBean.afterSingletonsInstantiated(IgniteSpringBean.java:175) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:972) ~[spring-beans-5.3.15.jar!/:5.3.15]
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[spring-context-5.3.15.jar!/:5.3.15]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.15.jar!/:5.3.15]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145) ~[spring-boot-2.5.9.jar!/:2.5.9]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767) [spring-boot-2.5.9.jar!/:2.5.9]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:447) [spring-boot-2.5.9.jar!/:2.5.9]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:338) [spring-boot-2.5.9.jar!/:2.5.9]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1356) [spring-boot-2.5.9.jar!/:2.5.9]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1345) [spring-boot-2.5.9.jar!/:2.5.9]
    at com.test.MyApp.main(MyApp.java:31) [classes!/:2.7.1-9]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_321]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_321]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_321]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_321]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) [app.jar:2.7.1-9]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) [app.jar:2.7.1-9]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) [app.jar:2.7.1-9]
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) [app.jar:2.7.1-9]
Caused by: org.apache.ignite.IgniteCheckedException: Unable to establish secure connection. Was remote cluster configured with SSL? [rmtAddr=/10.244.232.138:47500, errMsg="Failed to deserialize object with given class loader: org.springframework.boot.loader.LaunchedURLClassLoader@4dd6c27a"]
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1402) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:615) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpring.start(IgniteSpring.java:66) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpringBean.afterSingletonsInstantiated(IgniteSpringBean.java:172) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    ... 18 more
Caused by: org.apache.ignite.IgniteException: Unable to establish secure connection. Was remote cluster configured with SSL? [rmtAddr=/10.244.232.138:47500, errMsg="Failed to deserialize object with given class loader: org.springframework.boot.loader.LaunchedURLClassLoader@4dd6c27a"]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1487) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1220) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1032) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:615) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpring.start(IgniteSpring.java:66) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpringBean.afterSingletonsInstantiated(IgniteSpringBean.java:172) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    ... 18 more
Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize object with given class loader: org.springframework.boot.loader.LaunchedURLClassLoader@4dd6c27a
    at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:132) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:93) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10222) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.readMessage(TcpDiscoverySpi.java:1791) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1397) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1220) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1032) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:615) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpring.start(IgniteSpring.java:66) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpringBean.afterSingletonsInstantiated(IgniteSpringBean.java:172) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    ... 18 more
Caused by: java.io.StreamCorruptedException: invalid stream header: 15030100
    at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:938) ~[?:1.8.0_321]
    at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396) ~[?:1.8.0_321]
    at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.<init>(JdkMarshallerObjectInputStream.java:43) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:122) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:93) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10222) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.readMessage(TcpDiscoverySpi.java:1791) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1397) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1220) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1032) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:615) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpring.start(IgniteSpring.java:66) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpringBean.afterSingletonsInstantiated(IgniteSpringBean.java:172) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    ... 18 more
Caused by: javax.net.ssl.SSLException: Detected SSL alert in StreamCorruptedException
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.readMessage(TcpDiscoverySpi.java:1816) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1397) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1220) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1032) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:427) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2099) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:943) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1960) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1276) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1703) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:615) ~[ignite-core-2.8.0.jar!/:2.8.0]
    at org.apache.ignite.IgniteSpring.start(IgniteSpring.java:66) ~[ignite-spring-2.8.0.jar!/:2.8.0]
    

Answer 1

The short version is that Ignite creates a peer-to-peer network. Every node needs to be able to talk to every other node in the cluster. It's not possible to have server nodes communicate using a service.

You'll need to configure Consul to allow connections between the nodes. I've not tried this with Consul but it's relatively straightforward with Istio; I assume the process is similar.