'plugin/errors: 2 oap. A: dial udp 100.100.2.136:53: i/o timeout about kubernetes coredns error

Today the coredns one pod running into issue, I check the coredns pod and show log like this:

.:53
2022-05-23T08:41:36.664Z [INFO] CoreDNS-1.3.1
2022-05-23T08:41:36.665Z [INFO] linux/amd64, go1.11.4, 6b56a9c
CoreDNS-1.3.1
linux/amd64, go1.11.4, 6b56a9c
2022-05-23T08:41:36.665Z [INFO] plugin/reload: Running configuration MD5 = 8646128cd34ade07719a0787cce6943e
2022-05-23T09:40:08.490Z [ERROR] plugin/errors: 2 oap. A: dial udp 100.100.2.136:53: i/o timeout

currently the coredns have 2 pod, one of the pod have this issue. The dns pod ip is 172.30.112.19, why the dns tried to connect 100.100.2.136? why did this happen? what should I do to make it work?

kubernetescoredns


Solution 1:[1]

why the dns tried to connect 100.100.2.136?

When coreDNS gets request that is outside the cluster domain (eg. www.google.com), it will forward the request to upstream nameserver (likely the 100.100.2.136:53 in your case). You can check in coreDNS ConfigMap for forward . /etc/resolv.conf

why did this happen? what should I do to make it work?

If the other node that run the coreDNS pod is function correctly, there could be discrepancy in the nameserver and/or the node's network settings. It is possible also the CNI agent on the node has malfunction and messed up with IP table. You can try cordon this node and move the pod to another node.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 gohm'c

Related Questions