Sunday, January 24, 2021

Kubernetes CoreDNS External Resolving




The Problem 


I have a bare metal kubernetes deployment, and the external DNS resolving is not working.

What does it mean?

When I enter a specific pod using: kubectl exec -it... and then run ping to a deployed kubernetes service, everything works fine.

But, when I try to access an external DNS, such as google.com, I get an error that the:
name resolution had failed.


Checking the CoreDNS pods log using 
kubectl logs --namespace=kube-system -l k8s-app=kube-dns 
displays errors:

[ERROR] plugin/errors: 2 google.com. A: read udp 192.168.204.66:53816->192.168.1.1:53: i/o timeout


The whole issue seems to be related to a change in the Ubuntu DNS (probably the new systemd-resolved service) which prevents the kubernetes CoreDNS pod to forward external DNS resolving to the Ubuntu DNS service.



The Bypass


I have bypassed this by configuring the CoreDNS to use the external DNS directly, instead of the local Ubuntu DNS.


Use the following steps to do this:


Edit the CoreDNS configuration:
kubectl -n kube-system edit configmap coredns

Change the line:
 forward . /etc/resolve.conf {
to:
 forward . 8.8.8.8 {

Restart the CoreDNS pods:
kubectl --namespace=kube-system delete pod -l k8s-app=kube-dns 



Final Note


I hope that the kubernetes community would solve this issue in future version. It is very common to use bare metal kubernetes deployment on an Ubuntu machine, and it is a shame that we need to manually patch it to make it work.

No comments:

Post a Comment