Why is Kubernetes service discovery breaking between pods?

Kubernetes service discovery between pods usually “breaks” for one of a few common reasons. The most likely issue depends on whether pods are trying to reach each other by Service name, pod IP, or DNS name.

Common causes

1) The Service has no ready endpoints

If a Service exists but no pod matches its selector, or the pods aren’t Ready, Kubernetes won’t create usable endpoints.

Check:

kubectl get svc
kubectl get endpoints <service-name>
kubectl get pods -o wide

If endpoints is empty, the Service can’t route traffic.

---

2) Wrong labels / selector mismatch

A Service routes to pods via labels. If the Service selector doesn’t match the pod labels, discovery/routing fails.

Check:

kubectl describe svc <service-name>
kubectl get pods --show-labels

Make sure the Service selector matches the pods exactly.

---

3) DNS not resolving inside the cluster

Pods typically discover services using DNS like:

my-service.my-namespace.svc.cluster.local

If DNS is broken, service names won’t resolve.

Check from a pod:

nslookup my-service
nslookup my-service.my-namespace.svc.cluster.local

Also verify CoreDNS is running:

kubectl get pods -n kube-system | grep coredns
kubectl logs -n kube-system <coredns-pod>

---

4) Network policies are blocking traffic

Even if DNS works and a Service has endpoints, a NetworkPolicy may block pod-to-pod or pod-to-service traffic.

Check:

kubectl get networkpolicy -A

If policies exist, confirm they allow traffic between the source and destination namespaces/pods.

---

5) Service type / port mismatch

The Service may expose a different port than the container listens on, or the targetPort may be wrong.

Check:

kubectl describe svc <service-name>
kubectl describe pod <pod-name>

Verify:

• port on the Service
• targetPort maps to the container port
• the app is listening on that port

---

6) Pods are not actually reachable

The app may be listening only on 127.0.0.1 instead of 0.0.0.0, so other pods can’t connect even if Kubernetes routes correctly.

Inside the pod, ensure the app binds to all interfaces:

• good: 0.0.0.0:<port>
• bad: 127.0.0.1:<port>

---

7) Headless service behavior confusion

If you’re using a headless Service (clusterIP: None), Kubernetes returns pod IPs directly instead of load balancing. That can be fine, but your app must handle this behavior correctly.

---

8) Cross-namespace name usage

A service name alone only resolves within the same namespace. From another namespace, you need:

service-name.namespace.svc.cluster.local

---

9) Pod IP changed or stale client cache

If something connects directly to pod IPs instead of Services, it can break when pods restart because pod IPs are ephemeral. Use Services instead of direct pod IPs for stable discovery.

---

Quick debugging checklist

From a test pod:

kubectl exec -it <pod> -- sh

Then:

nslookup <service-name>
wget -qO- http://<service-name>:<port>

On the cluster side:

kubectl get svc,endpoints,pods
kubectl describe svc <service-name>
kubectl get networkpolicy -A

Most likely root causes

If I had to rank them:

1. Service selector mismatch
2. No ready endpoints
3. DNS/CoreDNS issue
4. NetworkPolicy blocking
5. Wrong ports / app binding

If you want, I can help you diagnose it step-by-step if you share:

• the Service YAML
• pod labels
• namespace names
• the exact hostname/URL being used
• the error message you see