A new application version has been deployed, the Kubernetes pods are running, and the deployment appears successful. Yet, when you test the application, the canary deployment receives no traffic.
This is a common troubleshooting scenario in Kubernetes environments where application deployment, service discovery, and traffic routing must work together.
A Kubernetes canary deployment allows teams to release a new version to a limited group of users before making it available to everyone. When traffic does not reach the canary, teams cannot properly validate the new version under real-world conditions.
In this guide, we explain how canary deployments work, why they may receive no traffic, and how to troubleshoot the problem using practical examples.
What Is a Kubernetes Canary Deployment?
A canary deployment is a release strategy in which a new application version runs alongside the existing stable version. Instead of replacing the entire application at once, teams gradually expose the new version to a controlled portion of traffic.
For example, suppose an application has two versions:
- Stable version: The current production release serving most users.
- Canary version: The new release being tested with a smaller portion of traffic.
A team might initially direct 5% of eligible traffic to the canary and keep 95% on the stable version. If monitoring confirms that the new version performs correctly, the team can gradually increase the canary's traffic share.
This approach supports safe release management, reduces the impact of defective releases, and makes it easier to roll back when problems occur.
However, Kubernetes does not automatically split traffic by percentage simply because two application versions are running. Traffic distribution requires an appropriate routing mechanism, such as an ingress controller, service mesh, or progressive delivery tool.
Why Is Your Canary Deployment Not Receiving Traffic?
Several configuration issues can prevent requests from reaching a canary version.
1. Incorrect Kubernetes Service Selectors
A Kubernetes Service uses label selectors to identify the pods that should receive traffic.
If the canary pods have labels that do not match the Service selector, those pods will not be selected as endpoints.
For example, a canary pod might use this label:
version: canary
But the Service may select only pods labeled:
version: stable
In this situation, the canary pods will not receive traffic through that Service.
How to fix it: Check the labels on your pods and compare them with the Service selector.
Run:
kubectl get pods --show-labels
Then inspect the Service configuration:
kubectl describe service <service-name>
Ensure that the intended pods match the selector. Remember that changing a shared Service selector to include the canary does not create percentage-based traffic splitting; it may simply add the canary pods to the same pool of endpoints.
2. Missing or Incorrect Traffic-Splitting Rules
A canary deployment may have healthy pods and a valid Service, but the routing layer may still send all traffic to the stable version.
This can happen when ingress rules, gateway configuration, or service mesh policies do not include the canary destination.
How to fix it: Review the traffic-routing configuration and confirm that the canary receives the intended percentage of requests.
If you use Istio, inspect the relevant VirtualService and DestinationRule resources. If you use another ingress controller or progressive delivery platform, verify its routing configuration and supported traffic-splitting behavior.
3. Canary Pods Are Not Ready
A pod can be running without being ready to serve requests. Kubernetes readiness probes help determine whether a container is ready, and unready pods are generally excluded from normal Service traffic.
If the canary application has startup problems, failed health checks, or unavailable dependencies, Kubernetes may not consider it ready.
How to fix it: Check pod status and readiness conditions.
kubectl get pods
For more information, inspect the pod:
kubectl describe pod <canary-pod-name>
Review readiness probe failures, application logs, and dependency errors. Correct the underlying issue before directing production traffic to the canary.
4. Incorrect Ingress or Gateway Configuration
Ingress controllers and gateways can route requests according to hostnames, paths, headers, or configured destinations. A rule that matches only the stable service can prevent requests from reaching the canary.
How to fix it: Verify the requested hostname and path, inspect the active routing rules, and confirm that the canary destination is configured correctly.
Also check whether an external load balancer, gateway, or another routing layer is overriding the expected behavior.
5. Traffic Is Being Tested Incorrectly
Sometimes the routing configuration is correct, but the test does not exercise the canary route.
For example, a canary may receive traffic only when requests contain a particular header or use a specific hostname. Testing the default URL may send every request to the stable version.
How to fix it: Test using the same hostname, path, headers, and routing conditions that are expected to activate the canary. Review access logs and metrics to verify which version handled each request.
How to Troubleshoot a Canary Deployment Step by Step
Use this checklist to identify where traffic is being lost.
Step 1: Confirm the canary pods are running and ready.
kubectl get pods -o wide
Step 2: Inspect pod labels and Service selectors.
kubectl get pods --show-labels kubectl describe service <service-name>
Step 3: Check the Service endpoints.
kubectl get endpointslice
Confirm that the expected canary pod addresses appear in the relevant EndpointSlices when they are ready and selected by the Service.
Step 4: Inspect routing rules.
Check your ingress, gateway, service mesh, or progressive delivery configuration. Confirm that the canary destination is included and that the intended traffic policy is active.
Step 5: Review application and proxy logs.
kubectl logs <canary-pod-name>
If the pod contains multiple containers, specify the appropriate container. For Istio deployments, inspect the Envoy proxy logs when investigating service mesh routing issues.
Step 6: Validate traffic distribution.
Send controlled test requests and examine access logs, request metrics, or version-specific counters. Verify that the canary receives requests before increasing its traffic share.
Best Practices for Safe Canary Releases
Reliable canary deployments require more than correct routing.
- Use clear labels: Distinguish stable and canary versions without creating selector conflicts.
- Configure explicit routing: Use an appropriate ingress, service mesh, or progressive delivery tool for controlled traffic splitting.
- Monitor key metrics: Track error rates, latency, availability, and resource utilization for each version.
- Automate validation: Use deployment checks and health metrics to support promotion or rollback decisions.
- Start with limited exposure: Increase traffic gradually after the canary meets defined success criteria.
- Maintain a rollback plan: Ensure that traffic can return to the stable version if the canary behaves unexpectedly.
These practices help teams reduce deployment risk while maintaining control over application releases.
How DevopsX Supports Safer Release Management
At DevopsX, we focus on modern DevOps practices, Kubernetes operations, cloud infrastructure, and deployment automation to help businesses build, deploy, and scale applications reliably.
Understanding Kubernetes traffic routing, canary deployments, CI/CD automation, and release management helps engineering teams identify configuration issues before they affect production users.
A successful canary release is not simply about deploying a new version. It is about ensuring that traffic reaches the correct application, performance is measured accurately, and rollback remains possible when something goes wrong.
With the right routing configuration, monitoring, and automation, teams can make application releases more predictable and reduce unnecessary production risks.
Frequently Asked Questions
Why is my Kubernetes canary deployment not receiving traffic?
Common causes include incorrect Service selectors, missing traffic-splitting rules, unready pods, misconfigured ingress or gateway policies, and tests that do not match the canary routing conditions.
Does Kubernetes automatically split traffic between stable and canary versions?
No. A standard Kubernetes Service distributes traffic across its eligible endpoints but does not provide percentage-based canary routing by itself. Controlled traffic splitting requires additional routing capabilities or tools.
How can I check whether my canary pods are receiving traffic?
Inspect pod readiness, Service selectors, EndpointSlices, routing configuration, application logs, and version-specific request metrics. Together, these help determine whether traffic reaches the canary.
How does a canary deployment improve release safety?
It limits initial exposure to a new version, allowing teams to evaluate its behavior before expanding the rollout. Monitoring and rollback procedures help reduce the impact of release problems.