Here’s a practical way to troubleshoot missing traces after an OpenTelemetry rollout, moving from the application outward so you can isolate where the data is disappearing.

1) Confirm spans are being created at all

Start at the source.

• Turn on SDK debug logging for your language/runtime.
• Verify your app is actually creating spans for the request path you expect.
• Check that the current request is not being filtered out by:
  • sampling
  • ignore rules
  • health-check exclusions
  • context not being propagated into the code path

Quick test: add a simple manual span around a known endpoint or job and see if it appears.

2) Check sampling first

A very common cause is that spans are generated but intentionally dropped.

Look for:

• traceidratio, parentbased_traceidratio, or custom samplers set too low
• head-based sampling in the SDK
• tail-based sampling in the collector dropping spans later
• parent-based sampling where the root span wasn’t sampled

Things to verify:

• What is the effective sampler in the app?
• Is the collector doing additional sampling?
• Did you expect “100%” but only configured it in one place?

3) Verify the exporter can reach the collector/backend

If spans exist but never leave the app, inspect the exporter path.

Check:

• OTLP endpoint is correct
• Protocol matches the receiver:
  • gRPC vs HTTP/protobuf
• TLS/credentials are correct
• Network policies, firewalls, proxies, sidecars, or service mesh rules
• DNS resolution for the collector endpoint
• Connection errors, timeouts, or retries in logs

A common mistake is pointing the app at the wrong port or using HTTP to a gRPC endpoint.

4) Confirm the collector is receiving data

If the app exports successfully, ensure the collector is actually getting it.

In the collector:

• enable debug logging temporarily
• check receiver metrics such as:
  • received spans
  • dropped spans
  • failed batches
• verify the pipeline includes the traces receiver → processors → exporter chain

Look for:

• misconfigured pipelines
• disabled receivers
• wrong endpoint bind address
• auth failures to downstream backend

5) Look for drops inside the collector

The collector can drop traces even when it receives them.

Common causes:

• memory limiter dropping data under load
• batch processor misconfiguration
• tail-sampling policies excluding spans
• exporter queue overflow
• resource exhaustion
• filtering processors excluding spans by service.name, attributes, or span names

If load increased after rollout, check collector CPU/memory first.

6) Verify trace context propagation

Sometimes spans exist, but traces appear broken or missing because context isn’t propagated.

Check:

• W3C trace context headers (traceparent, tracestate)
• baggage propagation if needed
• cross-service propagation across:
  • HTTP clients
  • gRPC
  • message queues
  • async tasks
  • background workers

Symptoms:

• root spans appear, but downstream child spans are in separate traces
• only one service shows up
• async work lacks parent linkage

7) Check instrumentation coverage

A rollout can be “successful” but still miss important code paths.

Verify whether:

• auto-instrumentation supports your framework/library version
• custom instrumentation is needed for internal libraries
• database/queue/cache clients are instrumented
• batch jobs, cron tasks, and consumers are included
• startup path vs request path is instrumented

If only some services were rolled out, you may simply be missing spans from uninstrumented components.

8) Validate resource attributes

Backends often group traces by service.name. If it’s missing or inconsistent, traces can seem “missing” even when present.

Check:

• service.name
• service.namespace
• service.version
• deployment.environment

If service.name changes between instances or deployments, spans can look fragmented or hard to find.

9) Compare what leaves the app vs what reaches the backend

A useful isolation strategy:

• App logs: are spans created and exported?
• Collector logs/metrics: are spans received?
• Backend ingest metrics/search: are spans indexed or queryable?

This tells you whether the loss happens:

1. in the app
2. in transit
3. in the collector
4. in the backend

10) Watch for environment-specific issues

Rollouts often fail in one environment only.

Examples:

• local works, prod doesn’t
• sidecar only in Kubernetes
• missing secrets in some pods
• different sampling in staging vs production
• different network routes in one cluster/region
• clock skew affecting trace visibility windows

11) Use a known-good minimal test

Create a tiny canary service or endpoint that:

• starts a span
• exports to the same collector/backend
• uses the same environment/config

If the canary works, the issue is likely app-specific. If it doesn’t, the issue is infrastructure/configuration.

12) Check backend-specific constraints

Some backends can ingest traces but hide them due to:

• indexing delay
• query time range
• retention settings
• tenant/org mismatch
• ingestion limits
• attribute cardinality issues
• permission/role issues

Make sure you’re searching the right service name, environment, and time window.

---

A fast triage checklist

If you need a quick sequence:

1. Confirm spans are created in app logs/debug.
2. Disable or raise sampling temporarily.
3. Check exporter connectivity to collector.
4. Check collector metrics/logs for received/dropped spans.
5. Verify propagation across service boundaries.
6. Confirm service.name and environment attributes.
7. Test with a minimal canary.

---

Common root causes after a rollout

Most frequent causes I see are:

• sampling set too aggressively
• wrong OTLP endpoint/protocol
• collector dropping spans due to resource limits
• missing trace context propagation between services
• incomplete instrumentation coverage
• inconsistent service.name

If you want, I can also give you:

• a language-specific troubleshooting checklist (Java, .NET, Python, Node, Go, etc.), or
• a step-by-step OpenTelemetry debugging playbook with exact logs/metrics to inspect.