Prompt
Production app observability with logs metrics and traces
Latest observation
Oct 20, 2026
OpenAI APIWeb search: off
For a production app, good observability means you can answer three questions quickly:
- What happened? → logs
- How bad is it / is it degrading? → metrics
- Why did it happen and where? → traces
1) Logs
Use logs for detailed event context.
Best practices
- Use structured logs (JSON), not plain text.
- Include:
timestamplevelserviceenvironmentrequest_id/trace_iduser_idor tenant ID if appropriate- event name and key fields
- Log important state changes, errors, retries, timeouts, and external calls.
- Avoid logging secrets, passwords, tokens, and PII.
Examples
- Request received
- Payment provider timeout
- Cache miss
- Job started / completed / failed
2) Metrics
Use metrics to monitor trends and alert on problems.
Core metrics
- Traffic: requests per second, job throughput
- Errors: error rate, failed jobs
- Latency: p50 / p95 / p99 response times
- Saturation: CPU, memory, disk, queue depth, thread pool usage
Best practices
- Use counters, gauges, and histograms
- Define SLIs/SLOs
- Example SLI: 99.9% of API requests under 300 ms and successful
- Alert on symptoms, not just causes
- Track golden signals:
- latency
- traffic
- errors
- saturation
3) Traces
Use distributed tracing to follow a request across services.
Best practices
- Propagate
trace_idandspan_idacross all services - Trace:
- inbound requests
- DB queries
- cache calls
- message queue operations
- HTTP calls to external services
- Add span attributes:
- route
- operation name
- dependency name
- status code
- retry count
Why it matters
- Helps identify where latency is introduced
- Makes microservice debugging much easier
- Correlates logs and metrics for a full picture
Recommended observability stack
A common production setup:
- Instrumentation: OpenTelemetry
- Metrics storage/dashboard: Prometheus + Grafana
- Logs: Loki, Elasticsearch/OpenSearch, or cloud logging
- Traces: Jaeger, Tempo, Honeycomb, or cloud tracing
- Alerting: Alertmanager, PagerDuty, Opsgenie, etc.
OpenTelemetry is a strong choice because it standardizes collection of logs, metrics, and traces across languages and tools.
Practical implementation checklist
- Add OpenTelemetry to all services
- Generate and propagate trace IDs
- Emit structured JSON logs
- Collect RED/USE metrics
- RED: Rate, Errors, Duration
- USE: Utilization, Saturation, Errors
- Build dashboards for:
- service health
- dependencies
- business KPIs
- Set alerts on:
- high error rate
- latency regressions
- queue buildup
- resource saturation
- Make logs, metrics, and traces linkable via trace ID
Simple production mindset
A healthy observability setup lets you:
- detect issues fast,
- narrow down the cause quickly,
- and prove whether a fix worked.
If you want, I can also give you:
- a reference architecture,
- a tooling comparison, or
- a sample OpenTelemetry setup for a specific stack like Node.js, Python, Java, or Go.