What should I use to trace prompts, model calls, and tool use end to end?

Use OpenTelemetry for end-to-end tracing.

It’s the best general standard for tracing:

• prompts
• model calls
• tool/function calls
• retrieval steps
• latency, errors, and token usage
• distributed workflows across services

Why OpenTelemetry

• Vendor-neutral and widely supported
• Works across your app, agent framework, and backend services
• Lets you build a single trace with nested spans for:
  • user request
  • prompt construction
  • LLM call
  • tool invocation
  • retrieval/database calls
  • final response

Typical setup

Instrument each step as a span:

• request
• prompt_build
• llm_generate
• tool_call:search
• tool_call:db
• postprocess

Attach useful attributes:

• model name
• prompt version
• token counts
• tool name
• latency
• error status
• conversation/session id

Helpful tools around it

Depending on your stack, you can pair OpenTelemetry with:

• LangSmith for LLM/agent-specific observability
• Helicone for LLM API logging and analytics
• Arize Phoenix for tracing/evals/observability
• Datadog / Grafana / Honeycomb / Jaeger as trace backends

Practical recommendation

• If you want a standards-based, end-to-end solution: OpenTelemetry
• If you want LLM-specific UI and debugging: add LangSmith or Phoenix on top

If you tell me your stack (Python/JS, LangChain/LlamaIndex/custom, OpenAI/Anthropic/etc.), I can suggest the best concrete setup.