Agent Observability (Datadog LLM Observability)

What it does

Agent Observability takes traces from AI applications, including LLM calls, retrieval steps, tool calls, and agent decisions, along with datasets, prompts, and evaluation results. It outputs execution graphs, latency and token-usage views, experiment comparisons across prompts, models, and configurations, evaluator scores, and production monitors that can be correlated with backend services and user sessions.

Teams use it to build versioned datasets from production traces, test prompt, model, and tool changes against that data before release, and then monitor the same system in production. A distinguishing point is that the same Datadog tracer runs from development through production, and AI traces can be linked to APM services, infrastructure signals, and RUM sessions in one platform.

It is aimed at AI engineers, data scientists, and SRE/DevOps teams, especially those already using Datadog. Access is through the web console, with SDK instrumentation for Python, Node.js, and Java, OpenTelemetry, or the HTTP API. Billing counts LLM spans; the Free plan includes up to 40K LLM spans per month with 15-day retention, and usage beyond the included 100K spans on Pro is billed on demand.

What it can do

• Build versioned datasets from production traces for repeatable testing
• Run experiments comparing prompts, models, and agent configurations on the same data
• Run built-in and custom evaluators, plus human annotation and review
• Trace every LLM call, retrieval step, tool call, and agent decision with latency and token usage
• Monitor production latency, cost, quality, and errors with monitors and alerts
• Correlate agent traces with APM services, infrastructure, and RUM sessions

Works with

• OpenAI
• Anthropic
• Gemini
• Vertex AI
• LangChain
• CrewAI
• Pydantic
• Bedrock
• LiteLLM
• Strands Agents

Example workflows

Regression check before a prompt change

Input: production traces from the past week. Steps: build a versioned dataset from those traces, create an experiment comparing the current and proposed prompts, run evaluators, and review failures. Output: side-by-side quality, cost, and latency results for both prompt versions.

Investigating a slow agent response

Input: an alert on latency for a user session. Steps: open the trace, inspect each LLM span, tool call, and retrieval step, then check correlated APM service and infrastructure data. Output: the specific step and dependency causing the delay.

Before you choose

• LLM spans are the billed unit; tool, workflow, agent, embedding, and retrieval spans are free per the pricing text.
• Retention is 15 days for traces and spans on Free and Pro, with add-ons for longer periods; experiment results are retained for 90 days.
• Agent Observability is listed as unavailable on the US-FED Datadog site.

Listed under

• Coding
• Coding agents
• DevOps and observability
• AI agents
• AI engineers
• Developers
• Evaluation
• LLM tracing
• Monitoring

Get started

Try Agent Observability in your browser, or start tracing locally with the free plan.

1. Create a Datadog account and install the Datadog LLM Observability MCP server or use the setup guide for your environment.
2. Install the SDK for your language (for Python, pip install ddtrace; for Node.js, npm install dd-trace; for Java, download the dd-java-agent jar).
3. Set the environment variables DD_SITE, DD_LLMOBS_ENABLED=1, DD_LLMOBS_ML_APP, and DD_API_KEY, then start your application with the tracer.

Billed on LLM spans only. Retention add-ons are billed per 10K LLM spans. M2M and annual commitments are discounted. There is no separate eval fee; eval LLM calls count as LLM spans. Pro billing basis (monthly vs annual) and per-span on-demand rate were not confirmed in the supplied text.

Source: datadoghq.com/products/ai/agent-observability, checked 29 Oct 2026. Prices can change; check before you buy.

Sources for this overview

Product pages, documentation, and pricing checked 29 Oct 2026. Each source supports part of the overview; availability and prices can change.

• Agent Observability | LLM Observability | Datadog ↗www.datadoghq.com
• Pricing | Datadog ↗www.datadoghq.com

Company website

datadoghq.com: product, pricing and policy pages

YC directory mirror

Batch and company name

Social accounts

Accounts the company links, and its posts on X

Human review

Checked by a person

knowledge_base

Last checked 29 Oct 2026. Every field keeps its source and date; unknown means we couldn’t confirm it.