Analytics

The Analytics surface for each application has two layers:

1. KPI cards on the Run History page — a quick read of the application's health.
2. Metric Details page — a separate deep-dive page opened by clicking a KPI card, with tabs for each metric and per-metric charts.

KPI cards on the Run History page

The set of KPI cards depends on the application type.

Tools (both task mode and Conversation mode) — 7 cards

Card	Primary metric	Secondary metric
Runs	Total runs in range	Runs / day (or week, based on range)
Success rate	Success rate across completed runs	Failed run count (top error type, optional)
Latency (P95)	P95 end-to-end run latency	Avg latency
Cost / run (avg)	Avg cost per run	Total cost in range
Tokens / run (avg)	Avg (input + output)	Input vs output split
Steps / run (avg)	Avg step count	Max steps in a run
Context utilization (P95 max)	P95 of per-run max utilization	Avg max utilization

Agents — 7 cards

Card	Primary metric	Secondary metric
Runs	Total runs in range	Runs / day
Completion rate	Completed / (terminal runs)	Counts by terminal state
Latency (P95)	P95 end-to-end run latency	Avg latency
Cost / run (avg)	Avg cost per run	Total cost
Tokens / run (avg total)	Avg total tokens / run	Prompt vs completion split
Tool calls / run (avg)	Avg tool calls / run	Max tool calls
Tool-call latency (P95)	P95 of tool call duration	Avg tool-call latency

All KPI cards are computed from run / span aggregation only — there is no model-based evaluation behind them. (For quality scoring, use Evaluation.)

Metric Details page

Clicking any KPI card opens a separate Metric Details page.

Tabs

The page has one tab per KPI card — so for Tools you get 7 tabs (Runs, Reliability, Latency, Cost, Tokens, Steps, Context Utilization), and for Agents you get 7 tabs (Runs, Completion / Reliability, Latency, Cost, Tokens, Tool calls, Tool-call latency).

The default opened tab matches the card you clicked (deep-link behavior).

Inherited filters

The Metric Details page inherits the Run History page's filters:

• Date range
• Version
• Users

Default time bucketing

For all time-series charts:

• Date range ≤ 48 hours → hourly buckets.
• Otherwise → daily buckets.
• An optional Granularity toggle (Hour / Day / Week) may be available per tab.

What's on each tab

Per-tab charts and sample tables. The combinations below apply to Tools; the Agents variants mirror these with agent-relevant dimensions.

Runs tab

• Run volume trend (line, time-bucketed).
• Run source trend (stacked area/column — UI / API / Embed).
• Run distribution by version (horizontal bar; shown only when more than one version is in scope).

Reliability / Completion tab

• Failure (or completion) rate trend over time.
• Failure reasons (bar chart by error type, top 10 + Other).
• Where failures happen (bar chart by step / span name).

Latency tab

• Latency percentiles over time — P50 / P95 / P99 (multi-line).
• Latency distribution (histogram with optional P50/P95/P99 overlay markers).
• Latency breakdown by step (top 10 + Other).
• Latency vs tokens (scatter; sampled if N is large).
• Slowest runs table (top 20 by latency; columns: Run start time, Run ID, Version, User, Status, Latency, Tokens, Steps, Link to trace).

Cost tab

• Total cost over time (line).
• Cost per run distribution (histogram with P50 / P95 markers).
• Cost breakdown by step type (LLM vs Retrieval vs External API vs Other).
• Most expensive runs table (top 20 by cost).

Tokens tab

• Prompt vs completion tokens over time (stacked area).
• Tokens per run distribution (histogram).
• Tokens breakdown by step / model (top 10 + Other).
• Token cost breakdown — available when model costs are known.
• Highest token runs table (top 20).

Steps tab

• Steps per run distribution (histogram).
• Steps per run over time (P50 / P95 line).
• Top steps (top 10 by P95 duration, total time, or frequency — toggle).
• Runs with highest step count table (top 20).

Context Utilization tab

• Utilization percentiles over time (P50 / P95 / P99).
• Utilization distribution (histogram; allows >1.0 if overflow occurs).
• Near-limit runs over time (count of runs where max utilization ≥ threshold). Default threshold: 0.80, adjustable per tab.

Per-run max utilization is computed as max_over_llm_calls( input_tokens / context_limit ).

Tool calls tab (Agents)

• Tool calls over time (avg + P95 lines).
• Tool calls per run distribution (histogram).
• Top tools by call volume (horizontal bar).
• Tool calls vs latency (scatter).
• Runs with unusually high tool-call counts table (top 20).

Tool-call latency tab (Agents)

• Tool-call latency percentiles over time (P50 / P95 / P99).
• Tool-call latency distribution (histogram).
• Top tools by total latency (horizontal bar; toggle between avg and P95).

Local controls

Many tabs include local controls — e.g., a percentile selector for breakdowns (P50 / P95 / P99), a metric toggle (Sum vs Avg per run), or a "Step name vs Model name" dimension toggle. These are local to the tab and don't change anything elsewhere on the page.