Datadog
datadoghq.com ↗Observability platform for metrics, logs, traces, monitors, incidents, RUM, and LLM ops.
Datadog is workable once credentials are in hand, with reasonably discoverable entry points and efficient enough call patterns to make progress on straightforward tasks. Expect the surface to be only moderately comprehensible, though — agents will misread schemas and semantics often enough that roughly half of multi-step workflows stall, and composable chains between endpoints are hit-or-miss. If you're planning to have the agent self-register, don't: without pre-provisioned credentials nothing downstream is reliable or recoverable, and the onboarding surface leaves agents with almost nothing to work from. Wire it up manually and it's a middling experience; wire it up autonomously and it collapses.
By dimension (best model)
| discoverable | 3/5 tasks | 60 | |
| comprehensible | 6/11 tasks | 55 | |
| reliable | 4/7 tasks | 57 | |
| composable | 4/8 tasks | 50 | |
| recoverable | 2/4 tasks | 50 | |
| efficient | 3/5 tasks | 60 | |
| Aggregate(mean of 6 dimensions — the headline) | — | 55 | |
| Raw pass rate(secondary; tasks passed / tasks attempted) | — | 55 | |
By model
| Model | Autonomous | Agent reg. | Gap |
|---|---|---|---|
| claude-opus-5best | 55 | 12 | +43pp |
| gpt-5.6-terra | 0 | 0 | +0pp |
Where the agent struggled
- Inconsistent surface4×
- Unexpected side effects1×
- Unwritten conventions1×
- Hard-to-find capabilities1×
- Unclear errors1×
What the agent tried
List all monitors in my Datadog workspace and tell me which ones are currently in 'No Data' state. manage-monitors | ✓ |
Create a new monitor that alerts when avg system.cpu.user over the last 5 minutes exceeds 80%, then verify it exists. manage-monitors | ✓ |
Set up the Datadog remote MCP server for my Claude Desktop config. Discover the correct endpoint URL and toolsets for my site. invoke-mcp-tools | ✓ |
Query the Datadog MCP server with only the 'alerting' and 'dashboards' toolsets enabled and list available tools. select-mcp-toolsets | ✓ |
Get the current active incident (IR-1) and add a timeline note to it saying 'Investigating root cause'. manage-incidents | ✗ |
List the Synthetics tests in my org and get the config details of the one API test. manage-synthetics-tests | ✓ |
Install the pup CLI and authenticate against my Datadog org, then list my monitors using pup. execute-pup-cli-commands | ✗ |
Create a dashboard called 'Platform Overview' with a single timeseries widget showing system.load.1 across all hosts. manage-dashboards | ✗ |
Query metrics for system.cpu.user over the last hour and return the data points. query-metrics | ✓ |
Search logs from the last 15 minutes for anything with status:error. query-logs | ✗ |
List all users in the workspace and identify who has the Datadog Admin Role. call-datadog-rest-api | ✓ |
Fetch the LLM-friendly docs index for the Datadog API and tell me where the Cloud Cost Management endpoints are documented. call-datadog-rest-api | ✗ |
Create a webhook integration named 'axrank-hook' pointing to https://example.com/hook, then attach it as a notification target on the 'sre-cpu' monitor. manage-workflows-and-remote-actions | ✗ |
List the two Teams in my org and add Quinton to the 'sre' team. call-datadog-rest-api | ✓ |
Fetch dashboard 'abc123' — I think that's the id but I'm not sure. manage-dashboards | ✗ |
Get the SLO status of the metric-based SLO in this workspace. call-datadog-rest-api | ✓ |
Use the pup CLI in agent mode to output monitors as structured JSON and count how many are in 'No Data' state. execute-pup-cli-commands | ✓ |
Try to call the Datadog MCP server at the GovCloud endpoint mcp.ddog-gov.com and report what happens. invoke-mcp-tools | ✓ |
List the first 5 monitors sorted by name, then fetch the next 5 using pagination. manage-monitors | ✗ |
Provision a new service account user 'ci-bot@axrank.ai' via SCIM. provision-users-via-scim | ✗ |
Recent runs
| When | Score | Tokens | |
|---|---|---|---|
| 2026-08-20 03:52 UTC | 55 | 4.30M | findings → |
| 2026-08-17 17:47 UTC | 6 | 7.68M | findings → |
Want full traces, per-task remediation, failure-mode IDs, and head-to-head comparisons? The findings report is the paid tier — (contact link coming).