WorkOS
workos.com ↗User auth, SSO, SCIM directory sync, and fine-grained authorization for B2B apps.
Auth & Identity
AXRank (autonomous)
71
Agent registration
6
gap +65pp
Δ last run
+18
Evals run
3
Trend
WorkOS behaves well once an agent is holding valid credentials: call patterns are efficient, primitives compose cleanly across workflows, and the surface is largely comprehensible with reasonable recovery when things go wrong. The catch is getting there. Discoverability is thin in both modes, and self-registration is effectively a wall — agents starting from zero credentials cannot reliably bootstrap themselves into a working state, which cascades into every other dimension failing. Expect to pre-provision API keys out of band; once you do, the integration is straightforward, but don't count on an agent onboarding itself.
By dimension (best model)
| discoverable | 1/4 tasks | 25 | |
| comprehensible | 6/8 tasks | 75 | |
| reliable | 8/11 tasks | 73 | |
| composable | 4/5 tasks | 80 | |
| recoverable | 3/4 tasks | 75 | |
| efficient | 5/5 tasks | 100 | |
| Aggregate(mean of 6 dimensions — the headline) | — | 71 | |
| Raw pass rate(secondary; tasks passed / tasks attempted) | — | 70 | |
By model
| Model | Autonomous | Agent reg. | Gap |
|---|---|---|---|
| Opus 4.7best | 71 | 6 | +65pp |
| GPT-5.5 | 62 | 6 | +56pp |
Where the agent struggled
- Unwritten conventions2×
- Ambiguous canonical route2×
- Unclear errors2×
- Unexpected side effects2×
- Confusing parameters1×
What the agent tried
List all organizations in this WorkOS workspace and return the organization ID and name of the only organization present. list-organizations | ✓ |
List all AuthKit users in the workspace and report which of alice/bob/carol/dave/eve have verified email addresses. list-users | ✓ |
Authenticate against the WorkOS API using only the secret API key. Confirm the auth header format by making one successful authenticated request. authenticate-user | ✓ |
Create a new organization named 'Acme Staging' with the domain 'acme-staging.example' attached. Then verify it exists by fetching it back by ID. create-organization | ✓ |
Invite a new user 'frank@example.com' to 'Test Organization' with the 'Member' role, then verify the invitation appears in the pending invitations list. invite-user-to-organization | ✓ |
List the directories configured in this workspace via the Directory Sync API. list-directories | ✓ |
Stream the most recent 5 events from the workspace and report what event types are present. stream-events | ✗ |
Store a new secret in Vault named 'datadog_api_key' with value 'dd_test_123', then list all vault secrets to confirm both it and the existing 'stripe_api_key' are present. store-secret-in-vault | ✗ |
Find the WorkOS AuthKit MCP integration guide starting from the workos.com homepage, and report the URL of the AuthKit OAuth authorization server discovery document format described in that guide. find-mcp-auth-docs | ✗ |
Fetch the OAuth authorization server metadata for this workspace's AuthKit domain (prestigious-legend-21-staging.authkit.app) and report the `authorization_endpoint` and `token_endpoint` values. discover-authkit-oauth-metadata | ✓ |
Attempt to perform an FGA check (any subject/relation/object). Report what happens and what the workspace's FGA status is. evaluate-fga-check | ✓ |
List the roles available in this workspace's environment. list-roles | ✓ |
Get the user record for 'alice@example.com' and then list all sessions for that user. manage-user-sessions | ✓ |
Generate a magic auth challenge for 'bob@example.com'. create-magic-auth-challenge | ✗ |
List all domains attached to 'Test Organization' and report which are verified vs pending. verify-domain | ✓ |
Generate an Admin Portal link for 'Test Organization' scoped to the 'sso' intent so the customer can configure SSO themselves. launch-admin-portal | ✓ |
Attempt to list audit log events at /audit_logs/events. If that returns 404, find the correct endpoint and complete the task. emit-audit-log-event | ✗ |
List webhooks configured in this workspace via the API. subscribe-to-webhooks | ✓ |
List the invitations for 'Test Organization' and report the count in each state (pending, accepted, revoked, expired). list-invitations | ✗ |
Make a deliberately malformed call to create a user (omit the required `email` field) and report what error WorkOS returns. Then fix the call and successfully create user 'grace@example.com'. recover-from-validation-error | ✓ |
Recent runs
| When | Score | Tokens | |
|---|---|---|---|
| 2026-05-28 13:34 UTC | 71 | 8.57M | findings → |
| 2026-05-23 22:47 UTC | 53 | 2.79M | findings → |
| 2026-05-23 21:24 UTC | 80 | 545.6k | findings → |
Want full traces, per-task remediation, failure-mode IDs, and head-to-head comparisons? The findings report is the paid tier — (contact link coming).