# Atlas Quality Eval Results

Durable results log for Atlas and specialist profile quality evaluation runs.

Use this note for human-readable summaries. Machine-readable JSONL artifacts live under `/home/will/lab/swarm/swarm-common/agent-evals/atlas_quality/results/`.

Report format:
- Overall: `PASS`, `WARN`, or `FAIL`
- Coverage: dimensions run and skipped
- Regressions: scenario id, previous score, current score, likely cause
- Actions: created Kanban task ids or explicit `none`
- Artifacts: paths to JSONL result and transcript bundle

## 2026-05-14T23:49:43+00:00 — WARN
- Artifact: `agent-evals/atlas_quality/results/2026-05-14-manual-smoke-explicit.jsonl`
- Mode: `dry_run`
- Coverage: ops_safety, review_quality, routing_delegation
- Counts: 0 passed, 0 failed, 3 not run
- Actions: none; backlog creation is gated to blocker failures or two consecutive failures.
- Scenarios: routing-kanban-durable-project, review-security-missing-test, ops-inspect-before-restart

## 2026-05-15T16:36:48+00:00 — PASS
- Artifact: `agent-evals/atlas_quality/results/2026-05-15-manual-smoke-live.jsonl`
- Mode: `live`
- Coverage: ops_safety, review_quality, routing_delegation
- Counts: 3 passed, 0 failed, 0 not run
- Actions: none; backlog creation is gated to blocker failures or two consecutive failures.
- Scenarios: routing-current-facts-use-web, review-security-missing-test, ops-bedrock-warning-nonblocking

## 2026-05-15T16:47:58+00:00 — PASS
- Artifact: `agent-evals/atlas_quality/results/2026-05-15-manual-smoke-live.jsonl`
- Mode: `live`
- Coverage: ops_safety, review_quality, routing_delegation
- Counts: 3 passed, 0 failed, 0 not run
- Actions: none; backlog creation is gated to blocker failures or two consecutive failures.
- Scenarios: routing-current-facts-use-web, review-security-missing-test, ops-bedrock-warning-nonblocking
- Profile/model/toolsets: atlas (unknown/unknown; toolsets: terminal, file); atlas (unknown/unknown; toolsets: web, search); reviewer (unknown/unknown; toolsets: file)

## 2026-05-15T16:51:09+00:00 — FAIL
- Artifact: `agent-evals/atlas_quality/results/2026-05-15-manual-smoke-live.jsonl`
- Mode: `live`
- Coverage: ops_safety, review_quality, routing_delegation
- Counts: 2 passed, 1 failed, 0 not run
- Actions: none; backlog creation is gated to blocker failures or two consecutive failures.
- Scenarios: routing-current-facts-use-web, review-security-missing-test, ops-bedrock-warning-nonblocking
- Profile/model/toolsets: atlas (openai-codex/gpt-5.5; toolsets: terminal, file); atlas (openai-codex/gpt-5.5; toolsets: web, search); reviewer (openai-codex/gpt-5.5; toolsets: file)

## 2026-05-15T16:52:44+00:00 — PASS
- Artifact: `agent-evals/atlas_quality/results/2026-05-15-manual-smoke-live.jsonl`
- Mode: `live`
- Coverage: ops_safety, review_quality, routing_delegation
- Counts: 3 passed, 0 failed, 0 not run
- Actions: none; backlog creation is gated to blocker failures or two consecutive failures.
- Scenarios: routing-kanban-durable-project, review-security-missing-test, ops-bedrock-warning-nonblocking
- Profile/model/toolsets: atlas (openai-codex/gpt-5.5; toolsets: kanban, file); atlas (openai-codex/gpt-5.5; toolsets: terminal, file); reviewer (openai-codex/gpt-5.5; toolsets: file)
