will/swarm-zap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(subagents): record live gpt-5.4 failure verification

Browse Source
This commit is contained in:
zap
2026-03-13 18:12:11 +00:00
parent 08c1981faa
commit 95135eb5f1
4 changed files with 44 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
HANDOFF.md
View File

@@ -4,7 +4,7 @@
Immediate baton-pass for the next fresh implementation session.
## Current objective
Investigate and improve subagent / ACP delegation reliability with evidence-first debugging. The current target is to verify the newly landed upstream fix for subagent error/outcome handling and then continue on any remaining real runtime failures.
Investigate and improve subagent / ACP delegation reliability with evidence-first debugging. The failure-path proof for the new subagent outcome handling is now captured; the remaining question is whether lower-level `agent.wait` semantics also need fixing or whether the issue is sufficiently solved at the subagent registry / completion layer.
## Use these state files first
1. `WIP.subagent-reliability.md` — canonical state for this pass
@@ -27,15 +27,18 @@ Investigate and improve subagent / ACP delegation reliability with evidence-firs
- An upstream patch for that error/outcome handling now exists in `external/openclaw-upstream` on branch `fix/subagent-wait-error-outcome` with targeted tests passing.
## Highest-priority next actions
1. The success side is now verified on a real fresh `gpt-5.4` subagent run.
2. Find and execute the smallest safe controlled-failure repro on a valid model/runtime (`gpt-5.4` preferred) so we can confirm:
- a failing child run is stored as `error` rather than `ok`
- a successful child run stores a useful frozen result / announcement payload
3. Re-check whether ACP-specific Claude/Codex runtime failures are still reproducible after separating them from the generic subagent reporting bug.
4. If another core bug appears, continue in `external/openclaw-upstream/` on a focused branch with targeted validation.
1. Treat the live `gpt-5.4` failure repro as proven for subagent persistence/announcement handling:
- run id `b50cb91f-6219-44f7-9d2f-a1264ac7ceaf`
- child transcript `~/.openclaw/agents/main/sessions/f114b831-000b-4070-a539-85c68d2b7057.jsonl`
- `runs.json` now stores `outcome.status: "error"`, `endedReason: "subagent-error"`, and a non-null `frozenResultText`
2. Decide whether raw gateway `agent.wait` should also report `status: "error"` for terminal assistant errors. Current live evidence for the same failed child:
- `agent.wait` returned `{"runId":"b50cb91f-6219-44f7-9d2f-a1264ac7ceaf","status":"ok","endedAt":1773425130881}`
3. Re-check whether ACP-specific Claude/Codex runtime failures are still reproducible after separating them from the generic subagent outcome bug.
4. Leave the dirty `/subagents log` UX diff out of this branch unless you intentionally spin a separate focused pass; it regression-passed `src/auto-reply/reply/commands.test.ts` but still lacks dedicated feature coverage.
5. Update WIP + memory + tasks before ending.
## Success criteria
- Real-run verification of the new error/outcome fix.
- Real-run verification of the new error/outcome fix. ✅ done for subagent persistence/announcement handling.
- Clear separation between resolved reporting bug(s) and any still-open ACP/runtime failures.
- Explicit decision on whether raw `agent.wait` behavior is acceptable or requires a follow-up fix.
- State files updated with paths, commands, and outcomes.

20
WIP.subagent-reliability.md
View File

@@ -61,8 +61,24 @@ This is the highest-leverage remaining open reliability item because it affects
- result: successful completion with a real final child result (`SUCCESS-PROBE-OK`)
- A later GLM-5 probe was invalid for entitlement reasons and was terminated; it should not be treated as the canonical failure-path verification.
- killed/failed run id: `4965775c-4764-41e9-a77a-692f1ab4c2fd`
- Remaining gap: we still need a controlled failure-path verification on a valid model/runtime so we can confirm failed child runs persist/announce as `error` rather than fake `ok`.
- Next step: continue in a fresh `gpt-5.4` subagent session, find the smallest safe controlled-failure repro that does not depend on unavailable GLM-5 access, run it, and update WIP/HANDOFF with exact evidence.
- Live failure-path verification on a valid working model/runtime is now complete on `gpt-5.4`.
- spawned child run: `b50cb91f-6219-44f7-9d2f-a1264ac7ceaf`
- requester session: `agent:main:subagent-reliability-failure-hex-1773425126098`
- child session: `agent:main:subagent:4c0dd686-cd2e-4cba-b80b-2fbf309a4594`
- child transcript: `~/.openclaw/agents/main/sessions/f114b831-000b-4070-a539-85c68d2b7057.jsonl`
- terminal child assistant message (transcript line 6) recorded:
- `provider: "openai-codex"`
- `model: "gpt-5.4"`
- `stopReason: "error"`
- `errorMessage: "Codex error: {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"code\":\"context_length_exceeded\",\"message\":\"Your input exceeds the context window of this model. Please adjust your input and try again.\",\"param\":\"input\"},\"sequence_number\":2}"`
- matching `~/.openclaw/subagents/runs.json` record now correctly persisted:
- `outcome.status: "error"`
- `outcome.error: "Codex error: {...context_length_exceeded...}"`
- `endedReason: "subagent-error"`
- `frozenResultText: "Codex error: {...context_length_exceeded...}"`
- Important nuance from the same live repro: raw gateway `agent.wait` still returned `{"runId":"b50cb91f-6219-44f7-9d2f-a1264ac7ceaf","status":"ok","endedAt":1773425130881}` for that failed child. So the current fix is verified for persisted/announced **subagent outcomes**, but **not** for the lower-level `agent.wait` RPC semantics.
- Side assessment on unrelated dirty upstream work: the `/subagents log` UX diff in `src/auto-reply/reply/commands-subagents/action-log.ts` + `shared.ts` is logically coherent and passed `pnpm test -- --run src/auto-reply/reply/commands.test.ts` (`44 tests`), but it is still out-of-scope for this reliability pass because there is no dedicated coverage for the new tool-only log behavior and it would muddy the focused branch.
- Next step if continuing core work: decide whether `agent.wait` itself should downgrade terminal assistant errors to `status: "error"`, or whether the current contract is acceptable now that subagent registry persistence/announcements are fixed.
## Constraints
- Prefer evidence over theory.

14
memory/2026-03-13.md
View File

@@ -23,6 +23,18 @@
- Targeted validation passed:
- `pnpm -C /home/openclaw/.openclaw/workspace/external/openclaw-upstream test -- --run src/agents/tools/sessions-helpers.terminal-text.test.ts src/agents/subagent-registry.persistence.test.ts src/gateway/server-methods/server-methods.test.ts`
- result: `50 tests` passed across `3` files
- Follow-up still needed: rerun a real delegated subagent using a known-working model entitlement (`gpt-5.4` preferred for now) to verify successful runs leave a useful frozen result and failed runs now persist as `error`.
- Real success-path verification later passed on `gpt-5.4` with run `23750d80-b481-4f50-b219-cc9245be405f` and final child result `SUCCESS-PROBE-OK`.
- Real failure-path verification later also passed on valid `gpt-5.4` by intentionally triggering a `context_length_exceeded` provider error with a token-dense oversized task payload.
- child run: `b50cb91f-6219-44f7-9d2f-a1264ac7ceaf`
- child session: `agent:main:subagent:4c0dd686-cd2e-4cba-b80b-2fbf309a4594`
- child transcript: `~/.openclaw/agents/main/sessions/f114b831-000b-4070-a539-85c68d2b7057.jsonl`
- transcript terminal assistant entry recorded `provider:"openai-codex"`, `model:"gpt-5.4"`, `stopReason:"error"`, `errorMessage:"Codex error: {...context_length_exceeded...}"`
- matching `~/.openclaw/subagents/runs.json` now correctly stored:
- `outcome.status: "error"`
- `outcome.error: "Codex error: {...context_length_exceeded...}"`
- `endedReason: "subagent-error"`
- `frozenResultText: "Codex error: {...context_length_exceeded...}"`
- Important remaining nuance: raw gateway `agent.wait` for that same failed child still returned `status:"ok"` with only `endedAt`, so the current fix is verified for subagent outcome persistence/announcements but not for lower-level `agent.wait` semantics.
- Side note: unrelated dirty `/subagents log` UX changes in `external/openclaw-upstream` regression-passed `src/auto-reply/reply/commands.test.ts` (44 tests) but were intentionally left out-of-scope for this focused reliability pass.
- Will also explicitly requested that zap keep a light eye on active subagents and check whether they look stuck instead of assuming they are fine until completion.
- Will explicitly reinforced on 2026-03-13 that once planning is done, zap should use subagents ASAP and start implementation in a fresh session rather than continuing to implement inside the long-lived main chat.

3
memory/tasks.json
View File

@@ -27,7 +27,8 @@
"Patch timestamp: 2026-03-04T22:31:50Z",
"Upstream patch committed in external/openclaw-upstream on branch fix/tui-hide-internal-runtime-context commit 0f66a4547 (suppress internal runtime completion context blocks in TUI formatter).",
"Validation: pnpm test:fast completed successfully (812 files / 6599 tests passing) at 2026-03-04T22:53:29Z",
"2026-03-13: confirmed corrected LiteLLM run was still failing (child transcript showed assistant 429/plan error for GLM-5) while runs.json incorrectly stored outcome.status=ok and frozenResultText=null; implemented upstream branch fix/subagent-wait-error-outcome to derive terminal subagent outcome from latest assistant error state, with targeted validation (50 tests passed across 3 files). Still needs live rerun with a known-working model such as gpt-5.4."
"2026-03-13: confirmed corrected LiteLLM run was still failing (child transcript showed assistant 429/plan error for GLM-5) while runs.json incorrectly stored outcome.status=ok and frozenResultText=null; implemented upstream branch fix/subagent-wait-error-outcome to derive terminal subagent outcome from latest assistant error state, with targeted validation (50 tests passed across 3 files).",
"2026-03-13 later: live gpt-5.4 success repro passed (run 23750d80-b481-4f50-b219-cc9245be405f). Live gpt-5.4 failure repro also passed for subagent persistence/announcement handling: child run b50cb91f-6219-44f7-9d2f-a1264ac7ceaf ended with transcript stopReason=error + context_length_exceeded, and runs.json now stored outcome.status=error / endedReason=subagent-error / frozenResultText non-null. Remaining open nuance: raw agent.wait for that same failed child still returned status=ok."
]
},
{