swarm-zap/WIP.md

# WIP.md

## Current focus
Google Workspace + n8n integration

## Goal
Use OpenClaw as the brain, n8n as the orchestration layer, and Google Workspace as a real execution surface for Gmail/Calendar workflows.

## Current status
Status: `completed`
Owner: `zap`
Started: `2026-03-12`
Completed: `2026-03-12`

### Architecture decision
- Keep `openclaw-action` as the narrow authenticated ingress into n8n.
- Keep approval/state/logging in n8n.
- Use workflow static data for tiny internal state and recent operational breadcrumbs.
- Do not adopt n8n data tables as the default state store yet; revisit only when shared row-based data, UI browsing/editing, or richer querying becomes a real need.
- Use host-side `gog` execution for Google Workspace actions for now.
- Attach execution results back into n8n history so approval items become auditable records.
- Prefer this over broad n8n Google credential sprawl unless a later need clearly justifies it.

### Where n8n is a good fit here
- approval-gated actions and human-in-the-loop workflows
- cross-system glue / action-bus routing
- inbound event filtering, normalization, and dedupe
- notification fanout and formatting
- retries / backoff / rate-limit wrappers around brittle integrations
- fetch-and-normalize pipelines for URLs or provider payloads
- compact audit trail / operator history

### Where n8n is not the default
- primary long-term database
- heavy archival logging
- large or fast-growing datasets
- business logic that is clearer and safer in normal code

## What is already done

### n8n action bus
- [x] Live `openclaw-action` workflow exists and is active.
- [x] Live workflow re-synced from the current workspace asset after implementation completed.
- [x] Core actions verified live:
  - `append_log`
  - `get_logs`
  - `notify`
  - `approval_queue_add`
  - `approval_queue_list`
  - `approval_queue_resolve`
  - `fetch_and_normalize_url`
  - `inbound_event_filter`
- [x] Approval-gated notification execution works live:
  - `send_notification_draft`
  - approve → real `notify`

### Google access
- [x] `gog` installed and authenticated for:
  - Gmail
  - Calendar
  - Drive
  - Contacts
  - Docs
  - Sheets
- [x] Local-only Gog automation env stored outside git at:
  - `/home/openclaw/.openclaw/credentials/gog.env`
- [x] Host bridge implemented:
  - `skills/n8n-webhook/scripts/resolve-approval-with-gog.py`
- [x] n8n history patch action implemented:
  - `approval_history_attach_execution`
- [x] Dry-run end-to-end bridge tests succeeded for:
  - Gmail draft creation
  - Calendar event creation
- [x] Headless non-interactive Gog execution works via the stored local env.
- [x] Direct real Gmail draft create test succeeded.
- [x] Direct real Gmail draft delete test succeeded.
- [x] Real n8n-routed Gmail draft test succeeded end-to-end (queue → approve via bridge → verify → delete).
  - approval id: `approval-mmnvjcak-qcuhbzqd`
  - draft id: `r348335896293726096`
  - subject: `[zap n8n e2e] Gmail draft test 20260312T194153Z`
- [x] Real n8n-routed Calendar event test succeeded end-to-end (queue → approve via bridge → verify → delete).
  - approval id: `approval-mmnvjyo5-uezhcw84`
  - event id: `il3ojkfnsnq3uhlepvrmaklpq4`
  - title: `[zap n8n e2e] Calendar test 20260312T194222Z`

## What is left

### Highest priority: prove the bridge with real Google writes
- [x] Real end-to-end Gmail draft via n8n path:
  1. queue `send_email_draft`
  2. approve via `resolve-approval-with-gog.py`
  3. verify draft exists in Gmail
  4. delete the test draft
- [x] Real end-to-end Calendar event via n8n path:
  1. queue `create_calendar_event`
  2. approve via bridge
  3. verify event exists in Calendar
  4. delete the test event

### Then expand useful Google actions
- [x] Add `delete_email_draft`
- [x] Add `list_email_drafts`
- [x] Add `send_gmail_draft` / send-approved-email path
- [x] Add `update_calendar_event`
- [x] Add `delete_calendar_event`
- [x] Add `list_upcoming_events`
- [ ] Decide whether Drive/Docs/Sheets need action-bus verbs next or can stay direct-tool only for now

### Then polish the operator experience
- [x] Add a compact operator command/reference section for common approval flows
- [x] Add one or two canned test payloads for real bridge verification flows
- [x] Decide/document approval defaults clearly per action family
- [x] Add low-noise reporting so history clearly shows:
  - queued
  - approved/rejected
  - executed
  - execution result id / draft id / event id

## Nice-to-have / future
- [ ] Evaluate whether native n8n Google nodes are worth adding later
- [ ] Add retry/backoff wrappers for fragile or rate-limited actions
- [ ] Expand notification fanout / routing rules if more channels become useful
- [ ] Add more inbound event normalization/filtering when new providers are wired in
- [ ] Revisit n8n data tables only if shared row-based operator data becomes useful
- [ ] If useful, sync this work into the LAN Gitea repo for safer backup/review and easier long-lived tracking

## Current recommendation
Execution should proceed in staged fresh sessions using `WIP.md` as the canonical state file.

Execution note:
- For the remaining implementation passes on this WIP, prefer Codex `gpt-5.4` to reduce iteration time and avoid model-availability churn.

Planned passes:
1. Gmail pass: ✅ complete
   - added `delete_email_draft`
   - added `list_email_drafts`
   - added `send_gmail_draft` / send-approved-email path
   - updated workflow contract/docs/test payloads/bridge + WIP evidence
2. Calendar pass: ✅ complete
   - added `update_calendar_event`
   - added `delete_calendar_event`
   - added `list_upcoming_events`
   - updated workflow contract/docs/test payloads/bridge + WIP evidence
3. Operator/polish pass: ✅ complete
   - documented approval defaults by action family (`notification`, `gmail`, `calendar`, `manual`)
   - added low-noise queue/history reporting (`pending_compact`, `history_compact`, `summary_line`, `result_refs`)
   - added compact operator command/reference docs
   - added two canned recurring verification payloads
   - refreshed `WIP.md` with evidence before ending the pass

## Fresh-session proof refresh (2026-03-12 19:44Z)
- Re-ran both target proofs through the real approval-routed path in a clean implementation session.
- Gmail draft flow:
  - approval id: `approval-mmnvn4t2-w2rjlwz2`
  - draft id: `r-3319106208870238577`
  - subject: `[zap n8n e2e] Gmail draft test 20260312T194450Z`
  - verified via `gog gmail drafts get`
  - cleaned via `gog gmail drafts delete --force`
- Calendar flow:
  - approval id: `approval-mmnvn6i8-e9eq8gdf`
  - event id: `m7prri8vk2opuo6loq3qgtvsv4`
  - title: `[zap n8n e2e] Calendar test 20260312T194450Z`
  - verified via `gog calendar get primary <eventId>`
  - cleaned via `gog calendar delete primary <eventId> --force`

## Gmail pass 1 completion (2026-03-12)
Implemented in this pass:
- workflow contract + router logic for:
  - `list_email_drafts`
  - `delete_email_draft`
  - `send_gmail_draft` (alias: `send_approved_email`)
- explicit per-action approval metadata in queued responses (`approval.policy`, `approval.required`, `approval.mutation_level`)
- host bridge executor coverage for new approval kinds:
  - `email_list_drafts` → `gog gmail drafts list`
  - `email_draft_delete` → `gog gmail drafts delete`
  - `email_draft_send` → `gog gmail drafts send`
- docs + sample payloads + workflow validator updates for the new contract

Targeted verification evidence:
- `python3 skills/n8n-webhook/scripts/validate-workflow.py skills/n8n-webhook/assets/openclaw-action.workflow.json`
  - result: `OK: workflow asset structure looks consistent`
- Router simulation against the shipped workflow JS (`verify-list-001`, `verify-delete-001`, `verify-send-001`, `verify-send-alias-001`):
  - all returned `status: queued_for_approval`
  - alias normalization confirmed (`send_approved_email` → `result.action = send_gmail_draft`)
  - approval metadata returned with expected mutation levels (`low` for list, `high` for delete/send)
  - generated pending ids in the simulation run:
    - `approval-mmny879w-5sncgd98`
    - `approval-mmny879w-a353xg8q`
    - `approval-mmny879w-yvqzokpz`
    - `approval-mmny879w-md99hqxs`
- Bridge dry-run command execution (via real `gog` CLI) for new Gmail kinds:
  - list: exit `0`, returned draft list JSON
  - delete: exit `0`, returned dry-run op `delete gmail draft r-example-draft-id`
  - send: exit `0`, returned dry-run op `gmail.drafts.send`
- `python3 -m py_compile skills/n8n-webhook/scripts/resolve-approval-with-gog.py` passed.

## Calendar pass 2 completion (2026-03-12)
Implemented in this pass:
- workflow contract + router logic for:
  - `list_upcoming_events`
  - `update_calendar_event`
  - `delete_calendar_event`
- host bridge executor coverage for new approval kinds:
  - `calendar_list_events` → `gog calendar events`
  - `calendar_event_update` → `gog calendar update`
  - `calendar_event_delete` → `gog calendar delete`
- docs + sample payloads + workflow validator updates for the expanded calendar contract
- explicit approval policy preserved:
  - `list_upcoming_events` → `approval.mutation_level = low`
  - `update_calendar_event` / `delete_calendar_event` → `approval.mutation_level = high`

Targeted verification evidence:
- `python3 skills/n8n-webhook/scripts/validate-workflow.py skills/n8n-webhook/assets/openclaw-action.workflow.json`
  - result: `OK: workflow asset structure looks consistent`
- Workflow asset inspection confirmed new router actions are present:
  - `list_upcoming_events`
  - `update_calendar_event`
  - `delete_calendar_event`
- Host bridge command-builder verification from shipped sample payloads:
  - `calendar_list_events` → `gog calendar events primary --account will@example.com --json --no-input --max 10 --days 7 --query zap --dry-run`
  - `calendar_event_update` → `gog calendar update primary example-calendar-event-id --account will@example.com --json --no-input --send-updates none --summary Updated call with vendor --from 2026-03-13T18:15:00Z --to 2026-03-13T18:45:00Z --description Updated by OpenClaw action bus. --location Updated room --dry-run`
  - `calendar_event_delete` → `gog calendar delete primary example-calendar-event-id --account will@example.com --json --no-input --force --send-updates none --dry-run`
- `python3 -m py_compile skills/n8n-webhook/scripts/resolve-approval-with-gog.py` passed.

## Operator/polish pass 3 completion (2026-03-12)
Implemented in this pass:
- added explicit approval-family defaults in the shipped workflow + docs:
  - `notification` → required `high`
  - `gmail` read-only (`list_email_drafts`) → required `low`
  - `gmail` mutating (`send_email_draft`, `delete_email_draft`, `send_gmail_draft`) → required `high`
  - `calendar` read-only (`list_upcoming_events`) → required `low`
  - `calendar` mutating (`create_calendar_event`, `update_calendar_event`, `delete_calendar_event`) → required `high`
- added compact operator-facing queue/history fields in the workflow:
  - `payload_preview`
  - `operator.summary_line`
  - `operator.execution_state`
  - `operator.result_refs`
  - `approval_queue_list.result.pending_compact`
  - `approval_queue_list.result.history_compact`
  - `approval_queue_resolve.result.item_compact`
  - `approval_history_attach_execution.result.item_compact`
- taught the host bridge to attach `execution.summary` + `execution.result_refs`
- added recurring verification payloads:
  - `skills/n8n-webhook/assets/test-verify-email-draft-cycle.json`
  - `skills/n8n-webhook/assets/test-verify-calendar-event-cycle.json`
- added operator runbook / recurring verification docs in:
  - `skills/n8n-webhook/references/openclaw-action.md`
  - `skills/n8n-webhook/references/payloads.md`
  - `skills/n8n-webhook/SKILL.md`

Targeted verification evidence:
- `python3 skills/n8n-webhook/scripts/validate-workflow.py skills/n8n-webhook/assets/openclaw-action.workflow.json`
  - result: `OK: workflow asset structure looks consistent`
  - validator now also checks the two new recurring verification payload files
- `python3 -m py_compile skills/n8n-webhook/scripts/resolve-approval-with-gog.py`
  - result: passed
- bridge helper proof via direct import/execution:
  - `execution_result_refs('gmail.drafts.create', {'draft': {'id': 'r-proof-draft-123'}})` → `{'draft_id': 'r-proof-draft-123'}`
  - `execution_summary(...)` → `gmail.drafts.create draft created (draft_id=r-proof-draft-123)`
  - `execution_result_refs('calendar.create', {'event': {'id': 'evt-proof-456'}, 'calendar': 'primary'})` → `{'event_id': 'evt-proof-456', 'calendar': 'primary'}`
  - `execution_summary(...)` → `calendar.create event created (event_id=evt-proof-456, calendar=primary)`
- workflow asset inspection confirmed low-noise operator fields are present:
  - `pending_compact`
  - `history_compact`
  - `summary_line`
  - `result_refs`
  - `default_mode`
  - `approval.family`
- recurring verification payload identity proofs:
  - `test-verify-email-draft-cycle.json` → request id `verify-email-draft-cycle-001`
  - `test-verify-calendar-event-cycle.json` → request id `verify-calendar-event-cycle-001`

## Live deploy + smoke verification (2026-03-12 21:36Z)
- Live workflow id: `Jwi54VWMdlLqYnRo`
- Synced the active n8n workflow in place from the current `skills/n8n-webhook/assets/openclaw-action.workflow.json` asset while preserving:
  - webhook credential binding
  - webhook registration id
  - active state
- First live sync revealed the old minimal router was still running; re-synced from the current full code-node asset and re-activated successfully.
- Safe smoke calls succeeded against the production webhook:
  - `append_log` → `ok: true`
  - `get_logs` → `ok: true`
  - `list_email_drafts` → `status: queued_for_approval`
  - `list_upcoming_events` → `status: queued_for_approval`
  - `approval_queue_list` → `ok: true`, with `pending_compact` + `history_compact` present
  - `fetch_and_normalize_url` against local n8n `/healthz` → `ok: true`, HTTP `200`
  - unknown action → expected HTTP `400` / `unknown_action`
- Smoke-created approval items were rejected and cleaned up:
  - `approval-mmnzm1ev-yjk46sd1`
  - `approval-mmnzm1gi-l7yszi92`
  - `approval-mmnzmw80-kb8szya2`
  - `approval-mmnzmw9w-c25hlml4`
- Remaining pending queue items after cleanup were pre-existing and left untouched:
  - `approval-mmnvgv1o-h06r397e`
  - `approval-mmnulm6r-mfaj7ea8`

## Next-session handoff
This WIP is complete. For the next fresh implementation session, review `HANDOFF.md` plus the proposed next-phase file `WIP.drive-docs-sheets.md`.

Immediate target:
- decide whether Drive / Docs / Sheets actually need action-bus verbs or can remain direct-tool workflows for now
- if Google action coverage expands again, preserve the same approval-family defaults and compact history contract
- refresh WIP/memory/tasks before ending

## Relevant files
- `skills/n8n-webhook/assets/openclaw-action.workflow.json`
- `skills/n8n-webhook/scripts/call-action.sh`
- `skills/n8n-webhook/scripts/resolve-approval-with-gog.py`
- `skills/n8n-webhook/references/openclaw-action.md`
- `memory/2026-03-12.md`
- `/home/openclaw/.openclaw/credentials/gog.env` (local-only, not for git)

## Current branch / checkpoints
- branch: `feat/n8n-action-bus-v2`
- key commits:
  - `ffe7a6b` — add operator approval runbook
  - `249e671` — add compact approval history views
  - `9dcc477` — expand action bus starter workflow
  - `dc990a1` — deploy and verify expanded action bus
  - `1eabaeb` — add approval-gated notification executor
  - `afa48a3` — bridge approvals to gog executors
  - `044e36f` — auto-load local gog automation env