docs(audit): add report, google auth runbook, and agent repo map

2026-02-23 17:12:41 -08:00
parent 092a9baeae
commit d8188b5425
7 changed files with 344 additions and 1 deletions
@@ -6,6 +6,7 @@ This documentation is written to be useful to both humans and AI agents. If you

 1. Architecture overview (agent-oriented)
   - `docs/architecture/AGENT_DIAGRAM.md`
+   - `docs/architecture/AGENT_REPO_MAP.md`
   - `docs/architecture/GATEWAY_SESSIONS_AND_QUEUE.md`
   - `docs/architecture/TYPESCRIPT_MAP.md`
   - `docs/architecture/SYMBOL_INDEX.md`
@@ -21,6 +22,7 @@ This documentation is written to be useful to both humans and AI agents. If you
   - `docs/performance/TUNING.md`
 6. Operations runbooks
   - `docs/operations/OPERATOR_PACK.md`
+   - `docs/operations/GOOGLE_AUTH.md`

 ## Quick Map (One Diagram)

@@ -0,0 +1,85 @@
+# Agent Repo Map
+
+This file is an agent-facing operational map of the Flynn repo: entrypoints, key modules, conventions, config schema anchors, and debug workflows.
+
+## Entry Points
+
+- Daemon start: `src/cli/index.ts` -> `src/daemon/index.ts#startDaemon`
+- One-shot send: `src/cli/send.ts`
+- TUI:
+  - Minimal: `src/frontends/tui/minimal.ts`
+  - Fullscreen Ink app: `src/frontends/tui/components/App.tsx`
+- Gateway server: `src/gateway/server.ts`
+
+## Core Message Flow
+
+1. Channel ingress/gateway request -> `src/daemon/routing.ts`
+2. Session/orchestration -> `src/backends/native/orchestrator.ts`
+3. Agent loop/tool execution -> `src/backends/native/agent.ts`
+4. Model dispatch -> `src/models/router.ts` + provider clients in `src/models/*`
+
+## High-Value Modules
+
+- Config schema + defaults:
+  - `src/config/schema.ts`
+  - `config/default.yaml`
+  - `config/profiles/*.overlay.yaml` (profile overlays)
+- Model wiring:
+  - `src/daemon/models.ts`
+  - `src/models/router.ts`
+  - `src/models/openai.ts`, `src/models/anthropic.ts`, `src/models/gemini.ts`, `src/models/bedrock.ts`, `src/models/github.ts`, `src/models/local/*`
+- Tool registration chain:
+  - Tool impl: `src/tools/builtin/*`
+  - Export: `src/tools/builtin/index.ts`
+  - Registry/bootstrap: `src/daemon/index.ts`, `src/daemon/tools.ts`
+- Channel adapters:
+  - `src/channels/*/adapter.ts`
+  - Registration: `src/daemon/channels.ts`
+- Auth stores:
+  - OpenAI/Anthropic/etc: `src/auth/*`
+  - Google services: `src/auth/google.ts` + `src/google/oauth.ts`
+- Observability:
+  - Audit logger: `src/audit/*`
+  - Doctor checks: `src/cli/doctor.ts`
+
+## Conventions
+
+- TypeScript strict mode, NodeNext modules.
+- Local imports use `.js` extension.
+- Keep provider-specific branches localized in provider clients; prefer shared helpers for cross-service auth logic.
+- Tests use Vitest (`*.test.ts`).
+
+## Config Schema Anchors
+
+- Model tiers and fallback chain: `models.*` in `src/config/schema.ts`
+- Tool policy and profiles: `tools.*`
+- Automation integrations:
+  - Gmail watcher: `automation.gmail`
+  - Google tools: `automation.gcal`, `automation.gdocs`, `automation.gdrive`, `automation.gtasks`
+- Server/gateway behavior: `server.*`
+
+## Run and Debug
+
+```bash
+pnpm dev                 # daemon watch mode
+pnpm tui                 # minimal terminal UI
+pnpm tui:fs              # fullscreen UI
+pnpm test:run            # full test run once
+pnpm lint
+pnpm typecheck
+```
+
+Focused debug commands:
+
+```bash
+pnpm test:run src/models/router.test.ts src/models/openai.oauth.test.ts
+pnpm test:run src/tools/builtin/gmail.test.ts src/automation/gmail.test.ts
+pnpm test:run src/cli/doctor.test.ts
+```
+
+Config profile sync:
+
+```bash
+pnpm config:profiles:generate
+pnpm config:profiles:check
+```
@@ -0,0 +1,68 @@
+# Google OAuth Runbook
+
+This runbook describes how Flynn acquires, stores, refreshes, and rotates Google OAuth tokens for Gmail, Calendar, Docs, Drive, and Tasks.
+
+## Scope Policy (Least Privilege)
+
+| Service | Command | Scope(s) |
+|---|---|---|
+| Gmail | `flynn gmail-auth` | `https://www.googleapis.com/auth/gmail.settings.basic`, `https://www.googleapis.com/auth/gmail.readonly` |
+| Calendar | `flynn gcal-auth` | `https://www.googleapis.com/auth/calendar.readonly` |
+| Docs | `flynn gdocs-auth` | `https://www.googleapis.com/auth/documents.readonly`, `https://www.googleapis.com/auth/drive.metadata.readonly` |
+| Drive | `flynn gdrive-auth` | `https://www.googleapis.com/auth/drive.readonly` |
+| Tasks | `flynn gtasks-auth` | `https://www.googleapis.com/auth/tasks.readonly` |
+
+Unified entrypoint:
+
+```bash
+flynn google-auth --service gmail
+flynn google-auth --service gcal
+flynn google-auth --service gdocs
+flynn google-auth --service gdrive
+flynn google-auth --service gtasks
+```
+
+Add `--manual` for copy/paste flow, or `--config <path>` for non-default config.
+
+## Storage Model
+
+- Canonical token store: `~/.config/flynn/auth.json`
+  - Path: `google.services.<service>.token`
+  - Metadata: scopes, token file path, credentials file path, `updated_at`
+- Compatibility token files remain supported:
+  - `~/.config/flynn/gmail-token.json`
+  - `~/.config/flynn/gcal-token.json`
+  - `~/.config/flynn/gdocs-token.json`
+  - `~/.config/flynn/gdrive-token.json`
+  - `~/.config/flynn/gtasks-token.json`
+
+Runtime behavior:
+
+1. Flynn checks auth store first.
+2. If missing, Flynn loads legacy token file and migrates it into auth store.
+3. If both are missing, tools/watchers fail with a service-specific re-auth command hint.
+
+## Refresh and Rotation Behavior
+
+- OAuth client refresh events write back to:
+  - auth store (`auth.json`)
+  - service token file (compatibility)
+- If auth store write is blocked (for example `EACCES`), Flynn keeps operating from token files.
+- Re-auth is required when Google returns insufficient-scope errors for a service.
+
+## Local Operation Checklist
+
+1. Configure `automation.<service>.credentials_file` in Flynn config.
+2. Run service auth command (or `flynn google-auth --service <service>`).
+3. Run `flynn doctor` and verify service checks:
+   - `Gmail configured`
+   - `Google Calendar configured`
+   - `Google Docs configured`
+   - `Google Drive configured`
+   - `Google Tasks configured`
+
+## Failure and Renewal Signals
+
+- Missing credentials file: hard failure with path.
+- Missing tokens (store + file): doctor warning + runtime auth error with command hint.
+- Scope mismatch: runtime error includes explicit re-auth command for the affected service.
@@ -72,6 +72,11 @@ If the daily briefing reports Calendar or Tasks as blocked with insufficient per
 1. `flynn gcal-auth`
 2. `flynn gtasks-auth`

+Equivalent unified form:
+
+1. `flynn google-auth --service gcal`
+2. `flynn google-auth --service gtasks`
+
 Expected scopes after re-auth:

 - Calendar: `https://www.googleapis.com/auth/calendar.readonly`
@@ -3,6 +3,36 @@
  "updated_at": "2026-02-24",
  "description": "Tracks the status of all Flynn plans and implementation phases",
  "plans": {
+    "full-audit-hardening-and-config-consolidation": {
+      "status": "completed",
+      "date": "2026-02-24",
+      "updated": "2026-02-24",
+      "summary": "Completed a repo-wide audit and implemented hardening/refactors for Google OAuth token handling, router fallback correctness, config-profile consolidation, and audit logging path behavior. Added Google service coverage to `flynn doctor`, introduced a unified `flynn google-auth` command, aligned fallback-chain defaults with runtime semantics, expanded provider capability type coverage, and produced operator/agent-facing documentation plus REPORT.md.",
+      "files_modified": [
+        "src/auth/google.ts",
+        "src/google/oauth.ts",
+        "src/tools/builtin/gmail.ts",
+        "src/tools/builtin/gcal.ts",
+        "src/tools/builtin/gdocs.ts",
+        "src/tools/builtin/gdrive.ts",
+        "src/tools/builtin/gtasks.ts",
+        "src/automation/gmail.ts",
+        "src/cli/google-auth.ts",
+        "src/cli/doctor.ts",
+        "src/models/router.ts",
+        "src/models/openai.ts",
+        "src/config/schema.ts",
+        "config/profiles/paas.overlay.yaml",
+        "scripts/generate-config-profiles.mjs",
+        "src/audit/logger.ts",
+        "README.md",
+        "docs/operations/GOOGLE_AUTH.md",
+        "docs/architecture/AGENT_REPO_MAP.md",
+        "REPORT.md",
+        "docs/plans/state.json"
+      ],
+      "test_status": "pnpm test:run (18 focused suites, 420 tests) + pnpm lint (0 errors) + pnpm typecheck + pnpm config:profiles:check passing"
+    },
    "daily-briefing-google-scope-remediation": {
      "status": "completed",
      "date": "2026-02-24",
@@ -6370,7 +6400,7 @@
    }
  },
  "overall_progress": {
-    "total_test_count": 1971,
+    "total_test_count": 1982,
    "all_tests_passing": true,
    "p0_completion": "3/3 (100%)",
    "p1_completion": "4/4 (100%)",
@@ -6395,6 +6425,9 @@
    "daily_briefing_google_scope_remediation": "completed — calendar.* and tasks.* now append explicit re-auth guidance (`flynn gcal-auth` / `flynn gtasks-auth`) for insufficient-scope errors, and operator runbook includes remediation steps",
    "council_tool_timeout_override": "completed — ToolExecutor supports per-tool timeout overrides and council.run now uses a 180s timeout to avoid false 30s council timeouts in the tool loop",
    "minimal_tui_multiline_paste_mode": "completed — minimal TUI now supports `/paste`/`/multiline` multiline compose mode ending with single '.' line, preventing newline truncation for pasted prompts",
+    "config_profile_consolidation": "completed — config/paas.yaml is now generated from canonical config/default.yaml + config/profiles/paas.overlay.yaml with CI-checkable drift detection",
+    "google_auth_hardening": "completed — shared Google OAuth runtime helper + auth store (auth.json), legacy token-file migration, refresh persistence, service-wide doctor checks, and unified `flynn google-auth` command",
+    "model_router_correctness": "completed — fallback paths now avoid duplicate clients, apply retry policy consistently, and reject unsupported OpenAI OAuth tool requests early",
    "native_audio_support": "completed — smart routing for native audio (Gemini/OpenAI/GitHub) vs Whisper transcription fallback, plus 2026-02-23 arg hydration hardening, tool.args_rewritten audit metric, transient fetch retry/timeout hardening, localhost->127.0.0.1 fallback for transcription endpoint connectivity, and whisper docker-compose entrypoint arg fix for port 18801",
    "remaining_phases_completion": "Phase 1: 3/3 (100%) — context levels, command registry, memory structure. Phase 2: 3/3 (100%) — component registry, confidence routing, history index. Phase 3: 2/2 (100%) — adaptive memory/compaction, truthfulness/autonomy hardening",
    "next_up": "Track OpenClaw evolution regularly for inspiration and feature ideas"