Agent-Oriented Project Diagram

This is a high-signal, agent-oriented view of Flynn's structure and execution flow.

If you're new to the codebase, start here, then jump to the referenced files.

System Map (Boundaries + Trust)

This is the fastest way to understand what runs where, and where the security boundaries sit.

flowchart LR
  subgraph EXT[External Systems]
    MP[Model Providers\nAnthropic/OpenAI/Gemini/...\nvia ModelClient]
    CH[Chat Networks\nTelegram/Discord/Slack/WhatsApp/...]
    WEB[Web\nsearch/fetch targets]
    GOOG[Google APIs\nGmail/Calendar/Docs/Drive/Tasks]
  end

  subgraph CFG[Config Sources]
    CD[config/default.yaml]
    CO[config/profiles/*.overlay.yaml]
    CG[Generated config profile\nconfig/paas.yaml]
    CE[ENV vars + expansion]
  end

  subgraph HOST[Host (Flynn Daemon)]
    CA[ChannelAdapters]
    GW[Gateway\nHTTP + WS JSON-RPC + Web UI]
    MC[MetricsCollector\nrun state + cancel latency + reactions]
    RT[Routing\ncreateMessageRouter()]
    PF[Preferences\n~/.local/share/flynn/preferences.json\nmodelTier + backendMode]
    SM[SessionManager\nSQLite]
  OR[AgentOrchestrator]
  NA[NativeAgent\n(tool loop)]
  EB[Optional External Backends\nclaude_code/opencode/codex/gemini/pi_embedded]
  MR[ModelRouter]
  TP[ToolPolicy + ToolRegistry]
  TE[ToolExecutor\nhooks + enforcement + audit]
  MEM[Memory Store\nfiles + vector/keyword\nuser/profile + user/working]
    AU[Audit Logger\nredacted]
    HS[Hooks/Autonomy\nconfirm/log/silent]
    GA[Google OAuth Runtime\nsrc/google/oauth.ts]
    AS[Auth Store\n~/.config/flynn/auth.json]
    TF[Legacy Token Files\n~/.config/flynn/*-token.json]
  end

  subgraph SBX[Sandbox (per-session Docker)]
    ST[Sandboxed Tools\nshell/process/fs writes]
    FS[Sandbox FS\nworkspace mount (scoped)]
    NET[Sandbox Network\n(egress policy)]
  end

  CD --> CG
  CO --> CG
  CE --> CG

  CG --> RT
  CE --> RT
  PF --> RT

  CH --> CA
  GW --> RT
  GW --> MC
  CA --> RT
  RT --> SM
  RT --> OR
  RT --> EB
  OR --> NA
  OR -->|session-start memory| MEM
  EB --> MP
  NA --> MR
  MR --> MP

  NA --> TP
  TP --> TE
  TE --> HS
  TE --> AU
  TE --> MEM

  TE -->|high-risk tools| ST
  ST --> FS
  ST --> NET

  TE -->|web tools| WEB
  TE -->|google tools| GA
  GA --> GOOG
  GA <--> AS
  GA --> TF

Big Picture (Runtime Data Flow)

Inbound Message
  (Telegram/Discord/Slack/WhatsApp/WebChat)
        |
        v
ChannelAdapter  ->  ChannelRegistry
        |               |
        |               v
        |        createMessageRouter()
        |               |
        |               +----> Runtime backend mode overrides
        |                      (/runtime status|activate pi|deactivate pi|use config)
        |                      (TUI `/runtime` is forwarded via gateway/daemon command path)
        |               |
        |               v
        |         SessionManager
        |               |
        |               v
        |        AgentOrchestrator
        |               |
        |               v
        |           NativeAgent
        |               |
        |        ModelRouter.chat()
        |               |
        |               v
        |          ModelClient
        |
        +----> (optional, non-tool turns) ExternalBackend
                         (claude_code/opencode/codex/gemini/pi_embedded)
        |
        +----> (optional) PairingManager gate for unknown senders

Tool Calls (inside NativeAgent loop)
  NativeAgent -> ToolRegistry (policy-filtered) -> ToolExecutor
                 |                                 |
                 |                                 v
                 |                          HookEngine + autonomy
                 |                                 |
                 |                                 v
                 |                               Tool.execute()
                 |                                 |
                 |                                 v
                 +---------------------------> AuditLogger (redacted)

Subagent sessions (multi-turn child agents)
  parent AgentOrchestrator -> subagent.* tools -> SubagentManager (TTL cleanup + queue/budget controls)
  SubagentManager -> child AgentOrchestrator (session namespace: subagent:<parent>:<id>, trace_id)
  SubagentManager -> AuditLogger (subagent.lifecycle + subagent.turn events)
  child AgentOrchestrator -> NativeAgent/tool loop (same policy engine, recursion tools removed)

Session start (when `memory.user_namespace` is set)
  AgentOrchestrator -> MemoryStore (user/profile + user/working)
  AgentOrchestrator -> System prompt (session context injection)

Outbound Reply
  -> ChannelAdapter.send()  (text + optional attachments)

Gateway streaming UX signals:

WebSocket agent.send emits run_state lifecycle events (start, cancel_requested, cancelled, complete, error) for UI/state rendering.
Routing applies reaction rules with deterministic priority/cooldown (and recursion guard) before intent routing.
Companion nodes re-register node.* capabilities after reconnect; runtime clients can auto-reconnect, optionally replay cached node state (register/status/location/push), and surface connection events.
flynn companion --export-bootstrap <path|-> can emit a resolved companion bootstrap manifest (gateway/node/runtime contract) for desktop/mobile packaging flows without opening a runtime connection.
flynn companion --export-release-bundle <dir> can emit a distributable shell bundle (bootstrap JSON + launcher + README + SHA-256 checksums) for desktop/mobile packaging pipelines.
flynn companion --export-release-bundle ... --signing-key <pem> can additionally emit CHECKSUMS.sha256.sig for signed artifact verification pipelines.
flynn companion --verify-release-bundle <dir> can validate bundle checksums and optional signatures before installation or rollout.
flynn companion --export-shell-template <dir> can emit platform starter shell templates (macOS/iOS/Android native scaffold files + bootstrap JSON), including iOS/Android runtime skeletons for registration, heartbeat/status, location/push updates, and handoff.
pnpm companion:bundle -- --output <dir> ... runs a build-and-verify release pipeline for repeatable companion artifact generation.
pnpm companion:reference-apps -- --output apps/companion regenerates in-repo macOS/iOS/Android reference app starter surfaces plus a runnable macos-app Swift Package menu-bar scaffold.
pnpm companion:reference-apps:check verifies committed apps/companion artifacts match generator output.
.github/workflows/companion-release-bundle.yml provides CI artifact generation for companion release bundles using the same build-and-verify pipeline.
.github/workflows/companion-reference-apps-check.yml enforces reference-app generator sync in CI.
flynn companion can bootstrap status/location/push metadata on connect (node.status.set + optional node.location.set/node.push_token.set) so thin companion shells can register operational context in one launch.
pnpm audit:phase0-baseline:live captures anonymized channel-origin live run/reaction baseline artifacts from real audit logs.
pnpm audit:phase0-baseline:live:pi and pnpm audit:phase0-baseline:live:native capture backend-scoped channel windows using backend.route timelines.
pnpm audit:phase0-baseline:live:gateway captures gateway-origin baseline windows by auto-selecting the latest cancel/cancelled session window (or use scripts/capture-phase0-live-baseline.ts --source gateway --since ... --until ... for explicit windows).
pnpm audit:phase0-baseline:live:refresh runs channel + gateway + backend-scoped (pi_embedded and native) capture commands in one cadence step.
pnpm audit:phase0-baseline:live:drift evaluates backend-scoped artifact freshness/drift gates and writes docs/plans/artifacts/phase0_baseline_live_backend_drift_<UTC-date>.md/.json; pnpm audit:phase0-baseline:live:refresh:drift runs capture + drift checks in one cadence step.
pnpm audit:phase0-baseline:live:refresh:drift:rolling runs the same full refresh+drift flow with a shared UTC timestamp tag (YYYY-MM-DD-HHMMSS) so each cadence run keeps distinct backend/drift artifacts for immediate baseline-vs-prior comparisons.
pnpm audit:phase0-baseline:live:prune provides dry-run retention planning for rolling-tag artifacts; pnpm audit:phase0-baseline:live:prune:apply deletes older rolling snapshots while keeping the newest tags per artifact family.
audit:phase0-baseline:live* scripts are cadence-safe by default (UTC-date tags auto-generated unless explicitly overridden).
Canvas artifacts are persisted by the gateway so session UI surfaces can recover after daemon restarts.
TTS synthesis uses an ordered provider chain with health cooldown tracking; if all providers fail, replies degrade to text-only without dropping the response.
Talk mode accepts spoken/text stop/cancel while active and maps it onto the same /stop run-control cancellation path used for text sessions.
Setup/onboarding now applies a first-run Personal Assistant Mode preset, prints model/channel/memory/automation live readiness checks, and emits a guided first-success task path after config save.

Key files:

Routing + per-session agent creation: src/daemon/routing.ts
Subagent session manager (child orchestrators): src/backends/native/subagents.ts
Runtime preference persistence (modelTier, backendMode): src/preferences.ts
Orchestration: src/backends/native/orchestrator.ts
Tool loop: src/backends/native/agent.ts
External backend adapters: src/backends/external.ts, src/backends/piEmbedded.ts
Model routing: src/models/router.ts
Tool policy + execution: src/tools/policy.ts, src/tools/executor.ts
Canary backend telemetry summarization (offline evaluation): src/audit/backendCanarySummary.ts, scripts/summarize-backend-canary.ts
Phase 0 baseline telemetry summarization: src/audit/phase0BaselineSummary.ts, scripts/summarize-phase0-baseline.ts

Component Graph (Agent-Safety Boundary)

                 +---------------------------+
                 |         Config            |
                 |   (Zod schema + YAML)     |
                 |   src/config/schema.ts    |
                 +-------------+-------------+
                               |
                               v
 +-------------------+   +-------------+   +------------------+
 |   SkillRegistry   |   | ToolPolicy  |   |   HookEngine      |
 | src/skills/*      |   | src/tools/* |   | src/hooks/*       |
 +---------+---------+   +------+------+   +---------+--------+
           |                    |                    |
           | (system prompt)    | (allow/deny)       | (confirm/log/silent)
           v                    v                    v
 +-------------------+   +-------------+   +------------------+
 | System Prompt     |   | ToolRegistry|   | ToolExecutor      |
 | src/daemon/services.ts| src/tools/* |   | src/tools/executor.ts
 +---------+---------+   +------+------+   +---------+--------+
           |                    |                    |
           v                    |                    |
 +-------------------+          |                    v
 | AgentOrchestrator |          |              +-----------+
 | src/backends/*    |          +------------> | AuditLogger|
 +---------+---------+                         | src/audit/*|
           |
           v
 +-------------------+
 |   NativeAgent     |
 | src/backends/*    |
 +---------+---------+
           |
           v
 +-------------------+
 |   ModelRouter      |
 | src/models/*       |
 +-------------------+

Skills + Capabilities (What Gets Enforced)

Skills are local directories with:

SKILL.md (instructions injected into the system prompt)
manifest.json (metadata + optional permissions)

Skill permissions enforcement points

Tool availability: ToolPolicy.resolveAllowedNames() intersects allowed tools with manifest.json.permissions.
Tool execution (defense in depth): ToolExecutor.execute() enforces:
- fs allowlists (permissions.fs.read / permissions.fs.write)
- net allowlists (best-effort for web.fetch)
- secret scopes (tools declare requiredSecretScopes, skills allow permissions.secrets)
- injection guard when untrusted content is present

Important default:

If a request is routed into a skill context but the skill has no permissions manifest, tool access is denied.

Key files:

Skill manifest types: src/skills/types.ts
Loader validation: src/skills/loader.ts
Policy intersection: src/tools/policy.ts
Executor enforcement: src/tools/executor.ts

Sandbox Execution (High-Risk Tools)

Flynn supports per-session Docker sandboxes.

Where sandboxing is applied today:

shell.exec and process.start can be replaced with sandboxed implementations.
Replacement is wired in src/daemon/routing.ts by cloning the ToolRegistry and swapping the tool implementations.

Skill context default:

High-risk tool execution defaults to sandbox in skill context (when available).
A skill can opt into host execution only by setting permissions.execution_environment: "host".

Key files:

Sandbox lifecycle: src/sandbox/manager.ts, src/sandbox/docker.ts
Sandboxed tool wrappers: src/sandbox/tools.ts
Wiring: src/daemon/routing.ts

Prompt Injection Hardening (Practical)

Flynn treats content provenance as part of the control boundary:

web.fetch, web.search, and browser.content outputs are treated as untrusted "fetched_content".
Tool results are wrapped in provenance markers inside the tool loop.
Once untrusted content is seen, ToolExecutor applies stricter gating (blocks obvious injection patterns for high-risk tools).
Browser workflow tools add execution guardrails in the tool layer: allowed_domains, explicit high-risk confirmations, bounded retry policies, and step-budget enforcement.

Key files:

Provenance wrapping: src/backends/native/agent.ts
Tool-call guard: src/tools/executor.ts
System prompt safety guidance: src/daemon/services.ts

Mermaid (For Fast Visual Scanning)

If your renderer supports Mermaid, this is the same information as a sequence diagram.

sequenceDiagram
  autonumber
  participant U as User
  participant CA as ChannelAdapter
  participant CR as ChannelRegistry
  participant SM as SessionManager
  participant AR as AgentOrchestrator
  participant NA as NativeAgent
  participant MR as ModelRouter
  participant MC as ModelClient
  participant FC as Fallback Client
  participant TP as ToolPolicy/Registry
  participant TE as ToolExecutor
  participant HE as HookEngine
  participant AL as AuditLogger
  participant GA as Google OAuth Runtime
  participant AS as Auth Store
  participant TF as Token Files
  participant GP as Google APIs

  U->>CA: message
  CA->>CR: onMessage(InboundMessage)
  CR->>SM: getSession(channel, sender)
  SM-->>CR: Session
  CR->>AR: getOrCreateAgent(session + routing)
  AR->>NA: process(userMessage)
  NA->>MR: chat(messages + tools)
  MR->>MC: provider request
  alt primary model success
    MC-->>MR: response (content or tool_calls)
  else primary model error
    Note over MR: retry + tier/global fallback\n(skip duplicate clients)
    MR->>FC: fallback provider request
    FC-->>MR: fallback response
  end
  MR-->>NA: ChatResponse

  alt model requests tool use
    NA->>TP: filtered tool list (skill + policy)
    NA->>TE: execute(tool, args, context)
    TE->>HE: confirm/log/silent (autonomy)
    HE-->>TE: approved/denied
    alt google.* tool execution
      TE->>GA: createGoogleOAuth2Client(service)
      GA->>AS: load stored token
      alt auth store token missing
        GA->>TF: read legacy token file
        TF-->>GA: token
        GA->>AS: migrate token record
      end
      GA->>GP: API request with refreshed OAuth creds
    end
    TE->>AL: audit (redacted)
    TE-->>NA: ToolResult
    NA->>MR: chat(tool_result blocks)
  end

  NA-->>AR: assistant response
  AR-->>CR: OutboundMessage
  CR-->>CA: send()
  CA-->>U: reply

16 KiB Raw Blame History

Agent-Oriented Project Diagram

System Map (Boundaries + Trust)

Big Picture (Runtime Data Flow)

Component Graph (Agent-Safety Boundary)

Skills + Capabilities (What Gets Enforced)

Skill permissions enforcement points

Sandbox Execution (High-Risk Tools)

Prompt Injection Hardening (Practical)

Mermaid (For Fast Visual Scanning)

16 KiB

Raw Blame History