will/flynn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7b170cff4d4038f27811772e4d80fadf7ccfd676
flynn/docs/architecture/AGENT_DIAGRAM.md
T
William Valentin 7b170cff4d Phase 2 reactions v2 priority and cooldown
2026-02-25 10:36:56 -08:00

328 lines
11 KiB
Markdown
Raw Blame History

# Agent-Oriented Project Diagram
This is a high-signal, agent-oriented view of Flynn's structure and execution flow.
If you're new to the codebase, start here, then jump to the referenced files.
## System Map (Boundaries + Trust)
This is the fastest way to understand what runs where, and where the security boundaries sit.
```mermaid
flowchart LR
subgraph EXT[External Systems]
MP[Model Providers\nAnthropic/OpenAI/Gemini/...\nvia ModelClient]
CH[Chat Networks\nTelegram/Discord/Slack/WhatsApp/...]
WEB[Web\nsearch/fetch targets]
GOOG[Google APIs\nGmail/Calendar/Docs/Drive/Tasks]
end
subgraph CFG[Config Sources]
CD[config/default.yaml]
CO[config/profiles/*.overlay.yaml]
CG[Generated config profile\nconfig/paas.yaml]
CE[ENV vars + expansion]
end
subgraph HOST[Host (Flynn Daemon)]
CA[ChannelAdapters]
GW[Gateway\nHTTP + WS JSON-RPC + Web UI]
MC[MetricsCollector\nrun state + cancel latency + reactions]
RT[Routing\ncreateMessageRouter()]
PF[Preferences\n~/.local/share/flynn/preferences.json\nmodelTier + backendMode]
SM[SessionManager\nSQLite]
OR[AgentOrchestrator]
NA[NativeAgent\n(tool loop)]
EB[Optional External Backends\nclaude_code/opencode/codex/gemini/pi_embedded]
MR[ModelRouter]
TP[ToolPolicy + ToolRegistry]
TE[ToolExecutor\nhooks + enforcement + audit]
MEM[Memory Store\nfiles + vector/keyword]
AU[Audit Logger\nredacted]
HS[Hooks/Autonomy\nconfirm/log/silent]
GA[Google OAuth Runtime\nsrc/google/oauth.ts]
AS[Auth Store\n~/.config/flynn/auth.json]
TF[Legacy Token Files\n~/.config/flynn/*-token.json]
end
subgraph SBX[Sandbox (per-session Docker)]
ST[Sandboxed Tools\nshell/process/fs writes]
FS[Sandbox FS\nworkspace mount (scoped)]
NET[Sandbox Network\n(egress policy)]
end
CD --> CG
CO --> CG
CE --> CG
CG --> RT
CE --> RT
PF --> RT
CH --> CA
GW --> RT
GW --> MC
CA --> RT
RT --> SM
RT --> OR
RT --> EB
OR --> NA
EB --> MP
NA --> MR
MR --> MP
NA --> TP
TP --> TE
TE --> HS
TE --> AU
TE --> MEM
TE -->|high-risk tools| ST
ST --> FS
ST --> NET
TE -->|web tools| WEB
TE -->|google tools| GA
GA --> GOOG
GA <--> AS
GA --> TF
```
## Big Picture (Runtime Data Flow)
```text
Inbound Message
(Telegram/Discord/Slack/WhatsApp/WebChat)
|
v
ChannelAdapter -> ChannelRegistry
| |
| v
| createMessageRouter()
| |
| +----> Runtime backend mode overrides
| (/runtime status|activate pi|deactivate pi|use config)
| (TUI `/runtime` is forwarded via gateway/daemon command path)
| |
| v
| SessionManager
| |
| v
| AgentOrchestrator
| |
| v
| NativeAgent
| |
| ModelRouter.chat()
| |
| v
| ModelClient
|
+----> (optional, non-tool turns) ExternalBackend
(claude_code/opencode/codex/gemini/pi_embedded)
|
+----> (optional) PairingManager gate for unknown senders
Tool Calls (inside NativeAgent loop)
NativeAgent -> ToolRegistry (policy-filtered) -> ToolExecutor
| |
| v
| HookEngine + autonomy
| |
| v
| Tool.execute()
| |
| v
+---------------------------> AuditLogger (redacted)
Outbound Reply
-> ChannelAdapter.send() (text + optional attachments)
```
Gateway streaming UX signals:
- WebSocket `agent.send` emits `run_state` lifecycle events (`start`, `cancel_requested`, `cancelled`, `complete`, `error`) for UI/state rendering.
- Routing applies reaction rules with deterministic priority/cooldown (and recursion guard) before intent routing.
Key files:
- Routing + per-session agent creation: `src/daemon/routing.ts`
- Runtime preference persistence (`modelTier`, `backendMode`): `src/preferences.ts`
- Orchestration: `src/backends/native/orchestrator.ts`
- Tool loop: `src/backends/native/agent.ts`
- External backend adapters: `src/backends/external.ts`, `src/backends/piEmbedded.ts`
- Model routing: `src/models/router.ts`
- Tool policy + execution: `src/tools/policy.ts`, `src/tools/executor.ts`
- Canary backend telemetry summarization (offline evaluation): `src/audit/backendCanarySummary.ts`, `scripts/summarize-backend-canary.ts`
- Phase 0 baseline telemetry summarization: `src/audit/phase0BaselineSummary.ts`, `scripts/summarize-phase0-baseline.ts`
## Component Graph (Agent-Safety Boundary)
```text
+---------------------------+
| Config |
| (Zod schema + YAML) |
| src/config/schema.ts |
+-------------+-------------+
|
v
+-------------------+ +-------------+ +------------------+
| SkillRegistry | | ToolPolicy | | HookEngine |
| src/skills/* | | src/tools/* | | src/hooks/* |
+---------+---------+ +------+------+ +---------+--------+
| | |
| (system prompt) | (allow/deny) | (confirm/log/silent)
v v v
+-------------------+ +-------------+ +------------------+
| System Prompt | | ToolRegistry| | ToolExecutor |
| src/daemon/services.ts| src/tools/* | | src/tools/executor.ts
+---------+---------+ +------+------+ +---------+--------+
| | |
v | |
+-------------------+ | v
| AgentOrchestrator | | +-----------+
| src/backends/* | +------------> | AuditLogger|
+---------+---------+ | src/audit/*|
|
v
+-------------------+
| NativeAgent |
| src/backends/* |
+---------+---------+
|
v
+-------------------+
| ModelRouter |
| src/models/* |
+-------------------+
```
## Skills + Capabilities (What Gets Enforced)
Skills are local directories with:
- `SKILL.md` (instructions injected into the system prompt)
- `manifest.json` (metadata + optional `permissions`)
### Skill permissions enforcement points
- Tool availability: `ToolPolicy.resolveAllowedNames()` intersects allowed tools with `manifest.json.permissions`.
- Tool execution (defense in depth): `ToolExecutor.execute()` enforces:
- fs allowlists (`permissions.fs.read` / `permissions.fs.write`)
- net allowlists (best-effort for `web.fetch`)
- secret scopes (tools declare `requiredSecretScopes`, skills allow `permissions.secrets`)
- injection guard when untrusted content is present
Important default:
- If a request is routed into a skill context but the skill has no `permissions` manifest, **tool access is denied**.
Key files:
- Skill manifest types: `src/skills/types.ts`
- Loader validation: `src/skills/loader.ts`
- Policy intersection: `src/tools/policy.ts`
- Executor enforcement: `src/tools/executor.ts`
## Sandbox Execution (High-Risk Tools)
Flynn supports per-session Docker sandboxes.
Where sandboxing is applied today:
- `shell.exec` and `process.start` can be replaced with sandboxed implementations.
- Replacement is wired in `src/daemon/routing.ts` by cloning the ToolRegistry and swapping the tool implementations.
Skill context default:
- High-risk tool execution defaults to `sandbox` in skill context (when available).
- A skill can opt into host execution only by setting `permissions.execution_environment: "host"`.
Key files:
- Sandbox lifecycle: `src/sandbox/manager.ts`, `src/sandbox/docker.ts`
- Sandboxed tool wrappers: `src/sandbox/tools.ts`
- Wiring: `src/daemon/routing.ts`
## Prompt Injection Hardening (Practical)
Flynn treats content provenance as part of the control boundary:
- `web.fetch`, `web.search`, and `browser.content` outputs are treated as untrusted "fetched_content".
- Tool results are wrapped in provenance markers inside the tool loop.
- Once untrusted content is seen, ToolExecutor applies stricter gating (blocks obvious injection patterns for high-risk tools).
Key files:
- Provenance wrapping: `src/backends/native/agent.ts`
- Tool-call guard: `src/tools/executor.ts`
- System prompt safety guidance: `src/daemon/services.ts`
## Mermaid (For Fast Visual Scanning)
If your renderer supports Mermaid, this is the same information as a sequence diagram.
```mermaid
sequenceDiagram
autonumber
participant U as User
participant CA as ChannelAdapter
participant CR as ChannelRegistry
participant SM as SessionManager
participant AR as AgentOrchestrator
participant NA as NativeAgent
participant MR as ModelRouter
participant MC as ModelClient
participant FC as Fallback Client
participant TP as ToolPolicy/Registry
participant TE as ToolExecutor
participant HE as HookEngine
participant AL as AuditLogger
participant GA as Google OAuth Runtime
participant AS as Auth Store
participant TF as Token Files
participant GP as Google APIs
U->>CA: message
CA->>CR: onMessage(InboundMessage)
CR->>SM: getSession(channel, sender)
SM-->>CR: Session
CR->>AR: getOrCreateAgent(session + routing)
AR->>NA: process(userMessage)
NA->>MR: chat(messages + tools)
MR->>MC: provider request
alt primary model success
MC-->>MR: response (content or tool_calls)
else primary model error
Note over MR: retry + tier/global fallback\n(skip duplicate clients)
MR->>FC: fallback provider request
FC-->>MR: fallback response
end
MR-->>NA: ChatResponse
alt model requests tool use
NA->>TP: filtered tool list (skill + policy)
NA->>TE: execute(tool, args, context)
TE->>HE: confirm/log/silent (autonomy)
HE-->>TE: approved/denied
alt google.* tool execution
TE->>GA: createGoogleOAuth2Client(service)
GA->>AS: load stored token
alt auth store token missing
GA->>TF: read legacy token file
TF-->>GA: token
GA->>AS: migrate token record
end
GA->>GP: API request with refreshed OAuth creds
end
TE->>AL: audit (redacted)
TE-->>NA: ToolResult
NA->>MR: chat(tool_result blocks)
end
NA-->>AR: assistant response
AR-->>CR: OutboundMessage
CR-->>CA: send()
CA-->>U: reply
```
Reference in New Issue View Git Blame Copy Permalink