docs: add safety docs and OpenClaw gap roadmap
This commit is contained in:
William Valentin
@@ -67,6 +67,7 @@ Flynn provides a full CLI via the `flynn` binary (or `npx tsx src/cli/index.ts`
|
||||
| `flynn setup` | Interactive setup wizard |
|
||||
| `flynn gmail-auth` | Authenticate with Gmail via OAuth2 |
|
||||
| `flynn gcal-auth` | Authenticate with Google Calendar via OAuth2 |
|
||||
| `flynn skills` | List/install/manage skills |
|
||||
|
||||
### Examples
|
||||
|
||||
@@ -117,6 +118,27 @@ hooks:
|
||||
silent: [notify]
|
||||
```
|
||||
|
||||
## Safety Model
|
||||
|
||||
Flynn is designed to be safe-by-default when expanded beyond "chat":
|
||||
|
||||
- **Tool policy** restricts which tools are even available to a given context (profiles + allow/deny + per-agent/per-provider overrides).
|
||||
- **Skills** can declare explicit capabilities (`manifest.json.permissions`) which are enforced at runtime.
|
||||
- **Sandboxing** can isolate high-risk execution (shell/process) per-session via Docker.
|
||||
- **Prompt-injection hardening** treats fetched content/tool output as untrusted data and blocks obviously unsafe tool calls when untrusted content is present.
|
||||
- **Audit logs** record tool usage and approvals with redaction.
|
||||
|
||||
Details: `docs/security/SAFE_PERSONAL_AGENT.md`
|
||||
|
||||
## Agent-Oriented Architecture Diagram
|
||||
|
||||
If you want a fast mental model of where to start as an AI agent / contributor:
|
||||
|
||||
- `docs/architecture/AGENT_DIAGRAM.md`
|
||||
- `docs/architecture/CONTRIBUTOR_MAP.md`
|
||||
- `docs/architecture/TYPESCRIPT_MAP.md`
|
||||
- `docs/architecture/SYMBOL_INDEX.md`
|
||||
|
||||
### Model Providers
|
||||
|
||||
| Provider | Config |
|
||||
|
||||
@@ -80,6 +80,12 @@ hooks:
|
||||
silent:
|
||||
- notify
|
||||
|
||||
# ── Safety Notes ─────────────────────────────────────────────────────
|
||||
# - Tool policy (tools.profile/allow/deny) controls which tools are available.
|
||||
# - Skills can declare capability permissions in skills/<name>/manifest.json under `permissions`.
|
||||
# Those permissions are enforced at runtime when requests are routed into a skill context.
|
||||
# - See: docs/security/SAFE_PERSONAL_AGENT.md
|
||||
|
||||
# ── Prompt Assembly ───────────────────────────────────────────────────
|
||||
# Tune how much context Flynn loads into the system prompt.
|
||||
#
|
||||
|
||||
+42
-37
@@ -54,6 +54,9 @@ export interface Tool {
|
||||
/** JSON Schema for input validation. */
|
||||
inputSchema: JSONSchema;
|
||||
|
||||
/** Secret scopes required to execute this tool (optional). */
|
||||
requiredSecretScopes?: string[];
|
||||
|
||||
/** Async function that executes the tool. */
|
||||
execute: (args: unknown) => Promise<ToolResult>;
|
||||
}
|
||||
@@ -441,26 +444,14 @@ Tool policy controls which tools are available to agents based on profiles and p
|
||||
|
||||
### Profiles
|
||||
|
||||
```typescript
|
||||
export const PROFILES = {
|
||||
minimal: {
|
||||
allow: ['system.info'],
|
||||
deny: []
|
||||
},
|
||||
messaging: {
|
||||
allow: ['system.info', 'memory.read', 'memory.write'],
|
||||
deny: ['shell.*', 'file.*', 'process.*']
|
||||
},
|
||||
coding: {
|
||||
allow: ['*'],
|
||||
deny: ['group:runtime']
|
||||
},
|
||||
full: {
|
||||
allow: ['*'],
|
||||
deny: []
|
||||
}
|
||||
};
|
||||
```
|
||||
Flynn ships 4 built-in profiles:
|
||||
|
||||
- `minimal`: read-only (file read/list + web.fetch + system.info)
|
||||
- `messaging`: read-only + web search + memory + connected read APIs (gmail/gcal/gdocs/gdrive/gtasks)
|
||||
- `coding`: adds filesystem writes, shell/process, and browser automation
|
||||
- `full`: all registered tools
|
||||
|
||||
The authoritative profile tool sets live in `src/tools/policy.ts`.
|
||||
|
||||
### Groups
|
||||
|
||||
@@ -471,39 +462,53 @@ Tools are organized into groups:
|
||||
- `group:web`: Web and browser tools
|
||||
- `group:memory`: Memory and search tools
|
||||
|
||||
There are additional groups for specific integrations (gmail/gcal/gdocs/gdrive/gtasks/cron). See `TOOL_GROUPS` in `src/tools/policy.ts`.
|
||||
|
||||
### Policy Resolution
|
||||
|
||||
When listing tools for an agent:
|
||||
When resolving tools for an execution context:
|
||||
|
||||
1. Start with profile's allow list
|
||||
2. Remove tools in deny list
|
||||
3. Apply per-agent overrides
|
||||
4. Apply per-provider overrides
|
||||
5. Apply hook patterns (confirm/log/silent)
|
||||
1. Start with global `tools.profile`
|
||||
2. Apply global `tools.allow` (adds tools back in)
|
||||
3. Apply global `tools.deny` (deny always wins)
|
||||
4. If `context.agent` override exists, intersect with agent override resolution
|
||||
5. If `context.provider` override exists, intersect with provider override resolution
|
||||
6. If `context.skillName` is set, intersect with skill capability allowlist (deny-by-default for skills)
|
||||
|
||||
Hooks/autonomy are enforced at execution-time (ToolExecutor), not during list resolution.
|
||||
|
||||
### Example Policy Config
|
||||
|
||||
```yaml
|
||||
tools:
|
||||
policy: 'coding' # Default profile
|
||||
|
||||
profiles:
|
||||
coding:
|
||||
allow: ['*']
|
||||
deny: ['group:runtime']
|
||||
profile: messaging
|
||||
allow: []
|
||||
deny: ["browser.*"]
|
||||
|
||||
# Per-agent overrides
|
||||
agents:
|
||||
my-agent:
|
||||
toolPolicy: 'full'
|
||||
fast:
|
||||
profile: minimal
|
||||
allow: []
|
||||
deny: []
|
||||
|
||||
# Per-provider overrides
|
||||
providers:
|
||||
anthropic:
|
||||
allow: ['*']
|
||||
deny: []
|
||||
ollama:
|
||||
profile: messaging
|
||||
allow: []
|
||||
deny: ["web.search"]
|
||||
```
|
||||
|
||||
### Skill Capabilities (Skill Context)
|
||||
|
||||
If a request is routed into a skill context (via intents), Flynn applies an additional restriction layer using the skill's `manifest.json.permissions`.
|
||||
|
||||
- A skill with no `permissions` manifest has no tool access.
|
||||
- `permissions.tools` (explicit allowlist) overrides `permissions.tool_groups`.
|
||||
|
||||
See `docs/security/SAFE_PERSONAL_AGENT.md`.
|
||||
|
||||
## Tool Execution Flow
|
||||
|
||||
### Execution Pipeline
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
# Agent-Oriented Project Diagram
|
||||
|
||||
This is a high-signal, agent-oriented view of Flynn's structure and execution flow.
|
||||
|
||||
If you're new to the codebase, start here, then jump to the referenced files.
|
||||
|
||||
## Big Picture (Runtime Data Flow)
|
||||
|
||||
```text
|
||||
Inbound Message
|
||||
(Telegram/Discord/Slack/WhatsApp/WebChat)
|
||||
|
|
||||
v
|
||||
ChannelAdapter -> ChannelRegistry
|
||||
| |
|
||||
| v
|
||||
| createMessageRouter()
|
||||
| |
|
||||
| v
|
||||
| SessionManager
|
||||
| |
|
||||
| v
|
||||
| AgentOrchestrator
|
||||
| |
|
||||
| v
|
||||
| NativeAgent
|
||||
| |
|
||||
| ModelRouter.chat()
|
||||
| |
|
||||
| v
|
||||
| ModelClient
|
||||
|
|
||||
+----> (optional) PairingManager gate for unknown senders
|
||||
|
||||
Tool Calls (inside NativeAgent loop)
|
||||
NativeAgent -> ToolRegistry (policy-filtered) -> ToolExecutor
|
||||
| |
|
||||
| v
|
||||
| HookEngine + autonomy
|
||||
| |
|
||||
| v
|
||||
| Tool.execute()
|
||||
| |
|
||||
| v
|
||||
+---------------------------> AuditLogger (redacted)
|
||||
|
||||
Outbound Reply
|
||||
-> ChannelAdapter.send() (text + optional attachments)
|
||||
```
|
||||
|
||||
Key files:
|
||||
|
||||
- Routing + per-session agent creation: `src/daemon/routing.ts`
|
||||
- Orchestration: `src/backends/native/orchestrator.ts`
|
||||
- Tool loop: `src/backends/native/agent.ts`
|
||||
- Model routing: `src/models/router.ts`
|
||||
- Tool policy + execution: `src/tools/policy.ts`, `src/tools/executor.ts`
|
||||
|
||||
## Component Graph (Agent-Safety Boundary)
|
||||
|
||||
```text
|
||||
+---------------------------+
|
||||
| Config |
|
||||
| (Zod schema + YAML) |
|
||||
| src/config/schema.ts |
|
||||
+-------------+-------------+
|
||||
|
|
||||
v
|
||||
+-------------------+ +-------------+ +------------------+
|
||||
| SkillRegistry | | ToolPolicy | | HookEngine |
|
||||
| src/skills/* | | src/tools/* | | src/hooks/* |
|
||||
+---------+---------+ +------+------+ +---------+--------+
|
||||
| | |
|
||||
| (system prompt) | (allow/deny) | (confirm/log/silent)
|
||||
v v v
|
||||
+-------------------+ +-------------+ +------------------+
|
||||
| System Prompt | | ToolRegistry| | ToolExecutor |
|
||||
| src/daemon/services.ts| src/tools/* | | src/tools/executor.ts
|
||||
+---------+---------+ +------+------+ +---------+--------+
|
||||
| | |
|
||||
v | |
|
||||
+-------------------+ | v
|
||||
| AgentOrchestrator | | +-----------+
|
||||
| src/backends/* | +------------> | AuditLogger|
|
||||
+---------+---------+ | src/audit/*|
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| NativeAgent |
|
||||
| src/backends/* |
|
||||
+---------+---------+
|
||||
|
|
||||
v
|
||||
+-------------------+
|
||||
| ModelRouter |
|
||||
| src/models/* |
|
||||
+-------------------+
|
||||
```
|
||||
|
||||
## Skills + Capabilities (What Gets Enforced)
|
||||
|
||||
Skills are local directories with:
|
||||
|
||||
- `SKILL.md` (instructions injected into the system prompt)
|
||||
- `manifest.json` (metadata + optional `permissions`)
|
||||
|
||||
### Skill permissions enforcement points
|
||||
|
||||
- Tool availability: `ToolPolicy.resolveAllowedNames()` intersects allowed tools with `manifest.json.permissions`.
|
||||
- Tool execution (defense in depth): `ToolExecutor.execute()` enforces:
|
||||
- fs allowlists (`permissions.fs.read` / `permissions.fs.write`)
|
||||
- net allowlists (best-effort for `web.fetch`)
|
||||
- secret scopes (tools declare `requiredSecretScopes`, skills allow `permissions.secrets`)
|
||||
- injection guard when untrusted content is present
|
||||
|
||||
Important default:
|
||||
|
||||
- If a request is routed into a skill context but the skill has no `permissions` manifest, **tool access is denied**.
|
||||
|
||||
Key files:
|
||||
|
||||
- Skill manifest types: `src/skills/types.ts`
|
||||
- Loader validation: `src/skills/loader.ts`
|
||||
- Policy intersection: `src/tools/policy.ts`
|
||||
- Executor enforcement: `src/tools/executor.ts`
|
||||
|
||||
## Sandbox Execution (High-Risk Tools)
|
||||
|
||||
Flynn supports per-session Docker sandboxes.
|
||||
|
||||
Where sandboxing is applied today:
|
||||
|
||||
- `shell.exec` and `process.start` can be replaced with sandboxed implementations.
|
||||
- Replacement is wired in `src/daemon/routing.ts` by cloning the ToolRegistry and swapping the tool implementations.
|
||||
|
||||
Skill context default:
|
||||
|
||||
- High-risk tool execution defaults to `sandbox` in skill context (when available).
|
||||
- A skill can opt into host execution only by setting `permissions.execution_environment: "host"`.
|
||||
|
||||
Key files:
|
||||
|
||||
- Sandbox lifecycle: `src/sandbox/manager.ts`, `src/sandbox/docker.ts`
|
||||
- Sandboxed tool wrappers: `src/sandbox/tools.ts`
|
||||
- Wiring: `src/daemon/routing.ts`
|
||||
|
||||
## Prompt Injection Hardening (Practical)
|
||||
|
||||
Flynn treats content provenance as part of the control boundary:
|
||||
|
||||
- `web.fetch`, `web.search`, and `browser.content` outputs are treated as untrusted "fetched_content".
|
||||
- Tool results are wrapped in provenance markers inside the tool loop.
|
||||
- Once untrusted content is seen, ToolExecutor applies stricter gating (blocks obvious injection patterns for high-risk tools).
|
||||
|
||||
Key files:
|
||||
|
||||
- Provenance wrapping: `src/backends/native/agent.ts`
|
||||
- Tool-call guard: `src/tools/executor.ts`
|
||||
- System prompt safety guidance: `src/daemon/services.ts`
|
||||
|
||||
## Mermaid (For Fast Visual Scanning)
|
||||
|
||||
If your renderer supports Mermaid, this is the same information as a sequence diagram.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant U as User
|
||||
participant CA as ChannelAdapter
|
||||
participant CR as ChannelRegistry
|
||||
participant SM as SessionManager
|
||||
participant AR as AgentOrchestrator
|
||||
participant NA as NativeAgent
|
||||
participant MR as ModelRouter
|
||||
participant MC as ModelClient
|
||||
participant TP as ToolPolicy/Registry
|
||||
participant TE as ToolExecutor
|
||||
participant HE as HookEngine
|
||||
participant AL as AuditLogger
|
||||
|
||||
U->>CA: message
|
||||
CA->>CR: onMessage(InboundMessage)
|
||||
CR->>SM: getSession(channel, sender)
|
||||
SM-->>CR: Session
|
||||
CR->>AR: getOrCreateAgent(session + routing)
|
||||
AR->>NA: process(userMessage)
|
||||
NA->>MR: chat(messages + tools)
|
||||
MR->>MC: provider request
|
||||
MC-->>MR: response (content or tool_calls)
|
||||
MR-->>NA: ChatResponse
|
||||
|
||||
alt model requests tool use
|
||||
NA->>TP: filtered tool list (skill + policy)
|
||||
NA->>TE: execute(tool, args, context)
|
||||
TE->>HE: confirm/log/silent (autonomy)
|
||||
HE-->>TE: approved/denied
|
||||
TE->>AL: audit (redacted)
|
||||
TE-->>NA: ToolResult
|
||||
NA->>MR: chat(tool_result blocks)
|
||||
end
|
||||
|
||||
NA-->>AR: assistant response
|
||||
AR-->>CR: OutboundMessage
|
||||
CR-->>CA: send()
|
||||
CA-->>U: reply
|
||||
```
|
||||
@@ -0,0 +1,250 @@
|
||||
# Contributor Map (Agent-Oriented)
|
||||
|
||||
This is a fast navigation guide for contributors (human or AI). It answers:
|
||||
|
||||
- Where do I add a new tool?
|
||||
- Where do I add a new skill?
|
||||
- Where do I change routing/policy?
|
||||
- What tests should I run?
|
||||
|
||||
For the execution-flow diagram, see `docs/architecture/AGENT_DIAGRAM.md`.
|
||||
|
||||
## 30-Second Repo Tour
|
||||
|
||||
```text
|
||||
src/
|
||||
daemon/ Start-up wiring, service init, message routing
|
||||
backends/ Native agent + orchestrator (tool loop lives here)
|
||||
tools/ Tool interfaces, policy, executor, builtins
|
||||
skills/ Skill loader/registry + install/watch infra
|
||||
hooks/ Confirm/log/silent policy + autonomy resolution
|
||||
sandbox/ Docker sandbox manager + sandboxed tool wrappers
|
||||
models/ Provider clients + model router + retry/cost/capabilities
|
||||
channels/ Chat adapters + pairing gate
|
||||
gateway/ WebSocket JSON-RPC server + web UI + handlers
|
||||
memory/ Hybrid search + embeddings + persistence
|
||||
session/ SQLite store + session mgmt
|
||||
cli/ CLI entrypoints + setup wizard
|
||||
automation/ Cron/webhooks/heartbeat/gmail watcher
|
||||
docs/
|
||||
api/ Tool and gateway protocol docs
|
||||
security/ Capability model, sandboxing, injection resistance
|
||||
architecture/ Diagrams + contributor maps
|
||||
config/
|
||||
default.yaml Example configuration
|
||||
```
|
||||
|
||||
## Adding a New Tool
|
||||
|
||||
### Where code goes
|
||||
|
||||
- Builtins live in `src/tools/builtin/`.
|
||||
- Core types live in `src/tools/types.ts`.
|
||||
- Tools are registered through the daemon wiring (see existing patterns in `src/daemon/index.ts`).
|
||||
|
||||
### Minimal tool skeleton
|
||||
|
||||
```ts
|
||||
import type { Tool, ToolResult } from '../types.js';
|
||||
|
||||
export const myTool: Tool = {
|
||||
name: 'my.tool',
|
||||
description: 'What it does (model-facing).',
|
||||
// Optional: gate credentialed actions
|
||||
requiredSecretScopes: ['my_scope'],
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
foo: { type: 'string', description: '...' },
|
||||
},
|
||||
required: ['foo'],
|
||||
},
|
||||
execute: async (rawArgs: unknown): Promise<ToolResult> => {
|
||||
// ...
|
||||
return { success: true, output: 'ok' };
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
### Security checklist
|
||||
|
||||
- If the tool calls an external service or uses credentials:
|
||||
- set `requiredSecretScopes` on the tool.
|
||||
- ensure skill permissions can gate it (`manifest.json.permissions.secrets`).
|
||||
- If it reads/writes files:
|
||||
- use `file.*` tools rather than bespoke FS access.
|
||||
- skills can restrict FS paths via `permissions.fs`.
|
||||
|
||||
### Tests to add
|
||||
|
||||
- Unit tests in `src/tools/builtin/<tool>.test.ts`.
|
||||
- If you touch policy/executor logic: add tests in `src/tools/policy.test.ts` or `src/tools/executor.test.ts`.
|
||||
|
||||
## Adding a New Skill
|
||||
|
||||
### What a skill is
|
||||
|
||||
A skill is a package with:
|
||||
|
||||
- `SKILL.md`: instructions injected into the system prompt.
|
||||
- `manifest.json`: metadata + capability declarations.
|
||||
|
||||
Where skills live:
|
||||
|
||||
- Bundled skills: `skills/`
|
||||
- Managed skills (installed by Flynn): configured skill directory (see config)
|
||||
|
||||
Skill loading:
|
||||
|
||||
- Loader: `src/skills/loader.ts`
|
||||
- Registry: `src/skills/registry.ts`
|
||||
- Watcher (optional): `src/skills/watcher.ts`
|
||||
|
||||
### Capability permissions
|
||||
|
||||
If the skill is used via routing (intent target type `skill`), add `permissions` to `manifest.json`.
|
||||
|
||||
Without `permissions`, a skill is still loadable, but in skill context it has no tool access.
|
||||
|
||||
Reference: `docs/security/SAFE_PERSONAL_AGENT.md`.
|
||||
|
||||
## Routing: Agents vs Skills vs Default
|
||||
|
||||
Where routing decisions happen:
|
||||
|
||||
- Inbound routing: `src/daemon/routing.ts`
|
||||
|
||||
Inputs to routing:
|
||||
|
||||
- Channel + sender (agent router)
|
||||
- Intent registry (regex rules) — can target `agent` or `skill`
|
||||
- Metadata overrides (gateway / channel adapters)
|
||||
|
||||
If you need a new routing rule type:
|
||||
|
||||
- Intent targets live in the intent registry/types (see `src/intents/registry.ts`).
|
||||
|
||||
## Tool Policy + Execution
|
||||
|
||||
You will usually touch these files for capability/security work:
|
||||
|
||||
- Tool allowlisting: `src/tools/policy.ts`
|
||||
- Tool runtime enforcement + audit: `src/tools/executor.ts`
|
||||
- Confirmation/autonomy: `src/hooks/engine.ts`, `src/hooks/autonomy.ts`
|
||||
|
||||
In skill context:
|
||||
|
||||
- `ToolPolicyContext.skillName` and `.skillPermissions` are set in `src/daemon/routing.ts`.
|
||||
- ToolPolicy filters available tools.
|
||||
- ToolExecutor enforces fs/net/secret/injection restrictions even if a tool is somehow called.
|
||||
|
||||
## Sandbox
|
||||
|
||||
Sandbox components:
|
||||
|
||||
- Docker sandbox manager: `src/sandbox/manager.ts`
|
||||
- Docker implementation: `src/sandbox/docker.ts`
|
||||
- Sandboxed tool wrappers: `src/sandbox/tools.ts`
|
||||
- Tool replacement wiring: `src/daemon/routing.ts`
|
||||
|
||||
Notes:
|
||||
|
||||
- Today the sandbox wiring replaces `shell.exec` and `process.start` when sandbox is enabled.
|
||||
- In skill context, high-risk execution defaults to sandbox unless the skill opts into host execution.
|
||||
|
||||
## Gateway / API Surface
|
||||
|
||||
Gateway protocol docs:
|
||||
|
||||
- `docs/api/PROTOCOL.md`
|
||||
|
||||
Gateway handlers:
|
||||
|
||||
- `src/gateway/handlers/` (JSON-RPC methods)
|
||||
|
||||
Useful places to start:
|
||||
|
||||
- `src/gateway/server.ts` (server lifecycle)
|
||||
- `src/gateway/protocol.ts` (types)
|
||||
|
||||
## Tests + Commands
|
||||
|
||||
Common checks:
|
||||
|
||||
```bash
|
||||
pnpm typecheck
|
||||
pnpm lint
|
||||
pnpm test:run
|
||||
```
|
||||
|
||||
Targeted tests for safety boundary changes:
|
||||
|
||||
- Tool policy: `pnpm test:run src/tools/policy.test.ts`
|
||||
- Tool executor: `pnpm test:run src/tools/executor.test.ts`
|
||||
- Skill loader: `pnpm test:run src/skills/loader.test.ts`
|
||||
- Routing: `pnpm test:run src/daemon/routing.test.ts`
|
||||
|
||||
## First 3 PRs to Pick Up (Good Agent On-Ramps)
|
||||
|
||||
These are small, high-leverage changes that teach you the architecture quickly.
|
||||
|
||||
### PR 1: Add a new "narrow" skill + permissions
|
||||
|
||||
Goal: add a skill that can only do one bounded thing (example: summarize a URL).
|
||||
|
||||
Deliverables:
|
||||
|
||||
- `skills/url-summarizer/SKILL.md`
|
||||
- `skills/url-summarizer/manifest.json` with permissions:
|
||||
- `tool_groups: ["group:web"]`
|
||||
- `net: [{"host":"*","ports":[443]}]` (or narrower if you prefer)
|
||||
- `execution_environment: "sandbox"` (default)
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Skill loads (`flynn doctor` / skills list)
|
||||
- In skill context, `shell.exec` is not available
|
||||
- `web.fetch` works for https URLs
|
||||
|
||||
### PR 2: Route into the skill via intents
|
||||
|
||||
Goal: make it easy to invoke the skill without special UI.
|
||||
|
||||
Deliverables:
|
||||
|
||||
- Add an `intents.rules[]` entry targeting `type: skill`
|
||||
- Patterns like: `summarize *`, `tldr *`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- A message like `summarize https://example.com` routes to the skill
|
||||
- Tool list is capability-filtered for that skill context
|
||||
|
||||
### PR 3: Add an end-to-end safety test
|
||||
|
||||
Goal: lock in behavior so future refactors don’t weaken the boundary.
|
||||
|
||||
Deliverables:
|
||||
|
||||
- A test that asserts: when routed to a skill context with web-only permissions:
|
||||
- `ToolPolicy` excludes `shell.exec` and `file.write`
|
||||
- `ToolExecutor` denies a direct attempt to call `file.write` outside allowed fs globs
|
||||
|
||||
Suggested test locations:
|
||||
|
||||
- `src/tools/policy.test.ts`
|
||||
- `src/tools/executor.test.ts`
|
||||
|
||||
## Where to Add What (Cheat Sheet)
|
||||
|
||||
```text
|
||||
New tool .................. src/tools/builtin/ + register in daemon
|
||||
Tool allow/deny logic ...... src/tools/policy.ts
|
||||
Tool runtime enforcement .... src/tools/executor.ts
|
||||
New skill .................. skills/<name>/{SKILL.md,manifest.json}
|
||||
Skill loader/validation ..... src/skills/loader.ts
|
||||
Skill routing (intents) ..... src/daemon/routing.ts + config intents
|
||||
Sandbox behavior ........... src/sandbox/* + src/daemon/routing.ts
|
||||
Confirmation UX ............ src/hooks/* + frontends/gateway
|
||||
Web UI changes ............. src/gateway/ui/
|
||||
```
|
||||
@@ -0,0 +1,143 @@
|
||||
# Symbol Index (Agent Quick-Jump)
|
||||
|
||||
This is a curated index of the most important exported types and functions, organized for fast navigation.
|
||||
|
||||
It is intentionally short: if something isn't here, it's probably not a primary control surface.
|
||||
|
||||
See also:
|
||||
|
||||
- `docs/architecture/TYPESCRIPT_MAP.md` (conceptual map + diagrams)
|
||||
- `docs/architecture/AGENT_DIAGRAM.md` (runtime flow)
|
||||
|
||||
## Daemon Entry Points
|
||||
|
||||
- `src/daemon/index.ts`
|
||||
- Creates/wires: config, tool registry, tool executor, skill registry, model router, gateway, channels.
|
||||
|
||||
- `src/daemon/routing.ts`
|
||||
- `createMessageRouter(deps)`
|
||||
- Main router factory used by channel adapters + gateway.
|
||||
|
||||
## Routing / Intents / Agents
|
||||
|
||||
- `src/agents/router.ts`
|
||||
- `AgentRouter.resolve(channel, senderId)`
|
||||
- Picks an agent config for a sender/channel.
|
||||
|
||||
- `src/agents/registry.ts`
|
||||
- `AgentConfigRegistry.get(name)`
|
||||
- Loads agent configs.
|
||||
|
||||
- `src/intents/registry.ts`
|
||||
- `IntentRegistry.match(text)`
|
||||
- Matches text to intent rules (targets: agent or skill).
|
||||
|
||||
## Native Agent Loop
|
||||
|
||||
- `src/backends/native/orchestrator.ts`
|
||||
- `AgentOrchestrator.process(message, options)`
|
||||
- Top-level entry for “run agent on this message”.
|
||||
|
||||
- `src/backends/native/agent.ts`
|
||||
- `NativeAgent` (class)
|
||||
- Internal hot path: tool loop that:
|
||||
- asks model
|
||||
- executes tools
|
||||
- returns tool results
|
||||
- repeats
|
||||
|
||||
## Models
|
||||
|
||||
- `src/models/router.ts`
|
||||
- `ModelRouter.chat(request)`
|
||||
- Chooses tier/provider fallback chain.
|
||||
|
||||
- `src/models/types.ts`
|
||||
- Core request/response types shared by providers.
|
||||
|
||||
## Tools
|
||||
|
||||
- `src/tools/types.ts`
|
||||
- `Tool`
|
||||
- `ToolResult`
|
||||
|
||||
- `src/tools/registry.ts`
|
||||
- `ToolRegistry.register(tool)`
|
||||
- `ToolRegistry.list()`
|
||||
- `ToolRegistry.clone()` / `ToolRegistry.replace(tool)` (used for sandbox substitution)
|
||||
|
||||
- `src/tools/policy.ts`
|
||||
- `ToolPolicy` (class)
|
||||
- `ToolPolicy.resolveAllowedNames(allToolNames, context)`
|
||||
- `ToolPolicyContext` (type)
|
||||
- `TOOL_GROUPS` (group expansion)
|
||||
|
||||
- `src/tools/executor.ts`
|
||||
- `ToolExecutor.execute(toolName, args, context)`
|
||||
- Central enforcement point:
|
||||
- policy allow/deny
|
||||
- hooks/autonomy confirmations
|
||||
- skill fs/net/secret constraints
|
||||
- untrusted-content injection guard
|
||||
- audit events w/ redaction
|
||||
|
||||
## Skills
|
||||
|
||||
- `src/skills/types.ts`
|
||||
- `SkillManifest`
|
||||
- `SkillPermissions`
|
||||
|
||||
- `src/skills/loader.ts`
|
||||
- `loadSkill(dir, tier)`
|
||||
- `loadAllSkills(...)`
|
||||
- Validates manifests.
|
||||
|
||||
- `src/skills/registry.ts`
|
||||
- `SkillRegistry.get(name)`
|
||||
- `SkillRegistry.getSystemPromptAdditions()`
|
||||
|
||||
## Hooks / Approval
|
||||
|
||||
- `src/hooks/engine.ts`
|
||||
- `HookEngine.getAction(toolName)`
|
||||
- `HookEngine.requestConfirmation(toolName, args)`
|
||||
|
||||
- `src/hooks/autonomy.ts`
|
||||
- `resolveAutonomy(toolName, baseAction, autonomyLevel)`
|
||||
|
||||
## Sandbox
|
||||
|
||||
- `src/sandbox/manager.ts`
|
||||
- Manages per-session sandbox lifecycle.
|
||||
|
||||
- `src/sandbox/tools.ts`
|
||||
- Creates sandboxed tool implementations for `shell.exec` / `process.start`.
|
||||
|
||||
- `src/sandbox/docker.ts`
|
||||
- Docker-specific implementation.
|
||||
|
||||
## Audit
|
||||
|
||||
- `src/audit/index.ts`
|
||||
- `auditLogger` singleton.
|
||||
|
||||
- `src/audit/types.ts`
|
||||
- Event types (`tool.start`, `tool.success`, `tool.denied`, `tool.approval`, ...)
|
||||
|
||||
- `src/audit/logger.ts`
|
||||
- `AuditLogger` methods: `toolStart`, `toolDenied`, `toolApproval`, ...
|
||||
|
||||
- `src/audit/redact.ts`
|
||||
- `redactForAudit(value)`
|
||||
|
||||
## Gateway
|
||||
|
||||
- `src/gateway/server.ts`
|
||||
- WebSocket server lifecycle.
|
||||
|
||||
- `src/gateway/handlers/*`
|
||||
- JSON-RPC methods grouped by area.
|
||||
|
||||
Protocol:
|
||||
|
||||
- `docs/api/PROTOCOL.md`
|
||||
@@ -0,0 +1,186 @@
|
||||
# TypeScript Map (Types + Hot Functions)
|
||||
|
||||
This doc is optimized for AI agents: it names the core TypeScript types and the handful of functions/methods that actually control behavior.
|
||||
|
||||
For runtime flow diagrams, see:
|
||||
|
||||
- `docs/architecture/AGENT_DIAGRAM.md`
|
||||
- `docs/architecture/CONTRIBUTOR_MAP.md`
|
||||
|
||||
## Core Domain Types (What Matters)
|
||||
|
||||
### Messages
|
||||
|
||||
- `InboundMessage` / `OutboundMessage`
|
||||
- Used by channel adapters and the message router.
|
||||
- Source: `src/channels/types.ts`, `src/daemon/routing.ts`
|
||||
|
||||
### Tools
|
||||
|
||||
- `Tool`
|
||||
- A single capability callable by the model.
|
||||
- Source: `src/tools/types.ts`
|
||||
|
||||
- `ToolResult`
|
||||
- Return value from a tool.
|
||||
- Source: `src/tools/types.ts`
|
||||
|
||||
- `ToolPolicyContext`
|
||||
- Dynamic context used to decide tool availability and enforcement.
|
||||
- Source: `src/tools/policy.ts`
|
||||
|
||||
Key fields to know:
|
||||
|
||||
- `agent`, `provider`, `autonomyLevel`
|
||||
- `skillName`, `skillPermissions`
|
||||
- `executionEnvironment` (`host` or `sandbox`)
|
||||
- `untrustedContent` (tightens guards after fetched content appears)
|
||||
|
||||
### Skills
|
||||
|
||||
- `SkillManifest`
|
||||
- `manifest.json` parsed and validated.
|
||||
- Source: `src/skills/types.ts`
|
||||
|
||||
- `SkillPermissions`
|
||||
- Capability declarations that get enforced at runtime.
|
||||
- Source: `src/skills/types.ts`
|
||||
|
||||
### Audit
|
||||
|
||||
- `AuditEventType` and tool events (`tool.start`, `tool.success`, `tool.denied`, `tool.approval`)
|
||||
- Source: `src/audit/types.ts`
|
||||
|
||||
## Hot Functions / Methods (Where Behavior Lives)
|
||||
|
||||
If you only read 10 definitions, read these:
|
||||
|
||||
- `createMessageRouter()`
|
||||
- Routes inbound messages, resolves intent targets (agent vs skill).
|
||||
- File: `src/daemon/routing.ts`
|
||||
|
||||
- `getOrCreateAgent()` (inner helper)
|
||||
- Builds per-session `AgentOrchestrator` and sets `toolPolicyContext`.
|
||||
- File: `src/daemon/routing.ts`
|
||||
|
||||
- `AgentOrchestrator.process()`
|
||||
- Runs the agent loop and streams output.
|
||||
- File: `src/backends/native/orchestrator.ts`
|
||||
|
||||
- `NativeAgent.toolLoop()`
|
||||
- The core loop: model -> tool calls -> tool results -> model -> final response.
|
||||
- Adds provenance markers to tool results.
|
||||
- File: `src/backends/native/agent.ts`
|
||||
|
||||
- `ToolPolicy.resolveAllowedNames()`
|
||||
- Computes the available tool set for a given context.
|
||||
- Enforces skill capability intersection (deny-by-default for skill context).
|
||||
- File: `src/tools/policy.ts`
|
||||
|
||||
- `ToolExecutor.execute()`
|
||||
- Defense-in-depth enforcement + hooks/autonomy + auditing.
|
||||
- Enforces skill fs/net/secret scope + injection guard.
|
||||
- File: `src/tools/executor.ts`
|
||||
|
||||
## Diagram: Key Types (Mermaid)
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class Tool {
|
||||
+string name
|
||||
+string description
|
||||
+JSONSchema inputSchema
|
||||
+string[]? requiredSecretScopes
|
||||
+execute(args): Promise~ToolResult~
|
||||
}
|
||||
|
||||
class ToolResult {
|
||||
+boolean success
|
||||
+string output
|
||||
+string? error
|
||||
}
|
||||
|
||||
class ToolPolicyContext {
|
||||
+string? agent
|
||||
+string? provider
|
||||
+string? autonomyLevel
|
||||
+string? sessionId
|
||||
+string? channel
|
||||
+string? sender
|
||||
+string? tier
|
||||
+string? skillName
|
||||
+SkillPermissions? skillPermissions
|
||||
+string? executionEnvironment
|
||||
+boolean? untrustedContent
|
||||
+string[]? allowedSecretScopes
|
||||
}
|
||||
|
||||
class SkillManifest {
|
||||
+string name
|
||||
+string description
|
||||
+string version
|
||||
+string tier
|
||||
+SkillPermissions? permissions
|
||||
}
|
||||
|
||||
class SkillPermissions {
|
||||
+string[]? tool_groups
|
||||
+string[]? tools
|
||||
+SkillFsPermissions? fs
|
||||
+SkillNetPermission[]? net
|
||||
+string[]? secrets
|
||||
+string? execution_environment
|
||||
}
|
||||
|
||||
class AuditToolEvent {
|
||||
+string tool_name
|
||||
+string? execution_id
|
||||
+string? execution_environment
|
||||
+string? skill_name
|
||||
+number? redactions_applied
|
||||
}
|
||||
|
||||
Tool --> ToolResult
|
||||
ToolPolicyContext --> SkillPermissions
|
||||
SkillManifest --> SkillPermissions
|
||||
```
|
||||
|
||||
## Diagram: Control Flow (Tool Call Path)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Model proposes tool call] --> B[ToolPolicy filters allowed tools]
|
||||
B --> C[ToolExecutor.execute]
|
||||
C --> D{Allowed by policy?}
|
||||
D -- no --> X[Denied + audit]
|
||||
D -- yes --> E{Hooks/autonomy confirm?}
|
||||
E -- denied --> X
|
||||
E -- approved --> F{Skill constraints}
|
||||
F -- violation --> X
|
||||
F -- ok --> G{Untrusted content guard}
|
||||
G -- blocked --> X
|
||||
G -- ok --> H[Tool.execute]
|
||||
H --> I[Audit (redacted)]
|
||||
I --> J[ToolResult returned to model]
|
||||
```
|
||||
|
||||
## Diagram: Module Entry Points
|
||||
|
||||
These are the places you typically jump to first.
|
||||
|
||||
```text
|
||||
src/daemon/index.ts
|
||||
- wires together: config + skillRegistry + toolRegistry + toolExecutor + router
|
||||
|
||||
src/daemon/routing.ts
|
||||
- inbound routing + intent match + per-session agent construction
|
||||
|
||||
src/backends/native/agent.ts
|
||||
- tool loop (the actual "agent")
|
||||
|
||||
src/tools/policy.ts
|
||||
- tool allow/deny resolution (+ skill capability intersection)
|
||||
|
||||
src/tools/executor.ts
|
||||
- enforcement + hooks + auditing
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,253 @@
|
||||
# Credential System v2 (API + OAuth/token) — Implementation Checklist
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Parent roadmap:** `docs/plans/2026-02-15-openclaw-gap-roadmap.md`
|
||||
|
||||
**Goal:** Close the gap item "OAuth subscription auth" by supporting **both** API-key credentials and OAuth/token-based credentials (provider-specific) with consistent UX, per-tier control, and deterministic resolution.
|
||||
|
||||
## Scope Summary
|
||||
|
||||
- Add `auth_mode` **per tier** (default/fast/complex/local and any `local_providers` entries).
|
||||
- Keep backward compatibility with existing `use_oauth` behavior.
|
||||
- Add stored credential support where it currently doesn't exist:
|
||||
- OpenAI: stored API key (OAuth already exists)
|
||||
- Anthropic: stored auth token (API key already exists)
|
||||
- Improve doctor output to surface which auth sources are present (without revealing secrets).
|
||||
|
||||
Non-goals (explicitly out of scope for this checklist):
|
||||
|
||||
- Inventing an Anthropic OAuth device flow.
|
||||
- Building new provider integrations (Vercel/MiniMax/etc.).
|
||||
|
||||
## Current Baseline (important constraints)
|
||||
|
||||
- OpenAI OAuth uses the ChatGPT/Codex backend endpoint (SSE) and currently **does not support tools** on that path.
|
||||
- Source: `src/models/openai.ts`
|
||||
- Anthropic supports `apiKey` and `authToken` in `AnthropicClientConfig`.
|
||||
- Source: `src/models/anthropic.ts`
|
||||
- Stored credentials live at `~/.config/flynn/auth.json`.
|
||||
- Source: `src/auth/openai.ts`, `src/auth/anthropic.ts`
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### 1) New config field: `auth_mode`
|
||||
|
||||
Add `auth_mode` to the per-tier model config:
|
||||
|
||||
- `auto` (default)
|
||||
- `api_key`
|
||||
- `oauth`
|
||||
|
||||
`oauth` is interpreted as "OAuth/token mode" (provider-specific). For Anthropic, that means `auth_token`.
|
||||
|
||||
### 2) Backward compatibility: `use_oauth`
|
||||
|
||||
Preserve `use_oauth` as a compatibility alias.
|
||||
|
||||
Recommended rule:
|
||||
|
||||
- If `auth_mode` is set: it wins.
|
||||
- Else if `use_oauth: true`: treat as `auth_mode: oauth`.
|
||||
|
||||
### 3) Credential resolution order
|
||||
|
||||
For each provider, resolve the required credential type by trying:
|
||||
|
||||
1) config (`api_key` / `auth_token`)
|
||||
2) env var
|
||||
3) auth store (`~/.config/flynn/auth.json`)
|
||||
|
||||
`auth_mode` controls which credential type is required.
|
||||
|
||||
## PR Breakdown (atomic, test-backed)
|
||||
|
||||
### PR 1 — Schema + docs: per-tier `auth_mode`
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add `auth_mode` to `modelConfigBaseSchema` in `src/config/schema.ts`.
|
||||
- [ ] Update `src/config/schema.test.ts` to cover defaults + validation.
|
||||
- [ ] Update `README.md` config examples (brief mention).
|
||||
- [ ] Update `config/default.yaml` comment/help text (brief mention).
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Config parses with no changes (defaults preserved).
|
||||
- Setting `auth_mode: oauth` or `auth_mode: api_key` validates.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/config/schema.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 2 — OpenAI auth store: add API-key storage
|
||||
|
||||
Goal: allow OpenAI to run without `api_key` in YAML.
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `src/auth/openai.ts` `AuthStore` shape to allow `openai.api_key` alongside existing OAuth info.
|
||||
- [ ] Add functions:
|
||||
- [ ] `loadStoredOpenAIApiKey()`
|
||||
- [ ] `storeOpenAIApiKey(key)`
|
||||
- [ ] `clearOpenAIApiKey()`
|
||||
- [ ] `getOpenAIApiKey()` (env override + store)
|
||||
- [ ] Keep existing OAuth store code working unchanged.
|
||||
- [ ] Add/extend tests for new store functions.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/auth/openai.ts`
|
||||
- `src/auth/openai.test.ts` (or add if missing)
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Stored OpenAI API key is written to `~/.config/flynn/auth.json` with `0600` permissions.
|
||||
- OAuth entry remains backward compatible.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/auth/openai.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 3 — Anthropic auth store: add auth-token storage
|
||||
|
||||
Goal: allow `auth_token` to be stored and selected with `auth_mode: oauth`.
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `src/auth/anthropic.ts` auth store shape to include `auth_token`.
|
||||
- [ ] Add functions:
|
||||
- [ ] `loadStoredAnthropicAuthToken()`
|
||||
- [ ] `storeAnthropicAuthToken(token)`
|
||||
- [ ] `clearAnthropicAuthToken()`
|
||||
- [ ] `getAnthropicAuthToken()`
|
||||
- [ ] Extend `src/auth/anthropic.test.ts`.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/auth/anthropic.ts`
|
||||
- `src/auth/anthropic.test.ts`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `auth_token` can be stored and resolved without being present in YAML.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/auth/anthropic.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 4 — CLI commands for managing new stored credentials
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add `flynn openai-key` command (store API key in auth.json).
|
||||
- [ ] Extend `flynn anthropic-auth` to support storing either API key or auth token:
|
||||
- [ ] recommended: `flynn anthropic-auth --token` OR `flynn anthropic-token`
|
||||
- [ ] Update `src/cli/index.ts` registration.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/cli/openai-key.ts` (new)
|
||||
- `src/cli/anthropic-auth.ts` (modify)
|
||||
- `src/cli/index.ts`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- CLI can store credentials without printing them.
|
||||
- Re-running commands detects existing stored credentials and exits cleanly.
|
||||
|
||||
Tests:
|
||||
|
||||
- Add targeted unit tests if the CLI layer has existing patterns; otherwise validate via integration tests where feasible.
|
||||
|
||||
---
|
||||
|
||||
### PR 5 — TUI `/login` UX: OpenAI choice (OAuth vs API key) + Anthropic token
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Update `/login openai` in `src/frontends/tui/minimal.ts`:
|
||||
- [ ] Present a simple prompt: "1) OAuth device flow 2) Paste API key"
|
||||
- [ ] Store selected credential via auth store
|
||||
- [ ] Add `/login anthropic` in `src/frontends/tui/minimal.ts`:
|
||||
- [ ] "1) Paste API key 2) Paste auth token"
|
||||
- [ ] Keep existing `/login github` and `/login zai` behavior intact.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/frontends/tui/minimal.ts`
|
||||
- `src/frontends/tui/commands.ts` (if command parsing needs to accept new provider)
|
||||
|
||||
Acceptance:
|
||||
|
||||
- TUI can store OpenAI API key or OAuth token.
|
||||
- TUI can store Anthropic API key or auth token.
|
||||
|
||||
Tests:
|
||||
|
||||
- Add or extend minimal TUI tests as needed (existing suite patterns exist for model switching).
|
||||
|
||||
---
|
||||
|
||||
### PR 6 — Model factory: enforce `auth_mode` per tier
|
||||
|
||||
This is the core runtime change.
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Update `src/daemon/models.ts`:
|
||||
- [ ] Read `cfg.auth_mode` (or inferred from `use_oauth`) per tier.
|
||||
- [ ] For OpenAI:
|
||||
- [ ] `auth_mode=oauth`: configure `OpenAIClient({ useOAuth: true })` and verify OAuth tokens exist.
|
||||
- [ ] `auth_mode=api_key`: configure `OpenAIClient({ apiKey: resolvedKey })`.
|
||||
- [ ] For Anthropic:
|
||||
- [ ] `auth_mode=oauth`: require auth token (config/env/store).
|
||||
- [ ] `auth_mode=api_key`: require API key (config/env/store).
|
||||
- [ ] For other providers:
|
||||
- [ ] define behavior explicitly (likely `api_key` only unless provider already supports token-style auth).
|
||||
- [ ] Ensure error messages name the expected auth type and remediation.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/daemon/models.ts`
|
||||
- potentially `src/models/openai.ts` (if you decide to unify API key vs OAuth selection naming)
|
||||
|
||||
Tests:
|
||||
|
||||
- `src/daemon/clientFactory.test.ts`
|
||||
- [ ] auth_mode precedence over use_oauth
|
||||
- [ ] auto -> api key path
|
||||
- [ ] oauth -> token path
|
||||
- [ ] correct failures when missing
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Selecting `auth_mode` changes runtime behavior deterministically.
|
||||
|
||||
---
|
||||
|
||||
### PR 7 — Doctor: report auth source availability
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `checkModelConnectivity` in `src/cli/doctor.ts` to reflect `auth_mode`:
|
||||
- [ ] If `auth_mode=api_key`, warn/fail when API key is absent from config/env/store.
|
||||
- [ ] If `auth_mode=oauth`, warn/fail when OAuth/token is absent.
|
||||
- [ ] If `auth_mode=auto`, keep current behavior but improve messaging.
|
||||
- [ ] Add tests in `src/cli/doctor.test.ts`.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Doctor output tells user what to do next (command name + env var) without exposing secrets.
|
||||
|
||||
## Final Integration Checks
|
||||
|
||||
- [ ] `pnpm typecheck`
|
||||
- [ ] `pnpm test:run`
|
||||
- [ ] Update `docs/plans/state.json` entry for this checklist once implemented (status, summary, test status).
|
||||
@@ -0,0 +1,343 @@
|
||||
# OpenClaw Gap Roadmap (Flynn)
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Source:** `docs/plans/2026-02-06-openclaw-feature-gap-analysis.md`
|
||||
|
||||
**Goal:** Turn the remaining **MISSING** items into an executable roadmap with clear milestones, acceptance criteria, and test strategy.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
A gap item is considered implemented when:
|
||||
|
||||
- It is behind config flags where appropriate.
|
||||
- It has tests for core logic (unit/integration, plus minimal adapter/provider mocks where applicable).
|
||||
- It has docs (README or relevant docs section).
|
||||
- `docs/plans/state.json` is updated with status + summary + test status.
|
||||
|
||||
## Remaining Gap Inventory (from gap analysis)
|
||||
|
||||
### Channels / Frontends (MISSING)
|
||||
|
||||
- Signal
|
||||
- Matrix
|
||||
- Google Chat
|
||||
- Microsoft Teams
|
||||
- iMessage/BlueBubbles
|
||||
- Zalo
|
||||
- LINE/Feishu/Mattermost
|
||||
|
||||
### Model Providers (MISSING)
|
||||
|
||||
- MiniMax / Moonshot
|
||||
- Vercel AI Gateway
|
||||
- OAuth subscription auth (support both API key and OAuth/token)
|
||||
|
||||
### Agent Runtime / UX (MISSING)
|
||||
|
||||
- Canvas / A2UI (agent-driven visual workspace)
|
||||
|
||||
### Memory (MISSING)
|
||||
|
||||
- QMD backend (experimental)
|
||||
|
||||
### Security (MISSING)
|
||||
|
||||
- Skill/plugin code safety scanner (static analysis)
|
||||
- Elevated mode (explicit host-exec escape hatch)
|
||||
|
||||
### Skills Ecosystem (MISSING)
|
||||
|
||||
- ClawHub / community skill registry
|
||||
|
||||
### Gateway / Infra / Deployment (MISSING)
|
||||
|
||||
- Nix deployment
|
||||
- Fly.io / Railway / Render
|
||||
- Announce delivery mode (isolated job delivery)
|
||||
- Bonjour/mDNS discovery
|
||||
|
||||
### Misc (MISSING)
|
||||
|
||||
- Presence tracking
|
||||
|
||||
### Companion Apps / Devices (MISSING)
|
||||
|
||||
- macOS menu bar app
|
||||
- iOS node
|
||||
- Android node
|
||||
- Voice Wake / Talk Mode
|
||||
- Camera / screen capture
|
||||
- Location access
|
||||
|
||||
## Roadmap Overview (Milestones)
|
||||
|
||||
This roadmap optimizes for: (1) high leverage, (2) low coupling, (3) ability to ship incrementally.
|
||||
|
||||
1) Credential System v2 (API + OAuth/token) [P0]
|
||||
2) Vercel AI Gateway provider [P1]
|
||||
3) Skill/plugin safety scanner [P1]
|
||||
4) Elevated mode (break-glass) [P2]
|
||||
5) Matrix channel adapter [P2]
|
||||
6) Deployment targets (Nix + PaaS) [P3]
|
||||
|
||||
Everything else is explicitly deferred until there is a strong user need.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 1 (P0): Credential System v2 — API + OAuth/token
|
||||
|
||||
This closes the gap item "OAuth subscription auth" as: providers should support both API-key credentials and OAuth/token-based credentials (where available), with consistent UX and deterministic resolution order.
|
||||
|
||||
### Scope
|
||||
|
||||
- Add an explicit `auth_mode` selector per model tier config:
|
||||
- `auto` (default): try the most specific configured credential sources in priority order
|
||||
- `api_key`: require API key sources
|
||||
- `oauth`: require OAuth/token sources
|
||||
|
||||
Rationale: per-tier enables common setups like:
|
||||
|
||||
- `fast` uses an API key for deterministic reliability.
|
||||
- `default` uses OAuth/token for personal subscription accounts.
|
||||
|
||||
### Provider Behavior (initial)
|
||||
|
||||
- OpenAI:
|
||||
- Support OAuth (existing): device flow + stored token (`flynn openai-auth`, `/login openai`, `use_oauth: true`).
|
||||
- Add API key storage in the same auth store (so YAML can omit secrets).
|
||||
|
||||
- Anthropic:
|
||||
- Today supports: API key (config/env/auth store) and `auth_token` (config/env).
|
||||
- Add auth-token storage and allow selecting it via `auth_mode: oauth`.
|
||||
- Note: do not invent an OAuth device flow for Anthropic unless a real flow exists; "oauth" here means token-based auth.
|
||||
|
||||
### Design
|
||||
|
||||
Credential sources (recommended priority):
|
||||
|
||||
- Config (`api_key` / `auth_token`) -> Env var(s) -> Auth store (`~/.config/flynn/auth.json`)
|
||||
|
||||
Auth store should be able to hold multiple types per provider, e.g.:
|
||||
|
||||
```json
|
||||
{
|
||||
"openai": { "api_key": "...", "oauth": { "refresh_token": "...", "created_at": "..." } },
|
||||
"anthropic": { "api_key": "...", "auth_token": "..." }
|
||||
}
|
||||
```
|
||||
|
||||
### Implementation Tasks
|
||||
|
||||
Config/schema:
|
||||
|
||||
- Add `auth_mode` to the model config schema (per tier) in `src/config/schema.ts`.
|
||||
- Update schema tests in `src/config/schema.test.ts`.
|
||||
|
||||
Auth store:
|
||||
|
||||
- Extend `src/auth/openai.ts` to support API-key storage in auth.json (store/load/clear).
|
||||
- Extend `src/auth/anthropic.ts` to support auth-token storage in auth.json (store/load/clear).
|
||||
- Update `src/auth/index.ts` exports.
|
||||
|
||||
CLI:
|
||||
|
||||
- Add CLI command(s) for managing stored API keys for OpenAI (name TBD, recommended: `flynn openai-key`).
|
||||
- Extend Anthropic auth CLI to store either API key or auth token (flag or separate command).
|
||||
|
||||
TUI:
|
||||
|
||||
- Extend `/login openai` flow to let user choose OAuth device flow OR paste API key.
|
||||
- Add `/login anthropic` "paste API key" and "paste auth token" (non-OAuth).
|
||||
|
||||
Model factory:
|
||||
|
||||
- Update `createClientFromConfig()` in `src/daemon/models.ts` to resolve creds according to `auth_mode`.
|
||||
- Ensure error messages are explicit about which credential type is missing.
|
||||
|
||||
Doctor:
|
||||
|
||||
- Update `src/cli/doctor.ts` to report:
|
||||
- whether the provider has an API key source
|
||||
- whether OAuth/token sources exist
|
||||
- without revealing any secret material
|
||||
|
||||
### Tests
|
||||
|
||||
- `src/daemon/clientFactory.test.ts`
|
||||
- auth_mode=auto resolves API key when present
|
||||
- auth_mode=oauth resolves token when present
|
||||
- auth_mode=api_key fails when only OAuth exists (and vice versa)
|
||||
|
||||
- `src/auth/openai.test.ts` (or new) and `src/auth/anthropic.test.ts`
|
||||
- store/load/clear for all supported credential types
|
||||
|
||||
- `src/config/schema.test.ts`
|
||||
- config parsing + defaults for auth_mode
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- OpenAI can authenticate via either:
|
||||
- stored OAuth token, or
|
||||
- stored API key,
|
||||
without secrets in YAML.
|
||||
|
||||
- Anthropic can authenticate via either:
|
||||
- stored API key, or
|
||||
- stored auth token,
|
||||
without secrets in YAML.
|
||||
|
||||
- `/model` switching does not silently fall back to a weaker tier due to missing credentials; it returns an explicit error.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 2 (P1): Vercel AI Gateway Provider
|
||||
|
||||
### Scope
|
||||
|
||||
- Add a new model provider for Vercel AI Gateway.
|
||||
- If OpenAI-compatible, implement via `OpenAIClient` with a configurable `baseURL`.
|
||||
|
||||
### Implementation Tasks
|
||||
|
||||
- Add provider id in `src/config/schema.ts` (`MODEL_PROVIDERS`).
|
||||
- Add `createClientFromConfig()` case in `src/daemon/models.ts`.
|
||||
- Update setup wizard provider list (optional): `src/cli/setup/providers.ts`.
|
||||
- Update doctor key checks if needed: `src/cli/doctor.ts`.
|
||||
|
||||
### Tests
|
||||
|
||||
- `src/daemon/clientFactory.test.ts`: constructs correct client and passes `baseURL`.
|
||||
|
||||
### Acceptance
|
||||
|
||||
- Tool calls + streaming work through the gateway in at least one tier.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 3 (P1): Skill/Plugin Code Safety Scanner
|
||||
|
||||
### Scope
|
||||
|
||||
Add a static scanner that runs during skill load (and optionally install) to prevent obvious unsafe skill packages.
|
||||
|
||||
Recommended baseline checks:
|
||||
|
||||
- Reject symlinks.
|
||||
- Reject binary blobs / huge files.
|
||||
- Validate `manifest.json.permissions` exists for skills that will be routed as intent targets.
|
||||
- Optionally scan `SKILL.md` for disallowed patterns (e.g. embedding secrets, injection markers).
|
||||
|
||||
### Best Insertion Point
|
||||
|
||||
- `src/skills/loader.ts` inside `loadSkill()` (covers daemon load, watcher reload, and CLI skill operations).
|
||||
|
||||
Optional second insertion:
|
||||
|
||||
- `src/skills/installer.ts` pre-copy scan to avoid persisting unsafe content into managed dir.
|
||||
|
||||
### Tests
|
||||
|
||||
- `src/skills/loader.test.ts` with fixture dirs for:
|
||||
- symlink rejection
|
||||
- oversized/binary rejection
|
||||
- missing/invalid permissions (when used for routing)
|
||||
|
||||
### Acceptance
|
||||
|
||||
- Unsafe skills do not load and do not get injected into the system prompt.
|
||||
- Clean skills behave exactly as before.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 4 (P2): Elevated Mode (Break Glass)
|
||||
|
||||
### Scope
|
||||
|
||||
Add a user-visible, auditable, time-bounded mechanism to permit host execution of high-risk tools.
|
||||
|
||||
Constraints:
|
||||
|
||||
- Must require explicit confirmation.
|
||||
- Must expire automatically.
|
||||
- Must emit audit events with reason + TTL.
|
||||
|
||||
### Integration Points
|
||||
|
||||
- `src/tools/executor.ts`: enforcement gate + audit fields
|
||||
- `src/daemon/routing.ts`: set `ToolPolicyContext.executionEnvironment` based on elevation state
|
||||
- `src/hooks/*`: confirmation UX
|
||||
|
||||
### Tests
|
||||
|
||||
- Unit tests for TTL expiry and denial without elevation.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 5 (P2): Matrix Channel Adapter
|
||||
|
||||
### Scope
|
||||
|
||||
Add Matrix as a channel adapter following existing patterns.
|
||||
|
||||
### Implementation Tasks
|
||||
|
||||
- Implement adapter: `src/channels/matrix/adapter.ts` (+ tests)
|
||||
- Export and register:
|
||||
- `src/channels/matrix/index.ts`
|
||||
- `src/channels/index.ts`
|
||||
- `src/daemon/channels.ts`
|
||||
- Config schema: `src/config/schema.ts`
|
||||
- Dashboard services reporting: `src/gateway/handlers/services.ts`
|
||||
- Config secret redaction: `src/gateway/handlers/config.ts`
|
||||
|
||||
### Acceptance
|
||||
|
||||
- Inbound messages normalize to `InboundMessage` with stable `senderId`.
|
||||
- Outbound send works.
|
||||
- Allowlists + mention gating work.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 6 (P3): Deployment Targets (Nix + Fly/Railway/Render)
|
||||
|
||||
### Nix
|
||||
|
||||
- Provide a flake/package that builds `dist/` and preserves `dist/gateway/ui` adjacency.
|
||||
- Optional NixOS module with:
|
||||
- service user
|
||||
- config path
|
||||
- data dir
|
||||
|
||||
### PaaS Targets
|
||||
|
||||
- Add first-class docs and templates.
|
||||
- Ensure network binding is correct (`server.localhost: false`).
|
||||
- Either:
|
||||
- add `PORT` env override support, or
|
||||
- document explicit config requirements.
|
||||
|
||||
---
|
||||
|
||||
## Deferred Items (P4+)
|
||||
|
||||
These are substantial UX/ecosystem projects or highly platform-specific; defer until there is a clear need:
|
||||
|
||||
- Canvas/A2UI
|
||||
- Companion apps (macOS/iOS/Android)
|
||||
- Voice wake/talk mode + camera/screen capture/location
|
||||
- Presence tracking
|
||||
- Bonjour/mDNS discovery
|
||||
- QMD backend
|
||||
- ClawHub registry
|
||||
- Signal/Teams/Google Chat (enterprise/ops heavy)
|
||||
- iMessage/BlueBubbles (Apple ecosystem)
|
||||
|
||||
## Suggested Next Execution Order
|
||||
|
||||
1) Credential System v2 (API + OAuth/token)
|
||||
2) Vercel AI Gateway provider
|
||||
3) Skill safety scanner
|
||||
4) Elevated mode
|
||||
5) Matrix adapter
|
||||
6) Deployment targets
|
||||
@@ -0,0 +1,146 @@
|
||||
# Skill/Plugin Safety Scanner — Implementation Checklist
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Parent roadmap:** `docs/plans/2026-02-15-openclaw-gap-roadmap.md`
|
||||
|
||||
**Goal:** Close the gap item "Skill/plugin code safety scanner" by adding static analysis gates for skills so unsafe skill packages are rejected (or marked unavailable) before they can be injected into prompts or used in routing.
|
||||
|
||||
## Scope
|
||||
|
||||
### In scope
|
||||
|
||||
- Add a static scanner for skill directories.
|
||||
- Run the scanner during skill load (covers daemon startup, watcher reloads, and CLI skill operations).
|
||||
- Optionally run the scanner during install/upgrade (pre-copy) to avoid persisting unsafe content.
|
||||
- Emit audit events (pass/fail + reason) without leaking sensitive content.
|
||||
|
||||
### Out of scope (for this milestone)
|
||||
|
||||
- Full SAST for arbitrary languages.
|
||||
- Code signing / provenance chains.
|
||||
- Remote registries.
|
||||
|
||||
## Baseline
|
||||
|
||||
Current skill load/install entry points:
|
||||
|
||||
- Load/validate: `src/skills/loader.ts` (`loadSkill()`)
|
||||
- Install/upgrade: `src/skills/installer.ts`
|
||||
- CLI workflows: `src/cli/skills.ts`
|
||||
- Prompt injection: `src/skills/registry.ts` -> `src/daemon/services.ts` (`# Available Skills` section)
|
||||
|
||||
## Scanner Policy (MVP)
|
||||
|
||||
### File system safety
|
||||
|
||||
- Deny any symlinks inside a skill directory.
|
||||
- Deny files above a size threshold (default: 1MB) unless allowlisted.
|
||||
- Deny binary blobs (heuristic: NUL bytes or high non-text ratio) for `SKILL.md` and `manifest.json`.
|
||||
|
||||
### Manifest safety
|
||||
|
||||
- `manifest.json` must parse as JSON if present.
|
||||
- If a skill is intended for routing (i.e. referenced by an intent target), require `manifest.json.permissions`.
|
||||
- This aligns with the deny-by-default runtime enforcement: skills without permissions should not be routable.
|
||||
|
||||
### Prompt content safety (lightweight)
|
||||
|
||||
- Scan `SKILL.md` for obvious prompt-injection patterns:
|
||||
- "ignore previous" / "system prompt" / "exfiltrate" / "send secrets" etc.
|
||||
- Treat these as warnings or failures (recommend: failure for now).
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### Scanner API
|
||||
|
||||
Create a small scanner module:
|
||||
|
||||
- `src/skills/scanner.ts`
|
||||
- `scanSkillDirectory(dir): { ok: boolean; issues: SkillScanIssue[] }`
|
||||
- `SkillScanIssue = { severity: 'error' | 'warn'; code: string; message: string; path?: string }`
|
||||
|
||||
Config knobs (optional in MVP):
|
||||
|
||||
- `skills.scan.enabled` (default true)
|
||||
- `skills.scan.max_file_size_bytes` (default 1_000_000)
|
||||
- `skills.scan.fail_on_warnings` (default false)
|
||||
|
||||
### Enforce during load
|
||||
|
||||
- In `src/skills/loader.ts` inside `loadSkill()`:
|
||||
- run scanner before returning a `Skill`
|
||||
- on failure:
|
||||
- either return `null` (hard fail), OR
|
||||
- return Skill with `available=false` and add reasons
|
||||
|
||||
Recommendation: mark skill unavailable (not null) so the user can see it in `flynn skills list` with reasons.
|
||||
|
||||
### Enforce during install (optional but recommended)
|
||||
|
||||
- In `src/skills/installer.ts`:
|
||||
- scan `sourceDir` before copying
|
||||
- fail install if scanner errors
|
||||
|
||||
### Audit events
|
||||
|
||||
- Add audit event types for skill scans:
|
||||
- `skills.scan.pass`
|
||||
- `skills.scan.fail`
|
||||
- Ensure messages do not include raw secret content; include issue codes and counts.
|
||||
|
||||
## PR Breakdown
|
||||
|
||||
### PR 1 — Scanner module + loader integration
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add `src/skills/scanner.ts` with MVP rules.
|
||||
- [ ] Integrate into `src/skills/loader.ts`.
|
||||
- [ ] Update `src/skills/loader.test.ts` with fixtures:
|
||||
- symlink skill rejected
|
||||
- oversized file rejected
|
||||
- injection marker in SKILL.md rejected
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `flynn doctor` / skill load does not crash on bad skills.
|
||||
- Bad skills become unavailable with clear reasons.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/skills/loader.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 2 — Installer integration
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add scan preflight to `src/skills/installer.ts install()`.
|
||||
- [ ] Add tests for install failure on scan errors.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/skills/installer.test.ts` (add if missing)
|
||||
|
||||
---
|
||||
|
||||
### PR 3 — Audit events
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `src/audit/types.ts` to include skill scan events.
|
||||
- [ ] Extend `src/audit/logger.ts` helpers.
|
||||
- [ ] Emit events from loader/install paths.
|
||||
- [ ] Add targeted tests validating event shape (if audit logger has test coverage).
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Audit logs show scan pass/fail with counts and stable issue codes.
|
||||
|
||||
## Final Checks
|
||||
|
||||
- [ ] `pnpm typecheck`
|
||||
- [ ] `pnpm test:run`
|
||||
- [ ] Update `docs/plans/state.json` entry to `completed` once implemented (include test status).
|
||||
@@ -0,0 +1,116 @@
|
||||
# Vercel AI Gateway Provider — Implementation Checklist
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Parent roadmap:** `docs/plans/2026-02-15-openclaw-gap-roadmap.md`
|
||||
|
||||
**Goal:** Close the gap item "Vercel AI Gateway" by adding a first-class model provider that routes through the Vercel AI Gateway using Flynn's existing OpenAI-compatible client path where possible.
|
||||
|
||||
## Scope
|
||||
|
||||
- Add a new provider id (`vercel` or `vercel_ai_gateway`).
|
||||
- Implement the provider via `OpenAIClient` when the gateway is OpenAI-compatible.
|
||||
- Ensure the provider works with:
|
||||
- streaming (if supported by OpenAIClient path)
|
||||
- tool calling
|
||||
- model tier switching via `/model <tier> <provider/model>`
|
||||
|
||||
Non-goals:
|
||||
|
||||
- Implementing gateway-specific “extras” (tracing, metadata) unless required.
|
||||
|
||||
## Design
|
||||
|
||||
### Provider id
|
||||
|
||||
Recommended id: `vercel` (short) or `vercel_ai_gateway` (explicit).
|
||||
|
||||
Pick one and use it consistently in:
|
||||
|
||||
- `MODEL_PROVIDERS` in `src/config/schema.ts`
|
||||
- docs + setup wizard provider list
|
||||
- doctor checks
|
||||
|
||||
### Config fields
|
||||
|
||||
Use the existing `ModelConfig` fields:
|
||||
|
||||
- `endpoint`: base URL of gateway (OpenAI-compatible)
|
||||
- `api_key`: gateway API key (or env var)
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
default:
|
||||
provider: vercel_ai_gateway
|
||||
model: gpt-4.1
|
||||
endpoint: "https://gateway.ai.example.com/v1"
|
||||
api_key: "${VERCEL_AI_GATEWAY_API_KEY}"
|
||||
```
|
||||
|
||||
## PR Breakdown
|
||||
|
||||
### PR 1 — Schema + factory wiring
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add provider id to `MODEL_PROVIDERS` in `src/config/schema.ts`.
|
||||
- [ ] Update `createClientFromConfig()` in `src/daemon/models.ts`:
|
||||
- [ ] map provider -> `new OpenAIClient({ model, apiKey, baseURL })`
|
||||
- [ ] require an API key (config or env var)
|
||||
- [ ] use `cfg.endpoint` as `baseURL` (or a sensible default if the gateway has one)
|
||||
- [ ] Update `/model` strict-tier switching support (should be automatic once provider id is recognized).
|
||||
|
||||
Tests:
|
||||
|
||||
- [ ] Update `src/config/schema.test.ts` to accept the new provider enum.
|
||||
- [ ] Add case to `src/daemon/clientFactory.test.ts`:
|
||||
- asserts the provider returns an OpenAI-compatible client
|
||||
- asserts `baseURL` is passed when `endpoint` is set
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `pnpm typecheck`
|
||||
- `pnpm test:run src/daemon/clientFactory.test.ts src/config/schema.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 2 — Doctor + setup wizard + docs
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Update `src/cli/doctor.ts` provider key checks:
|
||||
- if Vercel gateway requires a key, ensure doctor warns when missing
|
||||
- [ ] Update setup wizard provider picker (optional but recommended):
|
||||
- `src/cli/setup/providers.ts`
|
||||
- `src/cli/setup/providers.test.ts`
|
||||
- [ ] Document provider config in `README.md` (one short snippet; avoid long docs).
|
||||
|
||||
Tests:
|
||||
|
||||
- [ ] `pnpm test:run src/cli/setup/providers.test.ts` (if changed)
|
||||
- [ ] `pnpm test:run src/cli/doctor.test.ts` (if changed)
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `flynn doctor` guidance includes Vercel gateway key/env var info.
|
||||
|
||||
---
|
||||
|
||||
### PR 3 — Integration validation (optional)
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add a simple smoke test using the `synthetic` provider style or a mocked OpenAI SDK path if possible.
|
||||
- [ ] Ensure streaming works end-to-end via gateway.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- A real config can run `flynn send "hello"` using the gateway provider.
|
||||
|
||||
## Final Checks
|
||||
|
||||
- [ ] `pnpm typecheck`
|
||||
- [ ] `pnpm test:run`
|
||||
- [ ] Update `docs/plans/state.json` entry to `completed` once implemented (include test status).
|
||||
+57
-3
@@ -1,14 +1,68 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"updated_at": "2026-02-14",
|
||||
"updated_at": "2026-02-15",
|
||||
"description": "Tracks the status of all Flynn plans and implementation phases",
|
||||
|
||||
"plans": {
|
||||
"openclaw-gap-roadmap": {
|
||||
"file": "2026-02-15-openclaw-gap-roadmap.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Roadmap to close remaining OpenClaw gap analysis MISSING items, prioritizing per-tier credential system v2 (API + OAuth/token), Vercel AI Gateway provider, skill safety scanner, elevated mode, Matrix adapter, and deployment targets (Nix + PaaS)."
|
||||
},
|
||||
"credential-system-v2-api-and-oauth": {
|
||||
"file": "2026-02-15-credential-system-v2-api-and-oauth-checklist.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Executable PR-sized checklist to implement per-tier auth_mode with both API-key and OAuth/token credential sources, adding OpenAI API-key storage, Anthropic auth-token storage, CLI/TUI login UX, model factory enforcement, and doctor reporting."
|
||||
},
|
||||
"vercel-ai-gateway-provider": {
|
||||
"file": "2026-02-15-vercel-ai-gateway-provider-checklist.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Executable checklist to add a first-class Vercel AI Gateway model provider (OpenAI-compatible baseURL), including schema/provider registration, model factory wiring, doctor/setup updates, and tests."
|
||||
},
|
||||
"skill-safety-scanner": {
|
||||
"file": "2026-02-15-skill-safety-scanner-checklist.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Executable checklist to implement a static skill/plugin safety scanner integrated into skill load/install paths with audit events and tests, preventing unsafe skill packages from being injected into prompts or used via routing."
|
||||
},
|
||||
"openclaw-style-personal-agent-without-openclaw-risks": {
|
||||
"file": "2026-02-14-openclaw-style-personal-agent-without-openclaw-risks-plan.md",
|
||||
"status": "planned",
|
||||
"status": "completed",
|
||||
"date": "2026-02-14",
|
||||
"summary": "Milestone plan to reach OpenClaw-style personal-assistant efficiency with a safer trust boundary: capability-declared skills, sandbox-by-default for high-risk tools, prompt-injection firewall, secret scoping, and audit logging."
|
||||
"updated": "2026-02-14",
|
||||
"summary": "Implemented safe-by-default personal-agent hardening: capability-declared skills enforced via ToolPolicy + ToolExecutor, skill intent routing with sandbox-by-default for high-risk tools, prompt-injection provenance tags + tool-call guard, secret-scope gating for credentialed tools, and audit log hardening (redaction + correlation IDs).",
|
||||
"files_modified": [
|
||||
"README.md",
|
||||
"config/default.yaml",
|
||||
"docs/api/TOOLS.md",
|
||||
"docs/plans/2026-02-14-openclaw-safe-agent-implementation.md",
|
||||
"src/skills/types.ts",
|
||||
"src/skills/loader.ts",
|
||||
"src/tools/types.ts",
|
||||
"src/tools/policy.ts",
|
||||
"src/tools/executor.ts",
|
||||
"src/backends/native/agent.ts",
|
||||
"src/daemon/routing.ts",
|
||||
"src/daemon/index.ts",
|
||||
"src/daemon/services.ts",
|
||||
"src/cli/skills.ts",
|
||||
"src/cli/setup/security.ts",
|
||||
"src/gateway/handlers/services.ts",
|
||||
"src/audit/types.ts",
|
||||
"src/audit/logger.ts"
|
||||
],
|
||||
"files_created": [
|
||||
"docs/security/SAFE_PERSONAL_AGENT.md",
|
||||
"src/audit/redact.ts"
|
||||
],
|
||||
"test_status": "pnpm test:run + pnpm typecheck passing"
|
||||
},
|
||||
"openclaw-feature-gap-analysis": {
|
||||
"file": "2026-02-06-openclaw-feature-gap-analysis.md",
|
||||
|
||||
@@ -0,0 +1,240 @@
|
||||
# Safe-By-Default Personal Agent
|
||||
|
||||
This document describes Flynn's "OpenClaw-style" safety boundary: how skills declare capabilities, how those capabilities are enforced at runtime, how high-risk execution is sandboxed by default, how prompt injection is mitigated, and what gets logged (without leaking secrets).
|
||||
|
||||
If you're looking for API-level tool contracts, see `docs/api/TOOLS.md`.
|
||||
|
||||
## Overview
|
||||
|
||||
Flynn is built around a strict separation of:
|
||||
|
||||
- **Conversation** (LLM output)
|
||||
- **Capabilities** (tools)
|
||||
- **Policy** (what tools are allowed, under what conditions)
|
||||
|
||||
This milestone adds a skill capability layer and hardens the tool loop.
|
||||
|
||||
Core principles:
|
||||
|
||||
- Capability declarations beat intentions: skills get only what they declare.
|
||||
- Deny by default: a skill without a `permissions` manifest has no tool access.
|
||||
- Treat fetched/tool content as untrusted data, not instructions.
|
||||
- Never leak secrets into audit logs.
|
||||
|
||||
## Skills: Capability Manifests
|
||||
|
||||
Each skill lives in a directory with:
|
||||
|
||||
- `SKILL.md` (instructions injected into the system prompt)
|
||||
- `manifest.json` (metadata + optional capabilities)
|
||||
|
||||
The capability declaration is `manifest.json.permissions`.
|
||||
|
||||
See: `src/skills/types.ts`.
|
||||
|
||||
### `permissions` Schema (manifest.json)
|
||||
|
||||
```json
|
||||
{
|
||||
"permissions": {
|
||||
"tool_groups": ["group:web", "group:memory"],
|
||||
"tools": ["web.fetch", "web.search"],
|
||||
"fs": {
|
||||
"read": ["/home/will/Documents/**"],
|
||||
"write": ["/home/will/Documents/notes/**"]
|
||||
},
|
||||
"net": [
|
||||
{ "host": "api.todoist.com", "ports": [443] },
|
||||
{ "host": "*.github.com", "ports": [443] }
|
||||
],
|
||||
"secrets": ["gmail", "web_search"],
|
||||
"execution_environment": "sandbox"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Fields:
|
||||
|
||||
- `tool_groups`: tool-group allowlist using names from `src/tools/policy.ts` (`group:web`, `group:fs`, etc.)
|
||||
- `tools`: explicit tool-name/pattern allowlist (glob). If present, it overrides `tool_groups`.
|
||||
- `fs.read` / `fs.write`: allowed path globs (checked for `file.*` tools).
|
||||
- `net`: allowed hosts (glob) and optional port list (best-effort enforcement for `web.fetch`).
|
||||
- `secrets`: secret scopes allowed for this skill (used to gate credentialed tools).
|
||||
- `execution_environment`: `sandbox` (default) or `host` (escape hatch for high-risk operations).
|
||||
|
||||
### Backward Compatibility
|
||||
|
||||
Skills without `permissions` still load, but:
|
||||
|
||||
- If a skill is activated (via routing) and it has no `permissions` block, **it has no tool access**.
|
||||
- This is deliberate: skills should be auditable capability packages.
|
||||
|
||||
## Runtime Enforcement
|
||||
|
||||
Enforcement happens in two places:
|
||||
|
||||
1. **Tool listing / exposure** (ToolPolicy)
|
||||
2. **Tool execution** (ToolExecutor) — defense in depth
|
||||
|
||||
### ToolPolicy: Restricting Available Tools
|
||||
|
||||
When a skill context is active, the tool allow set is intersected with the skill's declared allowlist.
|
||||
|
||||
See: `src/tools/policy.ts`.
|
||||
|
||||
Important behaviors:
|
||||
|
||||
- If `skillName` is set but `skillPermissions` is missing, ToolPolicy returns an empty allowed set.
|
||||
- If `permissions.tools` is present, it overrides `permissions.tool_groups`.
|
||||
|
||||
### ToolExecutor: Enforcing Paths, Network, Secrets, and Injection Guards
|
||||
|
||||
See: `src/tools/executor.ts`.
|
||||
|
||||
When a skill context is active (`ToolPolicyContext.skillName`):
|
||||
|
||||
- Filesystem writes are blocked outside `permissions.fs.write`.
|
||||
- Filesystem reads are blocked outside `permissions.fs.read` (for `file.read`/`file.list`).
|
||||
- Credentialed tools require their `requiredSecretScopes` be present in the skill's allowed scopes.
|
||||
- If untrusted content has been seen, obviously malicious argument markers can block high-risk tool calls.
|
||||
|
||||
## Skill Routing (Intents)
|
||||
|
||||
Skills can be activated via intent rules.
|
||||
|
||||
See:
|
||||
|
||||
- Config schema: `src/config/schema.ts` (`intents.rules[].target.type = 'skill'`)
|
||||
- Routing: `src/daemon/routing.ts`
|
||||
|
||||
Example config:
|
||||
|
||||
```yaml
|
||||
intents:
|
||||
enabled: true
|
||||
match_threshold: 0.7
|
||||
rules:
|
||||
- name: "web-research"
|
||||
patterns: ["research *", "look up *"]
|
||||
target: { type: skill, name: my-web-skill }
|
||||
enabled: true
|
||||
```
|
||||
|
||||
When an intent routes to a skill:
|
||||
|
||||
- `toolPolicyContext.skillName` and `toolPolicyContext.skillPermissions` are set
|
||||
- High-risk execution defaults to sandbox (when available)
|
||||
|
||||
## Sandbox-By-Default (High-Risk Tools)
|
||||
|
||||
In skill context, high-risk tools are not allowed to run on the host unless the skill explicitly opts in.
|
||||
|
||||
High-risk tools include:
|
||||
|
||||
- `shell.exec`
|
||||
- `process.start`
|
||||
- `process.kill`
|
||||
- `file.write`, `file.edit`, `file.patch`
|
||||
- all `browser.*`
|
||||
|
||||
Behavior:
|
||||
|
||||
- Default (`execution_environment` omitted or `sandbox`):
|
||||
- If Docker sandbox is enabled and available, `shell.exec` and `process.start` run inside the per-session sandbox container.
|
||||
- If sandbox is not available, host execution for high-risk tools is denied for skill contexts.
|
||||
- Escape hatch (`execution_environment: host`): high-risk tools are permitted to run on host (still subject to tool policy + hooks/autonomy).
|
||||
|
||||
Note: today, only `shell.exec` and `process.start` are replaced with sandboxed implementations. Other high-risk tools are blocked-by-default in skill contexts unless host mode is explicitly allowed.
|
||||
|
||||
## Prompt Injection Mitigation
|
||||
|
||||
Flynn uses a practical defense-in-depth approach:
|
||||
|
||||
1. System prompt guidance: fetched/tool content is treated as untrusted data.
|
||||
2. Provenance tagging: tool results are wrapped in provenance markers.
|
||||
3. Tool-call guard: when untrusted content has been observed, tool calls with obvious injection markers are blocked.
|
||||
|
||||
### Provenance Wrapping
|
||||
|
||||
Tool results returned to the model are wrapped like:
|
||||
|
||||
```text
|
||||
[provenance=fetched_content tool=web.fetch untrusted=true]
|
||||
...tool output...
|
||||
[/provenance]
|
||||
```
|
||||
|
||||
See: `src/backends/native/agent.ts`.
|
||||
|
||||
### Tool-Call Guard
|
||||
|
||||
When `ToolPolicyContext.untrustedContent` is true:
|
||||
|
||||
- High-risk tool calls whose args contain obvious markers (e.g. `rm -rf`, `ignore previous`, `exfiltrate`, etc.) are blocked.
|
||||
- Network tools (`web.fetch`, `web.search`) refuse arguments containing secret-like fields.
|
||||
|
||||
See: `src/tools/executor.ts`.
|
||||
|
||||
## Secret Scopes
|
||||
|
||||
Tools can declare which secret scopes they require:
|
||||
|
||||
- `Tool.requiredSecretScopes?: string[]`
|
||||
|
||||
Skills declare which scopes they are allowed to use:
|
||||
|
||||
- `manifest.json.permissions.secrets?: string[]`
|
||||
|
||||
Enforcement:
|
||||
|
||||
- In skill context, if a tool requires scopes not allowed by the skill, ToolExecutor denies the tool.
|
||||
- Outside skill context, secrets are treated as "ambient" (allowed) to preserve backward compatibility.
|
||||
|
||||
See:
|
||||
|
||||
- `src/tools/types.ts`
|
||||
- `src/tools/executor.ts`
|
||||
- Examples: `src/tools/builtin/gmail.ts`, `src/tools/builtin/gcal.ts`, `src/tools/builtin/web-search.ts`
|
||||
|
||||
## Audit Logging (Without Secret Leaks)
|
||||
|
||||
Tool execution is audited, but sensitive values are redacted before writing to disk.
|
||||
|
||||
See:
|
||||
|
||||
- `src/audit/logger.ts`
|
||||
- `src/audit/types.ts`
|
||||
- `src/audit/redact.ts`
|
||||
|
||||
Notable fields:
|
||||
|
||||
- `execution_id`: a per-tool-call UUID for correlation
|
||||
- `execution_environment`: `host` or `sandbox`
|
||||
- `skill_name`: active skill (if any)
|
||||
- `redactions_applied`: count of redaction operations
|
||||
- `tool.approval`: emitted when a confirm hook is resolved
|
||||
|
||||
Example tool start event (JSONL):
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": 0,
|
||||
"level": "debug",
|
||||
"event_type": "tool.start",
|
||||
"event": {
|
||||
"tool_name": "shell.exec",
|
||||
"execution_id": "...",
|
||||
"execution_environment": "sandbox",
|
||||
"skill_name": "my-web-skill",
|
||||
"redactions_applied": 1,
|
||||
"tool_args": { "command": "echo [REDACTED_TOKEN]" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Recommended Operator Defaults
|
||||
|
||||
- Enable Docker sandboxing (`sandbox.enabled: true`).
|
||||
- Enable DM pairing (`pairing.enabled: true`) on any messaging surface.
|
||||
- Use a conservative tool profile for general chat (`tools.profile: messaging`).
|
||||
- Use skill intent routing for specialized workflows and keep skill permissions narrow.
|
||||
Reference in New Issue
Block a user