docs: add safety docs and OpenClaw gap roadmap
This commit is contained in:
William Valentin
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,253 @@
|
||||
# Credential System v2 (API + OAuth/token) — Implementation Checklist
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Parent roadmap:** `docs/plans/2026-02-15-openclaw-gap-roadmap.md`
|
||||
|
||||
**Goal:** Close the gap item "OAuth subscription auth" by supporting **both** API-key credentials and OAuth/token-based credentials (provider-specific) with consistent UX, per-tier control, and deterministic resolution.
|
||||
|
||||
## Scope Summary
|
||||
|
||||
- Add `auth_mode` **per tier** (default/fast/complex/local and any `local_providers` entries).
|
||||
- Keep backward compatibility with existing `use_oauth` behavior.
|
||||
- Add stored credential support where it currently doesn't exist:
|
||||
- OpenAI: stored API key (OAuth already exists)
|
||||
- Anthropic: stored auth token (API key already exists)
|
||||
- Improve doctor output to surface which auth sources are present (without revealing secrets).
|
||||
|
||||
Non-goals (explicitly out of scope for this checklist):
|
||||
|
||||
- Inventing an Anthropic OAuth device flow.
|
||||
- Building new provider integrations (Vercel/MiniMax/etc.).
|
||||
|
||||
## Current Baseline (important constraints)
|
||||
|
||||
- OpenAI OAuth uses the ChatGPT/Codex backend endpoint (SSE) and currently **does not support tools** on that path.
|
||||
- Source: `src/models/openai.ts`
|
||||
- Anthropic supports `apiKey` and `authToken` in `AnthropicClientConfig`.
|
||||
- Source: `src/models/anthropic.ts`
|
||||
- Stored credentials live at `~/.config/flynn/auth.json`.
|
||||
- Source: `src/auth/openai.ts`, `src/auth/anthropic.ts`
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### 1) New config field: `auth_mode`
|
||||
|
||||
Add `auth_mode` to the per-tier model config:
|
||||
|
||||
- `auto` (default)
|
||||
- `api_key`
|
||||
- `oauth`
|
||||
|
||||
`oauth` is interpreted as "OAuth/token mode" (provider-specific). For Anthropic, that means `auth_token`.
|
||||
|
||||
### 2) Backward compatibility: `use_oauth`
|
||||
|
||||
Preserve `use_oauth` as a compatibility alias.
|
||||
|
||||
Recommended rule:
|
||||
|
||||
- If `auth_mode` is set: it wins.
|
||||
- Else if `use_oauth: true`: treat as `auth_mode: oauth`.
|
||||
|
||||
### 3) Credential resolution order
|
||||
|
||||
For each provider, resolve the required credential type by trying:
|
||||
|
||||
1) config (`api_key` / `auth_token`)
|
||||
2) env var
|
||||
3) auth store (`~/.config/flynn/auth.json`)
|
||||
|
||||
`auth_mode` controls which credential type is required.
|
||||
|
||||
## PR Breakdown (atomic, test-backed)
|
||||
|
||||
### PR 1 — Schema + docs: per-tier `auth_mode`
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add `auth_mode` to `modelConfigBaseSchema` in `src/config/schema.ts`.
|
||||
- [ ] Update `src/config/schema.test.ts` to cover defaults + validation.
|
||||
- [ ] Update `README.md` config examples (brief mention).
|
||||
- [ ] Update `config/default.yaml` comment/help text (brief mention).
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Config parses with no changes (defaults preserved).
|
||||
- Setting `auth_mode: oauth` or `auth_mode: api_key` validates.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/config/schema.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 2 — OpenAI auth store: add API-key storage
|
||||
|
||||
Goal: allow OpenAI to run without `api_key` in YAML.
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `src/auth/openai.ts` `AuthStore` shape to allow `openai.api_key` alongside existing OAuth info.
|
||||
- [ ] Add functions:
|
||||
- [ ] `loadStoredOpenAIApiKey()`
|
||||
- [ ] `storeOpenAIApiKey(key)`
|
||||
- [ ] `clearOpenAIApiKey()`
|
||||
- [ ] `getOpenAIApiKey()` (env override + store)
|
||||
- [ ] Keep existing OAuth store code working unchanged.
|
||||
- [ ] Add/extend tests for new store functions.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/auth/openai.ts`
|
||||
- `src/auth/openai.test.ts` (or add if missing)
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Stored OpenAI API key is written to `~/.config/flynn/auth.json` with `0600` permissions.
|
||||
- OAuth entry remains backward compatible.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/auth/openai.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 3 — Anthropic auth store: add auth-token storage
|
||||
|
||||
Goal: allow `auth_token` to be stored and selected with `auth_mode: oauth`.
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `src/auth/anthropic.ts` auth store shape to include `auth_token`.
|
||||
- [ ] Add functions:
|
||||
- [ ] `loadStoredAnthropicAuthToken()`
|
||||
- [ ] `storeAnthropicAuthToken(token)`
|
||||
- [ ] `clearAnthropicAuthToken()`
|
||||
- [ ] `getAnthropicAuthToken()`
|
||||
- [ ] Extend `src/auth/anthropic.test.ts`.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/auth/anthropic.ts`
|
||||
- `src/auth/anthropic.test.ts`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `auth_token` can be stored and resolved without being present in YAML.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/auth/anthropic.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 4 — CLI commands for managing new stored credentials
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add `flynn openai-key` command (store API key in auth.json).
|
||||
- [ ] Extend `flynn anthropic-auth` to support storing either API key or auth token:
|
||||
- [ ] recommended: `flynn anthropic-auth --token` OR `flynn anthropic-token`
|
||||
- [ ] Update `src/cli/index.ts` registration.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/cli/openai-key.ts` (new)
|
||||
- `src/cli/anthropic-auth.ts` (modify)
|
||||
- `src/cli/index.ts`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- CLI can store credentials without printing them.
|
||||
- Re-running commands detects existing stored credentials and exits cleanly.
|
||||
|
||||
Tests:
|
||||
|
||||
- Add targeted unit tests if the CLI layer has existing patterns; otherwise validate via integration tests where feasible.
|
||||
|
||||
---
|
||||
|
||||
### PR 5 — TUI `/login` UX: OpenAI choice (OAuth vs API key) + Anthropic token
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Update `/login openai` in `src/frontends/tui/minimal.ts`:
|
||||
- [ ] Present a simple prompt: "1) OAuth device flow 2) Paste API key"
|
||||
- [ ] Store selected credential via auth store
|
||||
- [ ] Add `/login anthropic` in `src/frontends/tui/minimal.ts`:
|
||||
- [ ] "1) Paste API key 2) Paste auth token"
|
||||
- [ ] Keep existing `/login github` and `/login zai` behavior intact.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/frontends/tui/minimal.ts`
|
||||
- `src/frontends/tui/commands.ts` (if command parsing needs to accept new provider)
|
||||
|
||||
Acceptance:
|
||||
|
||||
- TUI can store OpenAI API key or OAuth token.
|
||||
- TUI can store Anthropic API key or auth token.
|
||||
|
||||
Tests:
|
||||
|
||||
- Add or extend minimal TUI tests as needed (existing suite patterns exist for model switching).
|
||||
|
||||
---
|
||||
|
||||
### PR 6 — Model factory: enforce `auth_mode` per tier
|
||||
|
||||
This is the core runtime change.
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Update `src/daemon/models.ts`:
|
||||
- [ ] Read `cfg.auth_mode` (or inferred from `use_oauth`) per tier.
|
||||
- [ ] For OpenAI:
|
||||
- [ ] `auth_mode=oauth`: configure `OpenAIClient({ useOAuth: true })` and verify OAuth tokens exist.
|
||||
- [ ] `auth_mode=api_key`: configure `OpenAIClient({ apiKey: resolvedKey })`.
|
||||
- [ ] For Anthropic:
|
||||
- [ ] `auth_mode=oauth`: require auth token (config/env/store).
|
||||
- [ ] `auth_mode=api_key`: require API key (config/env/store).
|
||||
- [ ] For other providers:
|
||||
- [ ] define behavior explicitly (likely `api_key` only unless provider already supports token-style auth).
|
||||
- [ ] Ensure error messages name the expected auth type and remediation.
|
||||
|
||||
Files:
|
||||
|
||||
- `src/daemon/models.ts`
|
||||
- potentially `src/models/openai.ts` (if you decide to unify API key vs OAuth selection naming)
|
||||
|
||||
Tests:
|
||||
|
||||
- `src/daemon/clientFactory.test.ts`
|
||||
- [ ] auth_mode precedence over use_oauth
|
||||
- [ ] auto -> api key path
|
||||
- [ ] oauth -> token path
|
||||
- [ ] correct failures when missing
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Selecting `auth_mode` changes runtime behavior deterministically.
|
||||
|
||||
---
|
||||
|
||||
### PR 7 — Doctor: report auth source availability
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `checkModelConnectivity` in `src/cli/doctor.ts` to reflect `auth_mode`:
|
||||
- [ ] If `auth_mode=api_key`, warn/fail when API key is absent from config/env/store.
|
||||
- [ ] If `auth_mode=oauth`, warn/fail when OAuth/token is absent.
|
||||
- [ ] If `auth_mode=auto`, keep current behavior but improve messaging.
|
||||
- [ ] Add tests in `src/cli/doctor.test.ts`.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Doctor output tells user what to do next (command name + env var) without exposing secrets.
|
||||
|
||||
## Final Integration Checks
|
||||
|
||||
- [ ] `pnpm typecheck`
|
||||
- [ ] `pnpm test:run`
|
||||
- [ ] Update `docs/plans/state.json` entry for this checklist once implemented (status, summary, test status).
|
||||
@@ -0,0 +1,343 @@
|
||||
# OpenClaw Gap Roadmap (Flynn)
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Source:** `docs/plans/2026-02-06-openclaw-feature-gap-analysis.md`
|
||||
|
||||
**Goal:** Turn the remaining **MISSING** items into an executable roadmap with clear milestones, acceptance criteria, and test strategy.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
A gap item is considered implemented when:
|
||||
|
||||
- It is behind config flags where appropriate.
|
||||
- It has tests for core logic (unit/integration, plus minimal adapter/provider mocks where applicable).
|
||||
- It has docs (README or relevant docs section).
|
||||
- `docs/plans/state.json` is updated with status + summary + test status.
|
||||
|
||||
## Remaining Gap Inventory (from gap analysis)
|
||||
|
||||
### Channels / Frontends (MISSING)
|
||||
|
||||
- Signal
|
||||
- Matrix
|
||||
- Google Chat
|
||||
- Microsoft Teams
|
||||
- iMessage/BlueBubbles
|
||||
- Zalo
|
||||
- LINE/Feishu/Mattermost
|
||||
|
||||
### Model Providers (MISSING)
|
||||
|
||||
- MiniMax / Moonshot
|
||||
- Vercel AI Gateway
|
||||
- OAuth subscription auth (support both API key and OAuth/token)
|
||||
|
||||
### Agent Runtime / UX (MISSING)
|
||||
|
||||
- Canvas / A2UI (agent-driven visual workspace)
|
||||
|
||||
### Memory (MISSING)
|
||||
|
||||
- QMD backend (experimental)
|
||||
|
||||
### Security (MISSING)
|
||||
|
||||
- Skill/plugin code safety scanner (static analysis)
|
||||
- Elevated mode (explicit host-exec escape hatch)
|
||||
|
||||
### Skills Ecosystem (MISSING)
|
||||
|
||||
- ClawHub / community skill registry
|
||||
|
||||
### Gateway / Infra / Deployment (MISSING)
|
||||
|
||||
- Nix deployment
|
||||
- Fly.io / Railway / Render
|
||||
- Announce delivery mode (isolated job delivery)
|
||||
- Bonjour/mDNS discovery
|
||||
|
||||
### Misc (MISSING)
|
||||
|
||||
- Presence tracking
|
||||
|
||||
### Companion Apps / Devices (MISSING)
|
||||
|
||||
- macOS menu bar app
|
||||
- iOS node
|
||||
- Android node
|
||||
- Voice Wake / Talk Mode
|
||||
- Camera / screen capture
|
||||
- Location access
|
||||
|
||||
## Roadmap Overview (Milestones)
|
||||
|
||||
This roadmap optimizes for: (1) high leverage, (2) low coupling, (3) ability to ship incrementally.
|
||||
|
||||
1) Credential System v2 (API + OAuth/token) [P0]
|
||||
2) Vercel AI Gateway provider [P1]
|
||||
3) Skill/plugin safety scanner [P1]
|
||||
4) Elevated mode (break-glass) [P2]
|
||||
5) Matrix channel adapter [P2]
|
||||
6) Deployment targets (Nix + PaaS) [P3]
|
||||
|
||||
Everything else is explicitly deferred until there is a strong user need.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 1 (P0): Credential System v2 — API + OAuth/token
|
||||
|
||||
This closes the gap item "OAuth subscription auth" as: providers should support both API-key credentials and OAuth/token-based credentials (where available), with consistent UX and deterministic resolution order.
|
||||
|
||||
### Scope
|
||||
|
||||
- Add an explicit `auth_mode` selector per model tier config:
|
||||
- `auto` (default): try the most specific configured credential sources in priority order
|
||||
- `api_key`: require API key sources
|
||||
- `oauth`: require OAuth/token sources
|
||||
|
||||
Rationale: per-tier enables common setups like:
|
||||
|
||||
- `fast` uses an API key for deterministic reliability.
|
||||
- `default` uses OAuth/token for personal subscription accounts.
|
||||
|
||||
### Provider Behavior (initial)
|
||||
|
||||
- OpenAI:
|
||||
- Support OAuth (existing): device flow + stored token (`flynn openai-auth`, `/login openai`, `use_oauth: true`).
|
||||
- Add API key storage in the same auth store (so YAML can omit secrets).
|
||||
|
||||
- Anthropic:
|
||||
- Today supports: API key (config/env/auth store) and `auth_token` (config/env).
|
||||
- Add auth-token storage and allow selecting it via `auth_mode: oauth`.
|
||||
- Note: do not invent an OAuth device flow for Anthropic unless a real flow exists; "oauth" here means token-based auth.
|
||||
|
||||
### Design
|
||||
|
||||
Credential sources (recommended priority):
|
||||
|
||||
- Config (`api_key` / `auth_token`) -> Env var(s) -> Auth store (`~/.config/flynn/auth.json`)
|
||||
|
||||
Auth store should be able to hold multiple types per provider, e.g.:
|
||||
|
||||
```json
|
||||
{
|
||||
"openai": { "api_key": "...", "oauth": { "refresh_token": "...", "created_at": "..." } },
|
||||
"anthropic": { "api_key": "...", "auth_token": "..." }
|
||||
}
|
||||
```
|
||||
|
||||
### Implementation Tasks
|
||||
|
||||
Config/schema:
|
||||
|
||||
- Add `auth_mode` to the model config schema (per tier) in `src/config/schema.ts`.
|
||||
- Update schema tests in `src/config/schema.test.ts`.
|
||||
|
||||
Auth store:
|
||||
|
||||
- Extend `src/auth/openai.ts` to support API-key storage in auth.json (store/load/clear).
|
||||
- Extend `src/auth/anthropic.ts` to support auth-token storage in auth.json (store/load/clear).
|
||||
- Update `src/auth/index.ts` exports.
|
||||
|
||||
CLI:
|
||||
|
||||
- Add CLI command(s) for managing stored API keys for OpenAI (name TBD, recommended: `flynn openai-key`).
|
||||
- Extend Anthropic auth CLI to store either API key or auth token (flag or separate command).
|
||||
|
||||
TUI:
|
||||
|
||||
- Extend `/login openai` flow to let user choose OAuth device flow OR paste API key.
|
||||
- Add `/login anthropic` "paste API key" and "paste auth token" (non-OAuth).
|
||||
|
||||
Model factory:
|
||||
|
||||
- Update `createClientFromConfig()` in `src/daemon/models.ts` to resolve creds according to `auth_mode`.
|
||||
- Ensure error messages are explicit about which credential type is missing.
|
||||
|
||||
Doctor:
|
||||
|
||||
- Update `src/cli/doctor.ts` to report:
|
||||
- whether the provider has an API key source
|
||||
- whether OAuth/token sources exist
|
||||
- without revealing any secret material
|
||||
|
||||
### Tests
|
||||
|
||||
- `src/daemon/clientFactory.test.ts`
|
||||
- auth_mode=auto resolves API key when present
|
||||
- auth_mode=oauth resolves token when present
|
||||
- auth_mode=api_key fails when only OAuth exists (and vice versa)
|
||||
|
||||
- `src/auth/openai.test.ts` (or new) and `src/auth/anthropic.test.ts`
|
||||
- store/load/clear for all supported credential types
|
||||
|
||||
- `src/config/schema.test.ts`
|
||||
- config parsing + defaults for auth_mode
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- OpenAI can authenticate via either:
|
||||
- stored OAuth token, or
|
||||
- stored API key,
|
||||
without secrets in YAML.
|
||||
|
||||
- Anthropic can authenticate via either:
|
||||
- stored API key, or
|
||||
- stored auth token,
|
||||
without secrets in YAML.
|
||||
|
||||
- `/model` switching does not silently fall back to a weaker tier due to missing credentials; it returns an explicit error.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 2 (P1): Vercel AI Gateway Provider
|
||||
|
||||
### Scope
|
||||
|
||||
- Add a new model provider for Vercel AI Gateway.
|
||||
- If OpenAI-compatible, implement via `OpenAIClient` with a configurable `baseURL`.
|
||||
|
||||
### Implementation Tasks
|
||||
|
||||
- Add provider id in `src/config/schema.ts` (`MODEL_PROVIDERS`).
|
||||
- Add `createClientFromConfig()` case in `src/daemon/models.ts`.
|
||||
- Update setup wizard provider list (optional): `src/cli/setup/providers.ts`.
|
||||
- Update doctor key checks if needed: `src/cli/doctor.ts`.
|
||||
|
||||
### Tests
|
||||
|
||||
- `src/daemon/clientFactory.test.ts`: constructs correct client and passes `baseURL`.
|
||||
|
||||
### Acceptance
|
||||
|
||||
- Tool calls + streaming work through the gateway in at least one tier.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 3 (P1): Skill/Plugin Code Safety Scanner
|
||||
|
||||
### Scope
|
||||
|
||||
Add a static scanner that runs during skill load (and optionally install) to prevent obvious unsafe skill packages.
|
||||
|
||||
Recommended baseline checks:
|
||||
|
||||
- Reject symlinks.
|
||||
- Reject binary blobs / huge files.
|
||||
- Validate `manifest.json.permissions` exists for skills that will be routed as intent targets.
|
||||
- Optionally scan `SKILL.md` for disallowed patterns (e.g. embedding secrets, injection markers).
|
||||
|
||||
### Best Insertion Point
|
||||
|
||||
- `src/skills/loader.ts` inside `loadSkill()` (covers daemon load, watcher reload, and CLI skill operations).
|
||||
|
||||
Optional second insertion:
|
||||
|
||||
- `src/skills/installer.ts` pre-copy scan to avoid persisting unsafe content into managed dir.
|
||||
|
||||
### Tests
|
||||
|
||||
- `src/skills/loader.test.ts` with fixture dirs for:
|
||||
- symlink rejection
|
||||
- oversized/binary rejection
|
||||
- missing/invalid permissions (when used for routing)
|
||||
|
||||
### Acceptance
|
||||
|
||||
- Unsafe skills do not load and do not get injected into the system prompt.
|
||||
- Clean skills behave exactly as before.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 4 (P2): Elevated Mode (Break Glass)
|
||||
|
||||
### Scope
|
||||
|
||||
Add a user-visible, auditable, time-bounded mechanism to permit host execution of high-risk tools.
|
||||
|
||||
Constraints:
|
||||
|
||||
- Must require explicit confirmation.
|
||||
- Must expire automatically.
|
||||
- Must emit audit events with reason + TTL.
|
||||
|
||||
### Integration Points
|
||||
|
||||
- `src/tools/executor.ts`: enforcement gate + audit fields
|
||||
- `src/daemon/routing.ts`: set `ToolPolicyContext.executionEnvironment` based on elevation state
|
||||
- `src/hooks/*`: confirmation UX
|
||||
|
||||
### Tests
|
||||
|
||||
- Unit tests for TTL expiry and denial without elevation.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 5 (P2): Matrix Channel Adapter
|
||||
|
||||
### Scope
|
||||
|
||||
Add Matrix as a channel adapter following existing patterns.
|
||||
|
||||
### Implementation Tasks
|
||||
|
||||
- Implement adapter: `src/channels/matrix/adapter.ts` (+ tests)
|
||||
- Export and register:
|
||||
- `src/channels/matrix/index.ts`
|
||||
- `src/channels/index.ts`
|
||||
- `src/daemon/channels.ts`
|
||||
- Config schema: `src/config/schema.ts`
|
||||
- Dashboard services reporting: `src/gateway/handlers/services.ts`
|
||||
- Config secret redaction: `src/gateway/handlers/config.ts`
|
||||
|
||||
### Acceptance
|
||||
|
||||
- Inbound messages normalize to `InboundMessage` with stable `senderId`.
|
||||
- Outbound send works.
|
||||
- Allowlists + mention gating work.
|
||||
|
||||
---
|
||||
|
||||
## Milestone 6 (P3): Deployment Targets (Nix + Fly/Railway/Render)
|
||||
|
||||
### Nix
|
||||
|
||||
- Provide a flake/package that builds `dist/` and preserves `dist/gateway/ui` adjacency.
|
||||
- Optional NixOS module with:
|
||||
- service user
|
||||
- config path
|
||||
- data dir
|
||||
|
||||
### PaaS Targets
|
||||
|
||||
- Add first-class docs and templates.
|
||||
- Ensure network binding is correct (`server.localhost: false`).
|
||||
- Either:
|
||||
- add `PORT` env override support, or
|
||||
- document explicit config requirements.
|
||||
|
||||
---
|
||||
|
||||
## Deferred Items (P4+)
|
||||
|
||||
These are substantial UX/ecosystem projects or highly platform-specific; defer until there is a clear need:
|
||||
|
||||
- Canvas/A2UI
|
||||
- Companion apps (macOS/iOS/Android)
|
||||
- Voice wake/talk mode + camera/screen capture/location
|
||||
- Presence tracking
|
||||
- Bonjour/mDNS discovery
|
||||
- QMD backend
|
||||
- ClawHub registry
|
||||
- Signal/Teams/Google Chat (enterprise/ops heavy)
|
||||
- iMessage/BlueBubbles (Apple ecosystem)
|
||||
|
||||
## Suggested Next Execution Order
|
||||
|
||||
1) Credential System v2 (API + OAuth/token)
|
||||
2) Vercel AI Gateway provider
|
||||
3) Skill safety scanner
|
||||
4) Elevated mode
|
||||
5) Matrix adapter
|
||||
6) Deployment targets
|
||||
@@ -0,0 +1,146 @@
|
||||
# Skill/Plugin Safety Scanner — Implementation Checklist
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Parent roadmap:** `docs/plans/2026-02-15-openclaw-gap-roadmap.md`
|
||||
|
||||
**Goal:** Close the gap item "Skill/plugin code safety scanner" by adding static analysis gates for skills so unsafe skill packages are rejected (or marked unavailable) before they can be injected into prompts or used in routing.
|
||||
|
||||
## Scope
|
||||
|
||||
### In scope
|
||||
|
||||
- Add a static scanner for skill directories.
|
||||
- Run the scanner during skill load (covers daemon startup, watcher reloads, and CLI skill operations).
|
||||
- Optionally run the scanner during install/upgrade (pre-copy) to avoid persisting unsafe content.
|
||||
- Emit audit events (pass/fail + reason) without leaking sensitive content.
|
||||
|
||||
### Out of scope (for this milestone)
|
||||
|
||||
- Full SAST for arbitrary languages.
|
||||
- Code signing / provenance chains.
|
||||
- Remote registries.
|
||||
|
||||
## Baseline
|
||||
|
||||
Current skill load/install entry points:
|
||||
|
||||
- Load/validate: `src/skills/loader.ts` (`loadSkill()`)
|
||||
- Install/upgrade: `src/skills/installer.ts`
|
||||
- CLI workflows: `src/cli/skills.ts`
|
||||
- Prompt injection: `src/skills/registry.ts` -> `src/daemon/services.ts` (`# Available Skills` section)
|
||||
|
||||
## Scanner Policy (MVP)
|
||||
|
||||
### File system safety
|
||||
|
||||
- Deny any symlinks inside a skill directory.
|
||||
- Deny files above a size threshold (default: 1MB) unless allowlisted.
|
||||
- Deny binary blobs (heuristic: NUL bytes or high non-text ratio) for `SKILL.md` and `manifest.json`.
|
||||
|
||||
### Manifest safety
|
||||
|
||||
- `manifest.json` must parse as JSON if present.
|
||||
- If a skill is intended for routing (i.e. referenced by an intent target), require `manifest.json.permissions`.
|
||||
- This aligns with the deny-by-default runtime enforcement: skills without permissions should not be routable.
|
||||
|
||||
### Prompt content safety (lightweight)
|
||||
|
||||
- Scan `SKILL.md` for obvious prompt-injection patterns:
|
||||
- "ignore previous" / "system prompt" / "exfiltrate" / "send secrets" etc.
|
||||
- Treat these as warnings or failures (recommend: failure for now).
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### Scanner API
|
||||
|
||||
Create a small scanner module:
|
||||
|
||||
- `src/skills/scanner.ts`
|
||||
- `scanSkillDirectory(dir): { ok: boolean; issues: SkillScanIssue[] }`
|
||||
- `SkillScanIssue = { severity: 'error' | 'warn'; code: string; message: string; path?: string }`
|
||||
|
||||
Config knobs (optional in MVP):
|
||||
|
||||
- `skills.scan.enabled` (default true)
|
||||
- `skills.scan.max_file_size_bytes` (default 1_000_000)
|
||||
- `skills.scan.fail_on_warnings` (default false)
|
||||
|
||||
### Enforce during load
|
||||
|
||||
- In `src/skills/loader.ts` inside `loadSkill()`:
|
||||
- run scanner before returning a `Skill`
|
||||
- on failure:
|
||||
- either return `null` (hard fail), OR
|
||||
- return Skill with `available=false` and add reasons
|
||||
|
||||
Recommendation: mark skill unavailable (not null) so the user can see it in `flynn skills list` with reasons.
|
||||
|
||||
### Enforce during install (optional but recommended)
|
||||
|
||||
- In `src/skills/installer.ts`:
|
||||
- scan `sourceDir` before copying
|
||||
- fail install if scanner errors
|
||||
|
||||
### Audit events
|
||||
|
||||
- Add audit event types for skill scans:
|
||||
- `skills.scan.pass`
|
||||
- `skills.scan.fail`
|
||||
- Ensure messages do not include raw secret content; include issue codes and counts.
|
||||
|
||||
## PR Breakdown
|
||||
|
||||
### PR 1 — Scanner module + loader integration
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add `src/skills/scanner.ts` with MVP rules.
|
||||
- [ ] Integrate into `src/skills/loader.ts`.
|
||||
- [ ] Update `src/skills/loader.test.ts` with fixtures:
|
||||
- symlink skill rejected
|
||||
- oversized file rejected
|
||||
- injection marker in SKILL.md rejected
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `flynn doctor` / skill load does not crash on bad skills.
|
||||
- Bad skills become unavailable with clear reasons.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/skills/loader.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 2 — Installer integration
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add scan preflight to `src/skills/installer.ts install()`.
|
||||
- [ ] Add tests for install failure on scan errors.
|
||||
|
||||
Tests:
|
||||
|
||||
- `pnpm test:run src/skills/installer.test.ts` (add if missing)
|
||||
|
||||
---
|
||||
|
||||
### PR 3 — Audit events
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Extend `src/audit/types.ts` to include skill scan events.
|
||||
- [ ] Extend `src/audit/logger.ts` helpers.
|
||||
- [ ] Emit events from loader/install paths.
|
||||
- [ ] Add targeted tests validating event shape (if audit logger has test coverage).
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Audit logs show scan pass/fail with counts and stable issue codes.
|
||||
|
||||
## Final Checks
|
||||
|
||||
- [ ] `pnpm typecheck`
|
||||
- [ ] `pnpm test:run`
|
||||
- [ ] Update `docs/plans/state.json` entry to `completed` once implemented (include test status).
|
||||
@@ -0,0 +1,116 @@
|
||||
# Vercel AI Gateway Provider — Implementation Checklist
|
||||
|
||||
**Date:** 2026-02-15
|
||||
|
||||
**Parent roadmap:** `docs/plans/2026-02-15-openclaw-gap-roadmap.md`
|
||||
|
||||
**Goal:** Close the gap item "Vercel AI Gateway" by adding a first-class model provider that routes through the Vercel AI Gateway using Flynn's existing OpenAI-compatible client path where possible.
|
||||
|
||||
## Scope
|
||||
|
||||
- Add a new provider id (`vercel` or `vercel_ai_gateway`).
|
||||
- Implement the provider via `OpenAIClient` when the gateway is OpenAI-compatible.
|
||||
- Ensure the provider works with:
|
||||
- streaming (if supported by OpenAIClient path)
|
||||
- tool calling
|
||||
- model tier switching via `/model <tier> <provider/model>`
|
||||
|
||||
Non-goals:
|
||||
|
||||
- Implementing gateway-specific “extras” (tracing, metadata) unless required.
|
||||
|
||||
## Design
|
||||
|
||||
### Provider id
|
||||
|
||||
Recommended id: `vercel` (short) or `vercel_ai_gateway` (explicit).
|
||||
|
||||
Pick one and use it consistently in:
|
||||
|
||||
- `MODEL_PROVIDERS` in `src/config/schema.ts`
|
||||
- docs + setup wizard provider list
|
||||
- doctor checks
|
||||
|
||||
### Config fields
|
||||
|
||||
Use the existing `ModelConfig` fields:
|
||||
|
||||
- `endpoint`: base URL of gateway (OpenAI-compatible)
|
||||
- `api_key`: gateway API key (or env var)
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
default:
|
||||
provider: vercel_ai_gateway
|
||||
model: gpt-4.1
|
||||
endpoint: "https://gateway.ai.example.com/v1"
|
||||
api_key: "${VERCEL_AI_GATEWAY_API_KEY}"
|
||||
```
|
||||
|
||||
## PR Breakdown
|
||||
|
||||
### PR 1 — Schema + factory wiring
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add provider id to `MODEL_PROVIDERS` in `src/config/schema.ts`.
|
||||
- [ ] Update `createClientFromConfig()` in `src/daemon/models.ts`:
|
||||
- [ ] map provider -> `new OpenAIClient({ model, apiKey, baseURL })`
|
||||
- [ ] require an API key (config or env var)
|
||||
- [ ] use `cfg.endpoint` as `baseURL` (or a sensible default if the gateway has one)
|
||||
- [ ] Update `/model` strict-tier switching support (should be automatic once provider id is recognized).
|
||||
|
||||
Tests:
|
||||
|
||||
- [ ] Update `src/config/schema.test.ts` to accept the new provider enum.
|
||||
- [ ] Add case to `src/daemon/clientFactory.test.ts`:
|
||||
- asserts the provider returns an OpenAI-compatible client
|
||||
- asserts `baseURL` is passed when `endpoint` is set
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `pnpm typecheck`
|
||||
- `pnpm test:run src/daemon/clientFactory.test.ts src/config/schema.test.ts`
|
||||
|
||||
---
|
||||
|
||||
### PR 2 — Doctor + setup wizard + docs
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Update `src/cli/doctor.ts` provider key checks:
|
||||
- if Vercel gateway requires a key, ensure doctor warns when missing
|
||||
- [ ] Update setup wizard provider picker (optional but recommended):
|
||||
- `src/cli/setup/providers.ts`
|
||||
- `src/cli/setup/providers.test.ts`
|
||||
- [ ] Document provider config in `README.md` (one short snippet; avoid long docs).
|
||||
|
||||
Tests:
|
||||
|
||||
- [ ] `pnpm test:run src/cli/setup/providers.test.ts` (if changed)
|
||||
- [ ] `pnpm test:run src/cli/doctor.test.ts` (if changed)
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `flynn doctor` guidance includes Vercel gateway key/env var info.
|
||||
|
||||
---
|
||||
|
||||
### PR 3 — Integration validation (optional)
|
||||
|
||||
Checklist:
|
||||
|
||||
- [ ] Add a simple smoke test using the `synthetic` provider style or a mocked OpenAI SDK path if possible.
|
||||
- [ ] Ensure streaming works end-to-end via gateway.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- A real config can run `flynn send "hello"` using the gateway provider.
|
||||
|
||||
## Final Checks
|
||||
|
||||
- [ ] `pnpm typecheck`
|
||||
- [ ] `pnpm test:run`
|
||||
- [ ] Update `docs/plans/state.json` entry to `completed` once implemented (include test status).
|
||||
+57
-3
@@ -1,14 +1,68 @@
|
||||
{
|
||||
"version": "1.0",
|
||||
"updated_at": "2026-02-14",
|
||||
"updated_at": "2026-02-15",
|
||||
"description": "Tracks the status of all Flynn plans and implementation phases",
|
||||
|
||||
"plans": {
|
||||
"openclaw-gap-roadmap": {
|
||||
"file": "2026-02-15-openclaw-gap-roadmap.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Roadmap to close remaining OpenClaw gap analysis MISSING items, prioritizing per-tier credential system v2 (API + OAuth/token), Vercel AI Gateway provider, skill safety scanner, elevated mode, Matrix adapter, and deployment targets (Nix + PaaS)."
|
||||
},
|
||||
"credential-system-v2-api-and-oauth": {
|
||||
"file": "2026-02-15-credential-system-v2-api-and-oauth-checklist.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Executable PR-sized checklist to implement per-tier auth_mode with both API-key and OAuth/token credential sources, adding OpenAI API-key storage, Anthropic auth-token storage, CLI/TUI login UX, model factory enforcement, and doctor reporting."
|
||||
},
|
||||
"vercel-ai-gateway-provider": {
|
||||
"file": "2026-02-15-vercel-ai-gateway-provider-checklist.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Executable checklist to add a first-class Vercel AI Gateway model provider (OpenAI-compatible baseURL), including schema/provider registration, model factory wiring, doctor/setup updates, and tests."
|
||||
},
|
||||
"skill-safety-scanner": {
|
||||
"file": "2026-02-15-skill-safety-scanner-checklist.md",
|
||||
"status": "planned",
|
||||
"date": "2026-02-15",
|
||||
"updated": "2026-02-15",
|
||||
"summary": "Executable checklist to implement a static skill/plugin safety scanner integrated into skill load/install paths with audit events and tests, preventing unsafe skill packages from being injected into prompts or used via routing."
|
||||
},
|
||||
"openclaw-style-personal-agent-without-openclaw-risks": {
|
||||
"file": "2026-02-14-openclaw-style-personal-agent-without-openclaw-risks-plan.md",
|
||||
"status": "planned",
|
||||
"status": "completed",
|
||||
"date": "2026-02-14",
|
||||
"summary": "Milestone plan to reach OpenClaw-style personal-assistant efficiency with a safer trust boundary: capability-declared skills, sandbox-by-default for high-risk tools, prompt-injection firewall, secret scoping, and audit logging."
|
||||
"updated": "2026-02-14",
|
||||
"summary": "Implemented safe-by-default personal-agent hardening: capability-declared skills enforced via ToolPolicy + ToolExecutor, skill intent routing with sandbox-by-default for high-risk tools, prompt-injection provenance tags + tool-call guard, secret-scope gating for credentialed tools, and audit log hardening (redaction + correlation IDs).",
|
||||
"files_modified": [
|
||||
"README.md",
|
||||
"config/default.yaml",
|
||||
"docs/api/TOOLS.md",
|
||||
"docs/plans/2026-02-14-openclaw-safe-agent-implementation.md",
|
||||
"src/skills/types.ts",
|
||||
"src/skills/loader.ts",
|
||||
"src/tools/types.ts",
|
||||
"src/tools/policy.ts",
|
||||
"src/tools/executor.ts",
|
||||
"src/backends/native/agent.ts",
|
||||
"src/daemon/routing.ts",
|
||||
"src/daemon/index.ts",
|
||||
"src/daemon/services.ts",
|
||||
"src/cli/skills.ts",
|
||||
"src/cli/setup/security.ts",
|
||||
"src/gateway/handlers/services.ts",
|
||||
"src/audit/types.ts",
|
||||
"src/audit/logger.ts"
|
||||
],
|
||||
"files_created": [
|
||||
"docs/security/SAFE_PERSONAL_AGENT.md",
|
||||
"src/audit/redact.ts"
|
||||
],
|
||||
"test_status": "pnpm test:run + pnpm typecheck passing"
|
||||
},
|
||||
"openclaw-feature-gap-analysis": {
|
||||
"file": "2026-02-06-openclaw-feature-gap-analysis.md",
|
||||
|
||||
Reference in New Issue
Block a user