Commit Graph

310 Commits

Author SHA1 Message Date
William Valentin ee0af0cc06 feat: add tool allow/deny profiles with per-agent and per-provider filtering
Implements configurable tool filtering with four built-in profiles
(minimal, messaging, coding, full), global and per-agent/per-provider
allow/deny lists with glob pattern support, and defense-in-depth
enforcement at both tool listing and execution time.

New: src/tools/policy.ts (ToolPolicy engine), src/tools/policy.test.ts (37 tests)
Modified: config schema, tool registry, tool executor, NativeAgent,
AgentOrchestrator, daemon wiring, gateway tool handler, test mocks
2026-02-06 15:30:34 -08:00
William Valentin 8238d3e981 docs: update state.json with P2 phase entries and test counts 2026-02-06 15:13:14 -08:00
William Valentin 4316dbd3be feat: add P2 features — retry policy, prompt templating, usage tracking, tech debt cleanup
- Extract shared splitMessage() into channels/utils.ts (dedup 4 adapters)
- Add Slack user name resolution with caching (users.info API)
- Add withRetry() with exponential backoff + jitter, isRetryable() filter
- Wire retry config into ModelRouter.chat() (non-streaming only)
- Add assembleSystemPrompt() multi-file template system (SOUL/AGENTS/IDENTITY/USER/TOOLS.md)
- Add usage tracking accumulators in NativeAgent + AgentOrchestrator
- Add estimateCost() with per-model pricing table
- Add /usage TUI command with full usage report formatting
- Add retrySchema and promptSchema to config schema

Tests: 569 passing, typecheck clean
2026-02-06 15:12:35 -08:00
William Valentin de68deb1b2 feat: add WhatsApp channel adapter (Phase 3c) 2026-02-06 14:42:07 -08:00
William Valentin 7a35b22458 feat: wire up all Phase 2-6 features into daemon and config
Integrate all new features into the shared infrastructure:
- Config schema: add memory, discord, slack, process, web_search schemas
- Daemon wiring: memory store init, tool registration, channel adapters
- Orchestrator: memory injection into system prompt, extraction on compaction
- Agent: add setSystemPrompt() for dynamic prompt updates
- Channel/tool index: export new adapters and tool factories
- Add @slack/bolt, discord.js, turndown, linkedom, @mozilla/readability deps
- Update state.json with Phase 3b completion (494 tests passing)
2026-02-06 14:24:39 -08:00
William Valentin 6d9e27a591 feat: enhance web-fetch with HTML-to-markdown extraction (Phase 6)
Add turndown + readability for clean content extraction:
- HTML-to-markdown conversion with smart article extraction
- Format parameter (markdown/text/html)
- Response caching for repeated fetches
- 10 tests
2026-02-06 14:24:28 -08:00
William Valentin 6af26f407c feat: add web search and background process tools (Phases 4-5)
Phase 4 - Web search tool:
- Brave Search API + SearXNG fallback
- Configurable provider, max results
- 14 tests

Phase 5 - Background process management:
- ProcessManager with start/status/output/kill/list tools
- Configurable max concurrent, max runtime, buffer size
- 28 tests
2026-02-06 14:24:23 -08:00
William Valentin eeaec53893 feat: add Slack channel adapter (Phase 3b)
Implement ChannelAdapter for Slack using @slack/bolt with Socket Mode:
- Thread-aware peer IDs (channelId:threadTs)
- Bot message and channel allowlist filtering
- Bot mention stripping (<@U\w+> pattern)
- Message chunking at 4000 chars for readability
- Error handling in connect/disconnect lifecycle
- Typed SlackMessageEvent interface
- 22 tests covering all behaviors
2026-02-06 14:24:17 -08:00
William Valentin 00db84f6a1 feat: add Discord channel adapter (Phase 3a)
Implement ChannelAdapter for Discord using discord.js:
- Bot mention filtering and mention stripping
- Guild and channel allowlist filtering
- Message chunking at 2000 chars
- Reset command detection (!reset / reset in DMs)
- 22 tests covering all behaviors
2026-02-06 14:24:11 -08:00
William Valentin 2e1071230a feat: add persistent memory system (Phase 2)
Implement file-based persistent memory with read/write/search tools:
- MemoryStore with namespace-scoped JSON storage
- memory-read, memory-write, memory-search builtin tools
- Auto-extraction of facts during context compaction
- Configurable via memory.enabled, memory.dir, memory.max_context_tokens
2026-02-06 14:23:59 -08:00
William Valentin 0180d4fb8f docs: add Phase 0/1 implementation plan and feature gap analysis 2026-02-06 13:17:51 -08:00
William Valentin 306e11bd2e feat: add multi-model delegation (Phase 0) and context compaction (Phase 1)
Phase 0 — Multi-Model Delegation:
- AgentOrchestrator wraps NativeAgent with delegate() for stateless
  single-turn calls to any model tier (fast/default/complex/local)
- DelegationConfig maps task types (compaction, classification, etc.)
  to model tiers
- Delegation prompts for compaction, memory extraction, classification,
  and tool summarisation
- Per-tier usage tracking for cost visibility
- Config schema: agents.delegation and agents.primary_tier

Phase 1 — Context Compaction:
- Token estimation (char/4 heuristic) with context window lookup
- shouldCompact() threshold check against context window percentage
- compactHistory() splits old/recent messages, delegates summary to
  fast tier, returns CompactionResult
- Automatic compaction in AgentOrchestrator.process() when configured
- Force-compact via orchestrator.compact() with session persistence
- Session.replaceHistory() with atomic SQLite transaction
- /compact TUI command with feedback on compacted token counts
- Config schema: compaction.enabled, threshold_pct, keep_turns,
  summary_max_tokens

Tests: 385 passing across 50 files (22 new tests in 2 new test files)
2026-02-06 13:17:02 -08:00
William Valentin f7cc87a4bb fix: sync agent tier when /model command switches model
The /model command was only updating the router's currentTier but not
the agent's currentTier. Since NativeAgent.chatWithRouter() passes its
own tier to router.chat(), switching to 'local' still sent requests
through the default (Anthropic) client first.
2026-02-06 10:05:32 -08:00
William Valentin e4b7f96d33 fix: provider-aware model routing with fallback visibility
- Extract createClientFromConfig() to dispatch on provider field instead
  of hardcoding all tiers as AnthropicClient
- Add fallback/fallbackReason metadata to ChatResponse and ChatStreamEvent
  so callers know when a fallback model was used
- Enhance doctor check to report full model stack and warn on missing
  API keys for cloud providers
- Log fallback warnings in NativeAgent and display them in TUI
- Support tier names and local_providers entries in fallback_chain
- Add 8 tests for createClientFromConfig covering all provider types
2026-02-06 09:58:56 -08:00
William Valentin c9b1c607d5 docs: add CLI, cron, and doctor documentation to README; create CHANGELOG
Document all 6 CLI commands with examples, cron scheduling config
format with field reference, doctor diagnostics output and check
details. Update architecture diagram. Add CHANGELOG covering
Phase 5a and the initial 0.1.0 release.
2026-02-06 00:48:52 -08:00
William Valentin c607ff4a4f feat(config): export CronJobConfig type from config index 2026-02-05 22:23:25 -08:00
William Valentin ba15b36e49 feat(daemon): wire CronScheduler into channel registry
Registers CronScheduler as a channel adapter when automation.cron
jobs are configured, enabling scheduled message delivery through
the agent pipeline.
2026-02-05 22:23:03 -08:00
William Valentin b9e008ea23 feat(automation): add CronScheduler channel adapter
Implements CronScheduler as a ChannelAdapter that fires InboundMessages
on cron schedules and routes agent responses to configured output
channels (e.g. Telegram). Includes 9 tests.
2026-02-05 22:22:13 -08:00
William Valentin c4d30fd0d3 feat(cli): implement doctor diagnostics with 10 health checks
Replace doctor stub with full implementation including checks for:
config existence, YAML parsing, schema validation, env vars,
data directory writability, session DB, model config, Telegram,
MCP servers, and skills loading.
2026-02-05 22:20:37 -08:00
William Valentin 826e217dce refactor: retire old entry points, delegate to CLI 2026-02-05 22:18:16 -08:00
William Valentin 1ed9a34715 feat(cli): implement tui command wrapping existing TUI logic 2026-02-05 22:17:44 -08:00
William Valentin 117f3405ce feat(cli): implement config display command 2026-02-05 22:16:58 -08:00
William Valentin 0699730627 feat(cli): implement sessions list command 2026-02-05 22:16:29 -08:00
William Valentin 237246a8cf feat(cli): implement send command for one-shot agent messages 2026-02-05 22:15:46 -08:00
William Valentin 72c75a8bd7 feat(cli): add CLI entry point with commander and start command 2026-02-05 22:14:42 -08:00
William Valentin 6f7b5b8f0f feat(cli): add shared utilities for config loading and output 2026-02-05 22:13:05 -08:00
William Valentin e157bc6102 feat(config): add automation.cron schema for scheduled jobs 2026-02-05 22:12:12 -08:00
William Valentin 69fc4dd531 chore: add commander and croner dependencies 2026-02-05 22:10:44 -08:00
William Valentin 224c023028 docs: add Phase 5a design and implementation plans 2026-02-05 22:10:41 -08:00
William Valentin 7c41ffad71 feat: add skills system for extensible capability packages
Implement a three-tier skill system (bundled/managed/workspace) that
extends Flynn's abilities via SKILL.md instructions injected into the
system prompt.

- SkillManifest/Skill types with requirements gating (OS, binaries, env)
- Loader: discovers skills from directories, validates manifests,
  checks system requirements, infers manifest from SKILL.md if missing
- SkillRegistry: holds skills, generates system prompt additions,
  supports override by name (workspace > managed > bundled)
- SkillInstaller: copies/removes skills in managed directory with
  upgrade support
- Config: add skills.workspace_dir, managed_dir, bundled_dir options
- Daemon: loads all skills at startup, injects available skill
  instructions into the system prompt
- Tests: 45 new tests (loader 22, registry 11, installer 12)
2026-02-05 20:20:03 -08:00
William Valentin cd839c7f0c feat: add MCP integration for external tool servers
Implement Model Context Protocol (MCP) support so Flynn can spawn MCP
server processes, discover their tools, and make them available to the
agent alongside builtin tools.

- McpClient: wraps @modelcontextprotocol/sdk with StdioClientTransport
  for process lifecycle, tool discovery (listTools), and invocation (callTool)
- McpManager: lifecycle management for multiple MCP servers with
  startAll/stopAll/restart, tool bridging into ToolRegistry
- Bridge: converts MCP tools to Flynn Tool interface with mcp:<server>:<tool>
  namespacing to avoid collisions with builtin tools
- Config: add env and cwd fields to mcp server schema
- ToolRegistry: add unregister() method for MCP server cleanup
- Daemon: wire McpManager into startup and shutdown lifecycle
- Tests: 28 new tests (bridge, manager, registry unregister)
2026-02-05 20:10:37 -08:00
William Valentin aa95f2132c feat: add channel adapter abstraction with Telegram and WebChat adapters
Implement Phase 3 channel adapters that decouple message sources from
the agent via a uniform ChannelAdapter interface and ChannelRegistry.

- Add ChannelAdapter/InboundMessage/OutboundMessage types
- Add ChannelRegistry for adapter lifecycle and message routing
- Add TelegramAdapter (grammy bot, auth middleware, confirmations, chunking)
- Add WebChatAdapter (thin shim over GatewayServer)
- Refactor daemon to use ChannelRegistry with per-channel-per-user agents
- Add config.get/config.patch gateway handlers (Phase 2 loose end)
- Add system.restart gateway handler (Phase 2 loose end)
- Add implementation plans and design docs

Tests: 225 passing (33 new channel adapter + gateway handler tests)
2026-02-05 20:00:36 -08:00
William Valentin 282a15d2b9 feat(gateway): add web UI with dashboard and chat interface
Refactor GatewayServer to serve HTTP and WebSocket on a shared
http.Server. Add static file serving with path traversal protection,
a dark-themed dashboard (system health, sessions, tools) and a
WebSocket chat interface with streaming tool events and markdown
rendering.
2026-02-05 19:39:53 -08:00
William Valentin f30a8bc318 feat(gateway): add WebSocket gateway with JSON-RPC protocol and auth
Phase 2 of the Flynn roadmap. Adds a WebSocket gateway server that
starts alongside the Telegram bot, providing real-time API access to
the agent, sessions, and tools.

Protocol: JSON-RPC-like (request/response/event) over WebSocket.
8 methods: agent.send, agent.cancel, sessions.list, sessions.history,
sessions.create, tools.list, tools.invoke, system.health.

Auth: Bearer token + Tailscale identity header support.
Session bridge: per-connection agent instances with shared model router.

New files: src/gateway/ (protocol, router, server, auth, session-bridge,
handlers for agent/sessions/tools/system).
57 new tests (181 total), typecheck clean.
2026-02-05 19:11:25 -08:00
William Valentin ad7fc241f1 feat(telegram): display tool execution status messages
Telegram bot now shows tool status during execution:
- Sends status message when tool starts (tool name + args snippet)
- Edits status message with result on completion
- Keeps typing indicator active during tool execution
- Adds setOnToolUse() to NativeAgent for per-message callback control
2026-02-05 17:53:54 -08:00
William Valentin 5088f7a6be feat(tui): display tool execution status in minimal TUI
TUI now creates a NativeAgent with tool registry/executor and uses
agent.process() for message handling. Tool calls display status lines
showing tool name, args, and success/error result. Falls back to
direct model client when agent is not configured.
2026-02-05 17:53:00 -08:00
William Valentin df92a9d95f feat: add tool capability descriptions to SOUL.md
Lists available tools (shell, file ops, web fetch) so the model knows
what it can do. Keeps SOUL.md as the single source of truth for the
system prompt.
2026-02-05 17:50:47 -08:00
William Valentin b686eb3fa3 test: add end-to-end tool integration tests
Tests the full agent -> tool execution -> model feedback loop:
- Shell tool execution with mock model
- Multi-tool chaining (file.write -> file.read) across iterations
- Verification that tool results are correctly passed back to model
2026-02-05 17:50:17 -08:00
William Valentin b9601b50ab feat(daemon): wire tool registry and executor into agent 2026-02-05 17:49:32 -08:00
William Valentin 4f87643341 feat(agent): add iterative tool use loop with max iterations
Rewrites NativeAgent.process() from single-turn to an iterative tool
loop. When toolRegistry and toolExecutor are provided, the agent calls
the model, executes any requested tool calls, feeds results back, and
loops until the model returns a text response or max iterations hit.

- Backward compatible: works exactly as before without tools
- Supports onToolUse callback for frontend status display
- Max iterations (default 10) prevents infinite loops
- Handles multiple tool calls per model response
- 5 new tests (8 total)
2026-02-05 17:48:38 -08:00
William Valentin 96ade25e98 feat(models): add tool use support to OpenAIClient 2026-02-05 17:44:04 -08:00
William Valentin 36c1cfc768 feat(models): add tool use support to AnthropicClient 2026-02-05 17:44:00 -08:00
William Valentin c96165fb2f feat(tools): add barrel exports and allBuiltinTools list 2026-02-05 17:40:13 -08:00
William Valentin b00706325b feat: add tool framework foundation (types, registry, executor, shell tool, model types, SOUL.md)
- Task 0: SOUL.md + loadSystemPrompt() in daemon
- Task 1: Tool type definitions (Tool, ToolCall, ToolResult, etc.)
- Task 2: ToolRegistry with Anthropic/OpenAI serialization
- Task 3: ToolExecutor with hooks, timeout, truncation
- Task 4: shell.exec builtin tool
- Task 8: Model types updated for tool use (ToolDefinition, ModelToolCall, etc.)
- Task 15: Model index exports for tool types
2026-02-05 17:39:40 -08:00
William Valentin 32dd3ad728 feat(tools): add file read/write/edit/list builtin tools 2026-02-05 17:39:20 -08:00
William Valentin b913941e4f docs: add README and AGENTS.md documentation 2026-02-05 15:51:50 -08:00
William Valentin d2a597d49d fix: add model parameter to LlamaCppClient requests 2026-02-05 15:51:33 -08:00
William Valentin 2fc07b4bca feat: add timestamps to session messages 2026-02-05 15:51:32 -08:00
William Valentin 5558687ab9 feat: wire up num_gpu config and updated clients to daemon and TUI 2026-02-05 15:51:30 -08:00
William Valentin c1f64d6ded feat: enhance TUI with colors, command hints, and improved rendering 2026-02-05 15:51:29 -08:00