Commit Graph

437 Commits

Author SHA1 Message Date
William Valentin f7d889e35e docs: update state.json with P6 completion and overall progress 2026-02-07 09:11:49 -08:00
William Valentin 2a962abcd0 feat: add audio transcription pipeline for voice messages
Adds Whisper-compatible audio transcription via configurable endpoint.
New functions: isSupportedAudio(), mimeToExtension(), transcribeAudio(),
buildUserMessageWithAudio(). Config schema gains audio section with
transcription_endpoint, api_key, and model. Daemon wires transcription
into the message router. Channel adapters extract audio from voice/audio
messages (Telegram voice+audio, Discord audio/*, Slack audio/*, WhatsApp
ptt+audio). Includes 57 media tests (was 25, now covers all audio paths).
2026-02-07 09:09:13 -08:00
William Valentin e052778b0a feat: add gateway protocol attachment support
Extends the gateway wire protocol with GatewayAttachment type and
attachment event. agent.send handler now accepts optional attachments
parameter and converts them for the agent pipeline. Includes 5 new
tests for protocol and handler layers.
2026-02-07 09:09:06 -08:00
William Valentin b9bfee9c5b feat: add outbound attachment support with media.send tool
Introduces OutboundAttachment type on OutboundMessage, an
OutboundAttachmentCollector (push/drain pattern), and a media.send
tool that queues files for outbound delivery. Each channel adapter
(Telegram, Discord, Slack, WhatsApp) sends attachments after the
text reply. Includes 15 tests for collector and tool.
2026-02-07 09:09:00 -08:00
William Valentin 1e6f6bb5a4 feat: add image.analyze tool for vision model analysis
Provides a factory createImageAnalyzeTool(modelClient) that sends images
to a vision-capable model and returns a textual analysis. Includes 15
tests covering base64, URL, multi-image, error, and edge cases.
2026-02-07 09:08:53 -08:00
William Valentin d4530a7034 feat: add runtime provider/model switching via /model <tier> <provider/model>
- ModelRouter: add setClient(), labels map, getLabel(), getAllLabels()
- TUI commands: parse /model <tier> <provider/model> syntax with autocompletion
- TUI minimal: handle provider switching via createClientFromConfig factory
- Daemon: wire initial labels into router config
- Fix /model alias mappings (opus=complex, sonnet=default, haiku=fast)
- Add design doc and update state.json with feature status
2026-02-06 23:42:14 -08:00
William Valentin e92ce69067 docs: add commit-often and subagent model selection rules to AGENTS.md 2026-02-06 23:42:06 -08:00
William Valentin 73fc5d173d feat: add auto-login for GitHub Copilot when no token is available
GitHubModelsClient now lazily resolves tokens at first API call. If no
token exists (env var, stored OAuth, or config), it triggers the OAuth
device flow automatically via an onLoginRequired callback wired in both
the TUI and daemon entry points.
2026-02-06 22:33:48 -08:00
William Valentin f363717f5f feat: add GitHub Copilot model provider with OAuth device flow
Add a new 'github' model provider backed by the Copilot API
(api.githubcopilot.com), with OAuth device flow for authentication.

- New src/auth/github.ts: device flow login, token storage at
  ~/.config/flynn/auth.json with 0600 permissions
- New src/models/github.ts: OpenAI-compatible client with streaming,
  tool calling, and Copilot-specific headers
- Add 'github' to provider enum in config schema
- Register provider in daemon factory and TUI client factory
- Refactor TUI to use provider-agnostic client factory (was hardcoded
  to AnthropicClient for all tiers)
- Add /login command to TUI for interactive OAuth authorization
- Add Copilot model cost tracking entries
2026-02-06 22:26:52 -08:00
William Valentin a515912537 feat: add multimodal media pipeline for image support across all providers and channels
Widen Message.content from string to string | MessageContentPart[] to support
multimodal content. Add Attachment type to channel layer, media conversion
utilities, and image extraction to all channel adapters (Telegram, Discord,
Slack, WhatsApp). Update all model clients (Anthropic, OpenAI, Gemini, Bedrock)
to convert structured content to provider-specific formats. Fix downstream
consumers (tokens, compaction, TUI, local models) to handle the widened type
via getMessageText() helper.
2026-02-06 17:17:21 -08:00
William Valentin cfdd448495 docs: add Docker sandbox and multi-agent routing design/implementation plans 2026-02-06 16:52:38 -08:00
William Valentin 880744846f feat: wire new providers, auth, mention-gating, and browser into daemon
Update config schema with server auth fields (token, tailscale_identity,
auth_http), channel mention settings, browser config, and openrouter/bedrock
provider enum values. Wire GeminiClient, BedrockClient, OpenRouter into
createClientFromConfig. Initialize BrowserManager and register browser tools
in daemon startup. Pass auth config and channel mention settings through to
gateway and adapters. Add puppeteer-core, @google/generative-ai, and
@aws-sdk/client-bedrock-runtime dependencies.
2026-02-06 16:52:18 -08:00
William Valentin 8c56a5a1a8 feat: add Chrome DevTools Protocol browser tools
Add BrowserManager (puppeteer-core) with page pool and auto-detection of
Chrome/Chromium. Six tools: browser.navigate, browser.screenshot,
browser.click, browser.type, browser.content, browser.eval. Feature is
opt-in (browser.enabled defaults to false). Add to coding tool profile.
Includes 22 unit tests for manager and all tools.
2026-02-06 16:52:03 -08:00
William Valentin 647d7779c7 feat: add group chat and mention-gating to channel adapters
Slack: add requireMention option, resolve bot user ID on connect.
Telegram: add group chat mention/reply-to-bot detection, strip @mention
from message text, default requireMention=true for groups.
WhatsApp: add allowedGroupIds for group chat support, mention detection
via mentionedIds and body text, strip bot mention from messages.
2026-02-06 16:51:52 -08:00
William Valentin 20930a4816 feat: add query-param token auth and optional HTTP auth to gateway
Support ?token= query parameter as a fallback for WebSocket clients that
cannot set Authorization headers (e.g. browsers). Add authHttp option to
GatewayServer so token auth can be applied to HTTP requests too, returning
401 with WWW-Authenticate header on failure.
2026-02-06 16:51:41 -08:00
William Valentin 0eb1f7a073 feat: add Gemini and Bedrock model providers
Add native GeminiClient using @google/generative-ai SDK and BedrockClient
using @aws-sdk/client-bedrock-runtime. Replace the previous Gemini fallback
(OpenAI-compatible shim) with the real implementation. Add OpenRouter as a
provider option (OpenAI-compatible with custom baseURL). Update model costs,
doctor CLI checks, and client factory tests.
2026-02-06 16:51:32 -08:00
William Valentin e8e4fcd758 docs: update state.json with Docker sandbox and multi-agent routing 2026-02-06 16:04:51 -08:00
William Valentin 4dfa242716 feat: wire Docker sandboxing and agent routing into daemon 2026-02-06 16:04:14 -08:00
William Valentin fecf02acd1 feat: add agents barrel export 2026-02-06 15:58:56 -08:00
William Valentin 0d5601fb13 feat: add SandboxManager for per-session container lifecycle 2026-02-06 15:58:34 -08:00
William Valentin 1314ac0163 feat: add ToolRegistry.clone() and replace() for per-session registries 2026-02-06 15:58:19 -08:00
William Valentin ed1e290ddd feat: add AgentRouter for config-based sender/channel routing 2026-02-06 15:53:50 -08:00
William Valentin 7cb5287ed3 feat: add DockerSandbox class for container lifecycle 2026-02-06 15:53:50 -08:00
William Valentin d65ce078b7 feat: add AgentConfigRegistry for named agent configurations 2026-02-06 15:52:58 -08:00
William Valentin 430cb3f96e feat: add sandboxed tool wrappers for shell.exec and process.start 2026-02-06 15:52:21 -08:00
William Valentin daf8cac3fe feat: add sandbox, agent_configs, and routing config schemas 2026-02-06 15:48:55 -08:00
William Valentin e4eea764e7 docs: update state.json with tool allow/deny profiles and test count (606) 2026-02-06 15:31:04 -08:00
William Valentin ee0af0cc06 feat: add tool allow/deny profiles with per-agent and per-provider filtering
Implements configurable tool filtering with four built-in profiles
(minimal, messaging, coding, full), global and per-agent/per-provider
allow/deny lists with glob pattern support, and defense-in-depth
enforcement at both tool listing and execution time.

New: src/tools/policy.ts (ToolPolicy engine), src/tools/policy.test.ts (37 tests)
Modified: config schema, tool registry, tool executor, NativeAgent,
AgentOrchestrator, daemon wiring, gateway tool handler, test mocks
2026-02-06 15:30:34 -08:00
William Valentin 8238d3e981 docs: update state.json with P2 phase entries and test counts 2026-02-06 15:13:14 -08:00
William Valentin 4316dbd3be feat: add P2 features — retry policy, prompt templating, usage tracking, tech debt cleanup
- Extract shared splitMessage() into channels/utils.ts (dedup 4 adapters)
- Add Slack user name resolution with caching (users.info API)
- Add withRetry() with exponential backoff + jitter, isRetryable() filter
- Wire retry config into ModelRouter.chat() (non-streaming only)
- Add assembleSystemPrompt() multi-file template system (SOUL/AGENTS/IDENTITY/USER/TOOLS.md)
- Add usage tracking accumulators in NativeAgent + AgentOrchestrator
- Add estimateCost() with per-model pricing table
- Add /usage TUI command with full usage report formatting
- Add retrySchema and promptSchema to config schema

Tests: 569 passing, typecheck clean
2026-02-06 15:12:35 -08:00
William Valentin de68deb1b2 feat: add WhatsApp channel adapter (Phase 3c) 2026-02-06 14:42:07 -08:00
William Valentin 7a35b22458 feat: wire up all Phase 2-6 features into daemon and config
Integrate all new features into the shared infrastructure:
- Config schema: add memory, discord, slack, process, web_search schemas
- Daemon wiring: memory store init, tool registration, channel adapters
- Orchestrator: memory injection into system prompt, extraction on compaction
- Agent: add setSystemPrompt() for dynamic prompt updates
- Channel/tool index: export new adapters and tool factories
- Add @slack/bolt, discord.js, turndown, linkedom, @mozilla/readability deps
- Update state.json with Phase 3b completion (494 tests passing)
2026-02-06 14:24:39 -08:00
William Valentin 6d9e27a591 feat: enhance web-fetch with HTML-to-markdown extraction (Phase 6)
Add turndown + readability for clean content extraction:
- HTML-to-markdown conversion with smart article extraction
- Format parameter (markdown/text/html)
- Response caching for repeated fetches
- 10 tests
2026-02-06 14:24:28 -08:00
William Valentin 6af26f407c feat: add web search and background process tools (Phases 4-5)
Phase 4 - Web search tool:
- Brave Search API + SearXNG fallback
- Configurable provider, max results
- 14 tests

Phase 5 - Background process management:
- ProcessManager with start/status/output/kill/list tools
- Configurable max concurrent, max runtime, buffer size
- 28 tests
2026-02-06 14:24:23 -08:00
William Valentin eeaec53893 feat: add Slack channel adapter (Phase 3b)
Implement ChannelAdapter for Slack using @slack/bolt with Socket Mode:
- Thread-aware peer IDs (channelId:threadTs)
- Bot message and channel allowlist filtering
- Bot mention stripping (<@U\w+> pattern)
- Message chunking at 4000 chars for readability
- Error handling in connect/disconnect lifecycle
- Typed SlackMessageEvent interface
- 22 tests covering all behaviors
2026-02-06 14:24:17 -08:00
William Valentin 00db84f6a1 feat: add Discord channel adapter (Phase 3a)
Implement ChannelAdapter for Discord using discord.js:
- Bot mention filtering and mention stripping
- Guild and channel allowlist filtering
- Message chunking at 2000 chars
- Reset command detection (!reset / reset in DMs)
- 22 tests covering all behaviors
2026-02-06 14:24:11 -08:00
William Valentin 2e1071230a feat: add persistent memory system (Phase 2)
Implement file-based persistent memory with read/write/search tools:
- MemoryStore with namespace-scoped JSON storage
- memory-read, memory-write, memory-search builtin tools
- Auto-extraction of facts during context compaction
- Configurable via memory.enabled, memory.dir, memory.max_context_tokens
2026-02-06 14:23:59 -08:00
William Valentin 0180d4fb8f docs: add Phase 0/1 implementation plan and feature gap analysis 2026-02-06 13:17:51 -08:00
William Valentin 306e11bd2e feat: add multi-model delegation (Phase 0) and context compaction (Phase 1)
Phase 0 — Multi-Model Delegation:
- AgentOrchestrator wraps NativeAgent with delegate() for stateless
  single-turn calls to any model tier (fast/default/complex/local)
- DelegationConfig maps task types (compaction, classification, etc.)
  to model tiers
- Delegation prompts for compaction, memory extraction, classification,
  and tool summarisation
- Per-tier usage tracking for cost visibility
- Config schema: agents.delegation and agents.primary_tier

Phase 1 — Context Compaction:
- Token estimation (char/4 heuristic) with context window lookup
- shouldCompact() threshold check against context window percentage
- compactHistory() splits old/recent messages, delegates summary to
  fast tier, returns CompactionResult
- Automatic compaction in AgentOrchestrator.process() when configured
- Force-compact via orchestrator.compact() with session persistence
- Session.replaceHistory() with atomic SQLite transaction
- /compact TUI command with feedback on compacted token counts
- Config schema: compaction.enabled, threshold_pct, keep_turns,
  summary_max_tokens

Tests: 385 passing across 50 files (22 new tests in 2 new test files)
2026-02-06 13:17:02 -08:00
William Valentin f7cc87a4bb fix: sync agent tier when /model command switches model
The /model command was only updating the router's currentTier but not
the agent's currentTier. Since NativeAgent.chatWithRouter() passes its
own tier to router.chat(), switching to 'local' still sent requests
through the default (Anthropic) client first.
2026-02-06 10:05:32 -08:00
William Valentin e4b7f96d33 fix: provider-aware model routing with fallback visibility
- Extract createClientFromConfig() to dispatch on provider field instead
  of hardcoding all tiers as AnthropicClient
- Add fallback/fallbackReason metadata to ChatResponse and ChatStreamEvent
  so callers know when a fallback model was used
- Enhance doctor check to report full model stack and warn on missing
  API keys for cloud providers
- Log fallback warnings in NativeAgent and display them in TUI
- Support tier names and local_providers entries in fallback_chain
- Add 8 tests for createClientFromConfig covering all provider types
2026-02-06 09:58:56 -08:00
William Valentin c9b1c607d5 docs: add CLI, cron, and doctor documentation to README; create CHANGELOG
Document all 6 CLI commands with examples, cron scheduling config
format with field reference, doctor diagnostics output and check
details. Update architecture diagram. Add CHANGELOG covering
Phase 5a and the initial 0.1.0 release.
2026-02-06 00:48:52 -08:00
William Valentin c607ff4a4f feat(config): export CronJobConfig type from config index 2026-02-05 22:23:25 -08:00
William Valentin ba15b36e49 feat(daemon): wire CronScheduler into channel registry
Registers CronScheduler as a channel adapter when automation.cron
jobs are configured, enabling scheduled message delivery through
the agent pipeline.
2026-02-05 22:23:03 -08:00
William Valentin b9e008ea23 feat(automation): add CronScheduler channel adapter
Implements CronScheduler as a ChannelAdapter that fires InboundMessages
on cron schedules and routes agent responses to configured output
channels (e.g. Telegram). Includes 9 tests.
2026-02-05 22:22:13 -08:00
William Valentin c4d30fd0d3 feat(cli): implement doctor diagnostics with 10 health checks
Replace doctor stub with full implementation including checks for:
config existence, YAML parsing, schema validation, env vars,
data directory writability, session DB, model config, Telegram,
MCP servers, and skills loading.
2026-02-05 22:20:37 -08:00
William Valentin 826e217dce refactor: retire old entry points, delegate to CLI 2026-02-05 22:18:16 -08:00
William Valentin 1ed9a34715 feat(cli): implement tui command wrapping existing TUI logic 2026-02-05 22:17:44 -08:00
William Valentin 117f3405ce feat(cli): implement config display command 2026-02-05 22:16:58 -08:00
William Valentin 0699730627 feat(cli): implement sessions list command 2026-02-05 22:16:29 -08:00