Commit Graph

82 Commits

Author SHA1 Message Date
William Valentin 2311826092 feat(metrics): add phase-0 baseline counters
Diagrams reviewed: docs/architecture/AGENT_DIAGRAM.md, docs/architecture/GATEWAY_SESSIONS_AND_QUEUE.md, docs/api/PROTOCOL.md (no changes required).
2026-02-25 09:05:31 -08:00
William Valentin 37be391a40 Unify TUI runtime commands with gateway and harden gateway restart 2026-02-24 13:14:53 -08:00
William Valentin 4188c68130 feat: add persisted manual pi backend mode controls 2026-02-23 23:06:40 -08:00
William Valentin 1dfa6ce2b4 fix(pi): inherit default model and api key for embedded agent 2026-02-23 22:06:42 -08:00
William Valentin 559fe61168 fix(daemon): wire external backends into router deps 2026-02-23 21:54:57 -08:00
William Valentin ac61c9c3fb feat(backends): add optional pi_embedded backend and config wiring 2026-02-23 21:12:52 -08:00
William Valentin 7e480f11fc fix(config): persist runtime patches to active overlay config path 2026-02-18 19:43:55 -08:00
William Valentin cdba111831 fix(confirmations): guarded-action handling across webchat and tui 2026-02-18 17:43:57 -08:00
William Valentin 02fa604c7c feat: add in-chat skill discovery and local registry install command 2026-02-18 10:41:12 -08:00
William Valentin f34a974210 feat: add session-scoped workflow approval gate commands 2026-02-18 10:35:42 -08:00
William Valentin b0d06d0650 chore(rebase): realign duplicate backend/channel/schema files with main 2026-02-17 15:44:47 -08:00
William Valentin 2273ffd020 feat(agent): implement auto-escalation retries and track remediation state 2026-02-17 15:41:41 -08:00
William Valentin 21a7a18008 fix(config): enforce unsupported backend flags and route by modelFor tags 2026-02-17 15:39:34 -08:00
William Valentin 776b47f80f feat: wire agent.delegate tool with sub-agent configs
- Export createAgentDelegateTool through builtin/index.ts → tools/index.ts
- Register agent.delegate in routing.ts with lazy orchestrator pattern
- Add agent.delegate + agents.list to messaging and coding policy profiles
- Add group:agents tool group to policy.ts
- Add research/code/comms agent config examples to default.yaml
- Add research/code/comms agent configs to user config.yaml
- Add 11 tests for agent-delegate tool (all pass)
- Typecheck clean, no regressions
2026-02-17 10:28:29 -08:00
William Valentin 63df791b26 feat(tools): add kubernetes homelab awareness tools 2026-02-16 14:45:45 -08:00
William Valentin 21c986b671 feat(automation): add scheduled minio memory sync jobs 2026-02-16 14:45:45 -08:00
William Valentin 22f225998f feat(tools): add minio prefix sync into memory 2026-02-16 14:45:45 -08:00
William Valentin 0be93c20b5 feat(tools): add minio ingestion into memory namespaces 2026-02-16 14:45:45 -08:00
William Valentin 93621bbe6e feat(analytics): add top tools and topics to session analytics 2026-02-16 14:45:45 -08:00
William Valentin 426145386f feat(tools): add minio.share upload and presigned link tool 2026-02-16 14:45:45 -08:00
William Valentin 8684c3a07d feat(backup): add scheduler alerts and recovery notifications 2026-02-16 14:45:45 -08:00
William Valentin ce621d1b72 feat(automation): add daily briefing preset and cron backup scheduling 2026-02-16 14:45:45 -08:00
William Valentin 01ee6ba53f feat(backup): add MinIO snapshot backups via CLI and scheduler 2026-02-16 13:16:29 -08:00
William Valentin 0220ec10dd feat(config): persist config.patch updates atomically 2026-02-15 22:03:21 -08:00
William Valentin ab89378fce feat(security): enforce elevated mode and sandbox execution 2026-02-15 17:02:05 -08:00
William Valentin 67058c8719 feat(security): harden tool provenance and skill isolation 2026-02-15 10:16:55 -08:00
William Valentin b773e2bbf3 feat(skills): enable watcher wiring through daemon lifecycle 2026-02-12 17:18:22 -08:00
William Valentin a8a2c59313 feat: implement model persistence with per-session overrides
- Add session_config SQLite table for per-session settings
- Update routing to support session override → agent config → global default resolution chain
- Upgrade WebChat SessionBridge from NativeAgent to AgentOrchestrator
- Add /model, /local, /cloud commands to Telegram adapter
- Add /model command to WebChat gateway handlers
- Clear session overrides on /reset command
- Pass memoryStore and config through to SessionBridge
- Add comprehensive tests for all new functionality

Fixes model persistence bug where TUI model changes didn't affect WebChat/Telegram sessions. Now:
- TUI /model sets global default (persists across restarts, affects all new sessions)
- WebChat/Telegram /model sets session override (only that conversation, cleared on /reset)
- WebChat sessions gain AgentOrchestrator features (delegation, compaction, memory)
2026-02-11 21:51:38 -08:00
William Valentin a875bcc4ae feat(audio): add audio.transcribe tool with Whisper-compatible API support
- Add createAudioTranscribeTool with OpenAI/Groq/Ollama/llama.cpp provider support
- Refactor audio config schema to nested audio.enabled + audio.provider structure
- Move audio tool registration to initTools() for conditional enablement
- Fix duplication bug in audio-transcribe.ts URL download handler
- Support base64 data and URL-based audio input with format detection
2026-02-11 18:13:19 -08:00
William Valentin d62e836b5d feat(audit): Add core audit logging infrastructure
- Add AuditLogger class with rotation support
- Add audit configuration to config schema
- Instrument tool execution with full audit logging
- Instrument session lifecycle (create, message, delete, transfer, compact)
- Add audit logger initialization in daemon
- Add cron scheduler audit logging

Audit events captured:
- tool.start/success/error/denied
- session.create/message/delete/transfer/compact
- cron.trigger/add/remove

All logs go to ~/.local/share/flynn/audit.log (JSON lines)
with rotation (10MB files, 30-day retention)
2026-02-11 15:58:07 -08:00
William Valentin f204ff1dd7 feat(tools): add Google Docs, Drive, and Tasks read-only tools
Add three new Google service integrations following the established
Gmail/GCal pattern:

- Google Docs (docs.list, docs.search, docs.read): list, search, and
  read document content as plain text via Docs + Drive APIs
- Google Drive (drive.list, drive.search, drive.read): list, search,
  and read files with export support for Workspace files (Docs→text,
  Sheets→CSV, Slides→text)
- Google Tasks (tasks.lists, tasks.list): list task lists and tasks
  with status, due dates, and notes

Each service has its own config section, OAuth auth command, tool
policy group, and test suite (53 new tests). The setup wizard now
offers to configure all Google services together and run OAuth auth
flows automatically after saving config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:59:15 -08:00
William Valentin 411c6d84a2 feat(tui): persist model tier selection and fix formatting
Persist /model tier choice to ~/.local/share/flynn/preferences.json so
it survives restarts. Decode HTML entities (e.g. &#39;) in markdown
renderer output. Suppress noisy logger.info and punycode deprecation
warnings in TUI startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:23:12 -08:00
William Valentin 94264e848c feat(tools): add Google Calendar tools and register Gmail/GCal in daemon
Add calendar.today, calendar.list, calendar.search tools mirroring the
Gmail tool pattern. Includes gcal-auth CLI command, config schema, tool
policy entries (messaging/coding profiles + group:gcal), and 17 tests.
Also wires up gmail and gcal tool registration in the daemon and TUI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:40:53 -08:00
William Valentin 62331c3a09 feat(daemon): wire PairingStore from SessionStore into PairingManager 2026-02-09 21:49:55 -08:00
William Valentin 35f4cab0dc feat: add log-level system to suppress noisy fallback debug output
Replace console.debug/log/warn calls in model router, retry, and daemon
startup with a structured logger that respects a configurable log_level.
Default level is 'info', suppressing verbose fallback debug messages in
the TUI while keeping them available via config when needed.

- Add src/logger.ts with debug/info/warn/error/silent levels
- Wire log_level into config schema (default: 'info')
- Initialize log level in both daemon and TUI startup paths
- Convert all console.debug in router.ts and retry.ts to logger.debug
- Convert console.log/warn in daemon/models.ts to logger.info/warn
2026-02-09 21:23:07 -08:00
William Valentin 701fcfcaed refactor(01-03): extract services/skills/gateway/mcp into services.ts, reduce index.ts to 140 lines
- Extract initSkills(), initMcp(), loadSystemPrompt(), initPairingManager(), createGateway(), startServices() into services.ts
- daemon/index.ts reduced from 386 to 140 lines (64% reduction, 87% from 1087 baseline)
- Organize imports with section comments (External, Config, Daemon Modules, Infrastructure)
- Add section dividers in startDaemon() (Data & Sessions, Core Services, Model & Prompt, Gateway & Channels, Tier 1 Tools, Lifecycle)
- Convert unused value imports to type-only imports
- DaemonContext interface and re-exports unchanged
2026-02-09 20:22:34 -08:00
William Valentin 35a0061de9 feat(01-02): extract channel adapter registration into src/daemon/channels.ts
- Move Telegram, Discord, Slack, WhatsApp, WebChat adapter setup to channels.ts
- Move CronScheduler, WebhookHandler, GmailWatcher registration to channels.ts
- Clean up index.ts imports (remove unused adapter value imports)
- index.ts calls registerChannels() and receives cronScheduler for tool wiring
2026-02-09 20:14:23 -08:00
William Valentin fb1199a1da refactor(01-01): extract tool registration into src/daemon/tools.ts
- Create initTools() factory encapsulating ToolRegistry, allBuiltinTools, web search tools, ProcessManager, BrowserManager, ToolExecutor, and ToolPolicy
- Replace ~70 lines of inline tool setup in startDaemon() with single initTools() call
- Clean up tool-specific imports from daemon/index.ts (ToolPolicy, allBuiltinTools, createWebSearchTools, createProcessTools, ProcessManager, createBrowserTools)
- Tier 1 agent tools (session, agents list, message send, cron) remain in daemon/index.ts as intended
- daemon/index.ts reduced to 457 lines (from 1088 baseline)
2026-02-09 20:12:46 -08:00
William Valentin efceb38cb6 feat(01-02): extract agent config and sandbox setup into src/daemon/agents.ts
- Create initAgents() function encapsulating AgentConfigRegistry, AgentRouter, SandboxManager init
- Replace ~26 lines in startDaemon() with single initAgents() call
- Lifecycle shutdown handler for sandbox cleanup included in agents.ts
- Zero type errors, routing tests pass
2026-02-09 20:11:32 -08:00
William Valentin 00f8f74aac refactor(01-01): extract memory initialization into src/daemon/memory.ts
- Create initMemory() factory encapsulating MemoryStore, VectorStore, HybridSearch, background indexer, and memory tools registration
- Replace ~65 lines of inline memory init in startDaemon() with single initMemory() call
- Clean up memory-specific imports from daemon/index.ts (MemoryStore, VectorStore, HybridSearch, createEmbeddingProvider, chunkText, contentHash, createMemoryTools)
2026-02-09 20:10:49 -08:00
William Valentin 08f5b6b8e7 feat(01-02): extract message routing into src/daemon/routing.ts
- Move createMessageRouter function (~220 lines) to dedicated routing module
- Add import from ./routing.js in daemon/index.ts
- routing.test.ts passes without modification
- Zero type errors
2026-02-09 20:09:28 -08:00
William Valentin 86cda91f6b refactor(01-01): extract model client logic into src/daemon/models.ts
- Move createClientFromConfig, anthropicToGitHubModel, createAutoFallbackClient, createModelRouter to dedicated module
- Add re-exports from daemon/index.ts for backward compatibility
- clientFactory.test.ts passes without modification
- Reduces daemon/index.ts by ~248 lines
2026-02-09 20:06:27 -08:00
William Valentin 1e29da4da2 feat: complete DM pairing codes with channel adapters, gateway handlers, and TUI command (Tier 4 feature 4) 2026-02-09 18:28:10 -08:00
William Valentin 4413c4dc7c feat: add gateway lock, shell completion, and tailscale serve (Tier 4 features 1-3) 2026-02-09 13:29:59 -08:00
William Valentin 9be8f76bc7 feat: implement Tier 3 features — lane queue, credential redaction, token dashboard, xAI, Voyage AI
- Lane Queue: per-session FIFO queue in gateway replacing reject-when-busy (9 tests)
- Credential Redaction: redactConfig() expanded to cover 18+ secret fields (16 tests)
- Web UI Token Dashboard: system.tokenUsage endpoint + Usage page with summary cards
- xAI (Grok) Provider: OpenAI-compatible client with model pricing
- Voyage AI Embeddings: new embedding provider with configurable dimensions (5 tests)
- Update gap analysis: 90→95 match (70%→74%), Tier 3 section marked DONE
- Update state.json: test count 1001→1034, add tier3_completion entry

Total: 1034 tests passing across 85 files, typecheck clean
2026-02-09 10:32:57 -08:00
William Valentin 1d126cddfb feat: add Zhipu AI (GLM) model provider support
Adds zhipuai as a new provider using the OpenAI-compatible API at
api.z.ai. Supports api_key config or ZHIPUAI_API_KEY env var, with
optional endpoint override.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:55:13 -08:00
William Valentin 06438bb44f feat: add Gmail Pub/Sub watcher for inbound email automation
New ChannelAdapter that monitors Gmail via Google Cloud Pub/Sub push
notifications with polling fallback. Supports OAuth2 auth, configurable
watch labels, template rendering with email metadata placeholders
(from, to, subject, snippet, date, id, labels).

Wired into daemon lifecycle and gateway (POST /gmail/push endpoint).
Includes 16 tests covering auth, templates, push notifications, and
channel routing.
2026-02-07 15:39:24 -08:00
William Valentin 88731a50e3 feat: add heartbeat monitor and vector memory search (Tier 2)
Heartbeat:
- HeartbeatMonitor with 5 checks: gateway, model, channels, memory, disk
- Configurable interval, failure threshold, notification channel
- Recovery notifications when health restores
- 25 new tests

Vector Memory Search:
- EmbeddingProvider interface with OpenAI, Gemini, Ollama, LlamaCpp backends
- SQLite-backed VectorStore with cosine similarity search
- Text chunker with paragraph-aware splitting and overlap
- HybridSearch merging keyword + vector results with configurable weight
- Background indexer with dirty-namespace tracking
- Graceful fallback to keyword search when embeddings unavailable
- 51 new tests

Config: automation.heartbeat + memory.embedding schema sections
Total: 950 tests passing, all types clean
2026-02-07 14:45:11 -08:00
William Valentin b50c140d25 feat: add Docker support and inbound webhooks (Tier 2)
- Dockerfile: multi-stage build (node:22-alpine), better-sqlite3 native deps handled
- .dockerignore + docker-compose.yml for deployment
- FLYNN_DATA_DIR env var support in daemon, CLI, and TUI
- WebhookHandler: ChannelAdapter for HTTP POST /webhooks/:name
- Per-webhook HMAC auth, template rendering ({{body}}, {{json.field}})
- Config schema: automation.webhooks array with name/secret/message/output
- Gateway routes webhook requests before static files (bypasses gateway auth)
- 23 new tests for webhook functionality, 874 total tests passing
2026-02-07 14:36:05 -08:00
William Valentin b322e8f29c fix: GitHub Copilot fallback — remove stale API version header and fix model name mapping
Two issues prevented the GitHub Models fallback from working:

1. The X-GitHub-Api-Version: 2022-11-28 header caused '400 invalid
   apiVersion' errors. The Copilot chat completions endpoint does not
   use this header — removed from both constructor and rebuildClient.

2. The anthropicToGitHubModel mapping was incomplete: it only knew
   three models and the generic date-stripping fallback produced wrong
   names (e.g. 'claude-sonnet-4-5' instead of 'claude-sonnet-4.5').
   GitHub Copilot uses dots for sub-versions, not hyphens.

   Updated with explicit mappings for all current models (sonnet 4,
   4.5; opus 4, 4.5, 4.6; haiku 4.5) and a smarter generic fallback
   that converts digit-hyphen-digit to digit.digit at the end.

3. createClientFromConfig now auto-maps Anthropic-style model names
   when the provider is 'github', so users can copy model names from
   their Anthropic config into fallback blocks without manual renaming.
2026-02-07 14:04:54 -08:00