Commit Graph

716 Commits

Author SHA1 Message Date
William Valentin f6dedf0fbe fix(tui): register Google Calendar tools when gcal is enabled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:45:05 -08:00
William Valentin 55d35c80b4 feat(tui): improve tool use display and register Gmail tools
Format tool names as human-readable labels (e.g. "Gmail: List") and
show args as compact key-value pairs instead of raw JSON. Also register
Gmail tools in the TUI when automation.gmail is enabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:41:35 -08:00
William Valentin 796e143d61 fix(agent): inject tool inventory note when tools change mid-session
Stale session history can cause the model to follow old "I can't do
that" patterns even when new tools are available. NativeAgent now tracks
a tool fingerprint and appends a system prompt note listing current
tools when the inventory changes, resetting on session reset.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:41:31 -08:00
William Valentin 94264e848c feat(tools): add Google Calendar tools and register Gmail/GCal in daemon
Add calendar.today, calendar.list, calendar.search tools mirroring the
Gmail tool pattern. Includes gcal-auth CLI command, config schema, tool
policy entries (messaging/coding profiles + group:gcal), and 17 tests.
Also wires up gmail and gcal tool registration in the daemon and TUI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:40:53 -08:00
William Valentin 4cc29f534a fix(tui): render inline markdown formatting with ANSI codes
Block-level renderer methods (paragraph, heading, blockquote, list) were
using raw token.text instead of this.parser.parseInline(tokens), causing
bold, italic, and inline code to never render. Add table renderer with
aligned columns and box-drawing separators. Remove unused marked-terminal
dependency (incompatible with marked v17).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:29:57 -08:00
William Valentin ff03f74404 feat(cli): add gmail-auth command for OAuth2 token setup
Implements `flynn gmail-auth` to complete the OAuth2 flow that
GmailWatcher references but was never built. Supports local callback
server (default) and --manual paste mode. Adds Gmail health check
to `flynn doctor`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 10:33:01 -08:00
William Valentin f4b9c850ab feat(setup): add contextual help text to all wizard flows
Each setup section now explains what's needed before prompting:
- Providers: links to API key consoles (Anthropic, OpenAI, Gemini, etc.)
- Channels: step-by-step bot creation (Telegram @BotFather, Discord dev
  portal, Slack app setup, WhatsApp QR)
- Gmail: Google Cloud Console OAuth setup walkthrough
- Memory: explains what vector search does and key reuse
- Security: describes each option (sandbox, pairing, tool profiles)
- Gateway: explains auth token, Tailscale Serve, lock mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 10:08:44 -08:00
William Valentin 7620616c7c test(setup): add integration tests and update shell completion
Adds comprehensive integration tests for the first-run wizard verifying config
generation for different provider/channel combinations. Updates shell completion
to include the 'setup' command with its options.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:38:53 -08:00
William Valentin f50d7d69fb feat(setup): wire setup command into CLI and start command
- Register setup command in CLI index
- Offer setup wizard when running `flynn start` with no config
- Guard telegram log output since telegram is now optional

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:36:58 -08:00
William Valentin d8b7b08270 feat(setup): add main orchestrator, menu, and CLI command
Implements Task 6 of the setup wizard:
- orchestrator.ts: runMenu() for interactive configuration loop
- orchestrator.ts: runFirstRunWizard() for new user onboarding
- orchestrator.test.ts: test for menu exit behavior
- setup.ts: registerSetupCommand() and runSetup() handler
  - Handles both first-run and existing config scenarios
  - Saves YAML config to disk
  - Optional daemon startup after first-run

All tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:35:32 -08:00
William Valentin 182d86957b feat(setup): add memory, automation, security, and gateway setup flows 2026-02-10 09:34:04 -08:00
William Valentin b673632b0f feat(setup): add channel setup flows
Implement setupChannels function with support for Telegram, Discord, Slack, and WhatsApp.
Includes WebChat gateway configuration and channel choice loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:32:52 -08:00
William Valentin 573cb43534 feat(setup): add model provider setup flows 2026-02-10 09:31:43 -08:00
William Valentin d35ce2beb5 feat(setup): add config builder and summary renderer
Add ConfigBuilder class to accumulate wizard answers into config objects with YAML
serialization, and renderSummary function to display configuration summary. Includes
9 test cases covering provider setup, channel configuration, and feature flags.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:29:56 -08:00
William Valentin 9cc03187b0 feat(setup): add prompt helpers for setup wizard
Created a Prompter interface and factory function for interactive CLI prompts:
- ask(): text input with optional default values
- confirm(): yes/no confirmation with default
- choose(): numbered menu selection with fallback
- password(): text input (no echo planned in TUI)
- println(): simple output helper

All 9 tests pass (ask, confirm, choose, password scenarios).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:28:19 -08:00
William Valentin 213dba855a refactor: make telegram config optional for non-telegram setups
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:27:18 -08:00
William Valentin 64e3716ede test(tui): add /pair command parsing tests 2026-02-09 21:56:27 -08:00
William Valentin 3ea4f64d6b feat(tui): wire /pair command execution with PairingManager 2026-02-09 21:56:27 -08:00
William Valentin 322852917c fix(tui): remove stale readline close listeners to prevent memory leak warning
Clean up the once('close') listener on the readline Interface when
rl.question() resolves normally. Previously, each prompt loop iteration
accumulated a close listener that was never removed, triggering
MaxListenersExceededWarning after 11 prompts.
2026-02-09 21:50:43 -08:00
William Valentin 62331c3a09 feat(daemon): wire PairingStore from SessionStore into PairingManager 2026-02-09 21:49:55 -08:00
William Valentin ecd3aca7c1 feat(session): add pairing_approved table and getPairingStore() 2026-02-09 21:46:51 -08:00
William Valentin 1e1a68924e feat(pairing): add PairingStore interface for persistence injection 2026-02-09 21:45:04 -08:00
William Valentin c3ca3f3776 feat(03-02): extend dashboard with live ops sections
- Core counters: messages processed, sessions, queue depth, uptime, active requests, errors
- Model performance table: recent calls with latency, tokens/sec, provider, status
- Event stream: scrollable log with color-coded levels (error/warn/info)
- Active requests: in-flight request table with session, channel, duration
- Channels grid: existing channel status cards preserved
- Dual timer refresh: 3s for metrics/events/requests, 10s for health/channels
- Targeted DOM updates via getElementById for flicker-free fast updates
2026-02-09 21:34:11 -08:00
William Valentin a0feff9637 feat(03-01): hook metrics recording into agent request flow
- Track active requests with startRequest/endRequest around lane queue work
- Increment messagesProcessed on successful agent.process completion
- Record errors and error events on agent.send failures
- Record tool failure events with tool name and error details
2026-02-09 21:29:14 -08:00
William Valentin bd1880a44c feat(03-01): create MetricsCollector and wire into gateway
- Add MetricsCollector class with counters, model call ring buffer, event ring buffer, and active request tracking
- Add system.metrics, system.events, system.activeRequests RPC handlers
- Add GET /health unauthenticated HTTP endpoint for Docker HEALTHCHECK
- Add totalPending() to LaneQueue for queue depth metrics
- Add 20 tests for MetricsCollector
2026-02-09 21:28:05 -08:00
William Valentin 35f4cab0dc feat: add log-level system to suppress noisy fallback debug output
Replace console.debug/log/warn calls in model router, retry, and daemon
startup with a structured logger that respects a configurable log_level.
Default level is 'info', suppressing verbose fallback debug messages in
the TUI while keeping them available via config when needed.

- Add src/logger.ts with debug/info/warn/error/silent levels
- Wire log_level into config schema (default: 'info')
- Initialize log level in both daemon and TUI startup paths
- Convert all console.debug in router.ts and retry.ts to logger.debug
- Convert console.log/warn in daemon/models.ts to logger.info/warn
2026-02-09 21:23:07 -08:00
William Valentin 6bd372162e feat(02-02): add overlay file validation to flynn doctor
- Import resolveOverlayPath from shared.ts
- Add checkOverlayExists check (skip when no FLYNN_ENV, pass/fail for overlay file)
- Insert after checkConfigExists in allChecks array
- All 1087 tests pass, typecheck clean
2026-02-09 21:00:18 -08:00
William Valentin 29bc18502f feat(02-01): wire FLYNN_ENV resolution into shared.ts with overlay-aware loadConfigSafe
- Add resolveOverlayPath() that maps FLYNN_ENV to {configDir}/{env}.yaml
- Update loadConfigSafe to pass overlay path through to loadConfig
- All CLI commands using loadConfigSafe() automatically get overlay support
- No FLYNN_ENV = exact same behavior as before (backward compatible)
- Full test suite passes (1087 tests, zero regressions)
2026-02-09 20:57:12 -08:00
William Valentin c2cc052694 feat(02-01): implement deepMerge and overlay-aware loadConfig with tests
- Add deepMerge utility for recursive object merging (arrays replace, not concat)
- Extend loadConfig with optional overlayPath parameter
- Merge happens before env var expansion and Zod validation
- Add 6 deepMerge unit tests and 4 overlay integration tests
- Re-export deepMerge from config/index.ts
- All 1087 existing tests still pass
2026-02-09 20:56:29 -08:00
William Valentin 701fcfcaed refactor(01-03): extract services/skills/gateway/mcp into services.ts, reduce index.ts to 140 lines
- Extract initSkills(), initMcp(), loadSystemPrompt(), initPairingManager(), createGateway(), startServices() into services.ts
- daemon/index.ts reduced from 386 to 140 lines (64% reduction, 87% from 1087 baseline)
- Organize imports with section comments (External, Config, Daemon Modules, Infrastructure)
- Add section dividers in startDaemon() (Data & Sessions, Core Services, Model & Prompt, Gateway & Channels, Tier 1 Tools, Lifecycle)
- Convert unused value imports to type-only imports
- DaemonContext interface and re-exports unchanged
2026-02-09 20:22:34 -08:00
William Valentin 35a0061de9 feat(01-02): extract channel adapter registration into src/daemon/channels.ts
- Move Telegram, Discord, Slack, WhatsApp, WebChat adapter setup to channels.ts
- Move CronScheduler, WebhookHandler, GmailWatcher registration to channels.ts
- Clean up index.ts imports (remove unused adapter value imports)
- index.ts calls registerChannels() and receives cronScheduler for tool wiring
2026-02-09 20:14:23 -08:00
William Valentin fb1199a1da refactor(01-01): extract tool registration into src/daemon/tools.ts
- Create initTools() factory encapsulating ToolRegistry, allBuiltinTools, web search tools, ProcessManager, BrowserManager, ToolExecutor, and ToolPolicy
- Replace ~70 lines of inline tool setup in startDaemon() with single initTools() call
- Clean up tool-specific imports from daemon/index.ts (ToolPolicy, allBuiltinTools, createWebSearchTools, createProcessTools, ProcessManager, createBrowserTools)
- Tier 1 agent tools (session, agents list, message send, cron) remain in daemon/index.ts as intended
- daemon/index.ts reduced to 457 lines (from 1088 baseline)
2026-02-09 20:12:46 -08:00
William Valentin efceb38cb6 feat(01-02): extract agent config and sandbox setup into src/daemon/agents.ts
- Create initAgents() function encapsulating AgentConfigRegistry, AgentRouter, SandboxManager init
- Replace ~26 lines in startDaemon() with single initAgents() call
- Lifecycle shutdown handler for sandbox cleanup included in agents.ts
- Zero type errors, routing tests pass
2026-02-09 20:11:32 -08:00
William Valentin 00f8f74aac refactor(01-01): extract memory initialization into src/daemon/memory.ts
- Create initMemory() factory encapsulating MemoryStore, VectorStore, HybridSearch, background indexer, and memory tools registration
- Replace ~65 lines of inline memory init in startDaemon() with single initMemory() call
- Clean up memory-specific imports from daemon/index.ts (MemoryStore, VectorStore, HybridSearch, createEmbeddingProvider, chunkText, contentHash, createMemoryTools)
2026-02-09 20:10:49 -08:00
William Valentin 08f5b6b8e7 feat(01-02): extract message routing into src/daemon/routing.ts
- Move createMessageRouter function (~220 lines) to dedicated routing module
- Add import from ./routing.js in daemon/index.ts
- routing.test.ts passes without modification
- Zero type errors
2026-02-09 20:09:28 -08:00
William Valentin 86cda91f6b refactor(01-01): extract model client logic into src/daemon/models.ts
- Move createClientFromConfig, anthropicToGitHubModel, createAutoFallbackClient, createModelRouter to dedicated module
- Add re-exports from daemon/index.ts for backward compatibility
- clientFactory.test.ts passes without modification
- Reduces daemon/index.ts by ~248 lines
2026-02-09 20:06:27 -08:00
William Valentin 1e29da4da2 feat: complete DM pairing codes with channel adapters, gateway handlers, and TUI command (Tier 4 feature 4) 2026-02-09 18:28:10 -08:00
William Valentin 9d4d440ecf feat: add PairingManager and gateway lock tests (Tier 4 feature 4 foundation) 2026-02-09 13:32:59 -08:00
William Valentin 4413c4dc7c feat: add gateway lock, shell completion, and tailscale serve (Tier 4 features 1-3) 2026-02-09 13:29:59 -08:00
William Valentin 9be8f76bc7 feat: implement Tier 3 features — lane queue, credential redaction, token dashboard, xAI, Voyage AI
- Lane Queue: per-session FIFO queue in gateway replacing reject-when-busy (9 tests)
- Credential Redaction: redactConfig() expanded to cover 18+ secret fields (16 tests)
- Web UI Token Dashboard: system.tokenUsage endpoint + Usage page with summary cards
- xAI (Grok) Provider: OpenAI-compatible client with model pricing
- Voyage AI Embeddings: new embedding provider with configurable dimensions (5 tests)
- Update gap analysis: 90→95 match (70%→74%), Tier 3 section marked DONE
- Update state.json: test count 1001→1034, add tier3_completion entry

Total: 1034 tests passing across 85 files, typecheck clean
2026-02-09 10:32:57 -08:00
William Valentin 1d126cddfb feat: add Zhipu AI (GLM) model provider support
Adds zhipuai as a new provider using the OpenAI-compatible API at
api.z.ai. Supports api_key config or ZHIPUAI_API_KEY env var, with
optional endpoint override.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 09:55:13 -08:00
William Valentin 6ed8a4a8bf fix: gracefully handle Ollama models without tool support
Check model capabilities via /api/show before sending tools.
Models without 'tools' capability get requests without tools
(they can still answer, just without tool use). Result is cached
per client instance. Defense-in-depth: 'does not support' added
to retry nonRetryablePatterns to avoid wasting retries on
permanent errors.
2026-02-07 17:44:47 -08:00
William Valentin fb20acfbcd feat: add tool calling support to Ollama and llama.cpp clients
- Ollama: pass tools to API, parse tool_calls responses, handle thinking field from reasoning models (deepseek-r1, glm-4.7-flash)
- llama.cpp: pass tools via OpenAI-compatible endpoint, parse tool_calls, accumulate streaming tool call deltas
- Both clients now set stopReason to 'tool_use' when tool calls are present
- Tests: 12 new tests (8 Ollama + 5 llama.cpp, total 983→995)
2026-02-07 17:20:27 -08:00
William Valentin 8bf88049bf feat: add runtime context awareness — system.info tool + date/time in system prompt
- assembleSystemPrompt() now injects '# Runtime Context' with current date/time
- New system.info tool: date, time, hostname, platform, arch, uptime, memory, Node.js version
- Tool available in all profiles (minimal/messaging/coding/full)
- 983 tests passing (+7 new)
2026-02-07 16:22:17 -08:00
William Valentin 06438bb44f feat: add Gmail Pub/Sub watcher for inbound email automation
New ChannelAdapter that monitors Gmail via Google Cloud Pub/Sub push
notifications with polling fallback. Supports OAuth2 auth, configurable
watch labels, template rendering with email metadata placeholders
(from, to, subject, snippet, date, id, labels).

Wired into daemon lifecycle and gateway (POST /gmail/push endpoint).
Includes 16 tests covering auth, templates, push notifications, and
channel routing.
2026-02-07 15:39:24 -08:00
William Valentin 131d23989c feat: add file.patch tool for multi-hunk structured patches
Implements apply_patch equivalent: a single tool call can make multiple
line-based edits (replacements, insertions, deletions) across one or more
files. Hunks are applied bottom-up to preserve line numbers.

Includes 10 tests covering replacement, multi-hunk, insertion, deletion,
multi-file, overlapping hunks error, OOB error, and edge cases.
2026-02-07 15:39:15 -08:00
William Valentin 88731a50e3 feat: add heartbeat monitor and vector memory search (Tier 2)
Heartbeat:
- HeartbeatMonitor with 5 checks: gateway, model, channels, memory, disk
- Configurable interval, failure threshold, notification channel
- Recovery notifications when health restores
- 25 new tests

Vector Memory Search:
- EmbeddingProvider interface with OpenAI, Gemini, Ollama, LlamaCpp backends
- SQLite-backed VectorStore with cosine similarity search
- Text chunker with paragraph-aware splitting and overlap
- HybridSearch merging keyword + vector results with configurable weight
- Background indexer with dirty-namespace tracking
- Graceful fallback to keyword search when embeddings unavailable
- 51 new tests

Config: automation.heartbeat + memory.embedding schema sections
Total: 950 tests passing, all types clean
2026-02-07 14:45:11 -08:00
William Valentin b50c140d25 feat: add Docker support and inbound webhooks (Tier 2)
- Dockerfile: multi-stage build (node:22-alpine), better-sqlite3 native deps handled
- .dockerignore + docker-compose.yml for deployment
- FLYNN_DATA_DIR env var support in daemon, CLI, and TUI
- WebhookHandler: ChannelAdapter for HTTP POST /webhooks/:name
- Per-webhook HMAC auth, template rendering ({{body}}, {{json.field}})
- Config schema: automation.webhooks array with name/secret/message/output
- Gateway routes webhook requests before static files (bypasses gateway auth)
- 23 new tests for webhook functionality, 874 total tests passing
2026-02-07 14:36:05 -08:00
William Valentin b322e8f29c fix: GitHub Copilot fallback — remove stale API version header and fix model name mapping
Two issues prevented the GitHub Models fallback from working:

1. The X-GitHub-Api-Version: 2022-11-28 header caused '400 invalid
   apiVersion' errors. The Copilot chat completions endpoint does not
   use this header — removed from both constructor and rebuildClient.

2. The anthropicToGitHubModel mapping was incomplete: it only knew
   three models and the generic date-stripping fallback produced wrong
   names (e.g. 'claude-sonnet-4-5' instead of 'claude-sonnet-4.5').
   GitHub Copilot uses dots for sub-versions, not hyphens.

   Updated with explicit mappings for all current models (sonnet 4,
   4.5; opus 4, 4.5, 4.6; haiku 4.5) and a smarter generic fallback
   that converts digit-hyphen-digit to digit.digit at the end.

3. createClientFromConfig now auto-maps Anthropic-style model names
   when the provider is 'github', so users can copy model names from
   their Anthropic config into fallback blocks without manual renaming.
2026-02-07 14:04:54 -08:00
William Valentin e12eb3a0be fix: TUI now uses shared model router with auto-fallback support
The TUI was building its own ModelRouter with a duplicated client factory
that lacked auto same-model fallback, local_providers resolution, retry
config, and per-tier fallback logic. When Anthropic failed, it skipped
GitHub Models and fell straight to the local Ollama model.

Replace the duplicated ~50-line createClient + router setup in tui.ts
with a single call to the daemon's createModelRouter(), which already
handles all of these correctly. This removes ~50 lines of duplicated
code and ensures TUI and daemon have identical fallback behavior.
2026-02-07 13:58:34 -08:00