Commit Graph

483 Commits

Author SHA1 Message Date
William Valentin 6761dca1c2 fix: normalize message roles for local model backends (llama.cpp, Ollama)
Local backends using strict chat templates (e.g. Mistral 3) rejected
Flynn's Anthropic-style tool_use/tool_result content blocks, causing
'roles must alternate' errors. Added getMessageTextWithTools() and
normalizeMessagesForLocal() to serialize structured blocks to plain
text, drop empty messages, and merge consecutive same-role messages.
Also fixed compaction to ensure kept messages start with user role.
2026-02-10 22:04:17 -08:00
William Valentin 2f6d045e2a feat: load .env file at startup using Node built-in loadEnvFile
Adds process.loadEnvFile() to CLI entry point so API keys (ZHIPUAI_API_KEY,
OPENROUTER_API_KEY, XAI_API_KEY, etc.) can be stored in a project .env file
instead of shell environment or systemd service config. Uses Node >= 20.12
built-in — no dotenv dependency needed. Silent no-op if .env doesn't exist.

Updates .env.example with placeholders for all provider API keys.
2026-02-10 21:43:09 -08:00
William Valentin 5c90640e2a fix: clear error messages for missing API keys on provider switch
Previously, switching to zhipuai/openrouter/xai via /model would throw a
confusing 'OPENAI_API_KEY missing' error from the OpenAI SDK. Now
createClientFromConfig validates API keys before constructing the client,
throwing errors that name the correct env var (e.g. ZHIPUAI_API_KEY).

Also fixes the misleading 'as anthropic' type cast in the /model handler
to validate against MODEL_PROVIDERS and use the ModelProvider type.
2026-02-10 21:32:44 -08:00
William Valentin aaaf4a361a fix(webchat): move action buttons outside message bubble
Wrap each message in a .message-wrapper div and render copy/edit buttons
below the bubble instead of overlapping inside it. Improves readability
and prevents buttons from covering message content.
2026-02-10 21:26:22 -08:00
William Valentin 25482b8516 feat: sync PROVIDER_NAMES with config schema and update README docs
Extract MODEL_PROVIDERS const from config schema as single source of truth
for provider names. PROVIDER_NAMES in TUI commands now imports from schema
instead of maintaining a hardcoded list. Adds tests verifying sync.

Updates README TUI Commands section with /model hot-swap documentation,
supported providers, and runtime model switching examples.
2026-02-10 21:26:18 -08:00
William Valentin 27ee3b2c10 feat(webchat): add copy and edit buttons on chat messages
Copy button on all messages (clipboard API with checkmark feedback).
Edit button on user messages populates the input textarea.
Buttons appear on hover (desktop) or always visible (mobile).
2026-02-10 20:53:49 -08:00
William Valentin 4c8ba3f20c feat(webchat): add slash commands, autocomplete popup, and web search button
Add 6 slash commands (/help, /reset, /compact, /usage, /status, /model)
with autocomplete popup (arrow keys, Enter/Tab/Escape navigation).
Search button toggles web search mode by prepending instruction to message.
Backend agent.send extended with metadata for server-side command routing.
2026-02-10 20:45:14 -08:00
William Valentin 7a69794418 fix: sync model tier between TUI and WebChat when switching models
ModelRouter now supports multiple tier-change listeners via addOnTierChange(),
SessionBridge subscribes to tier changes and propagates them to all WebChat
agents (both existing and newly created), and the fullscreen TUI now also
updates the agent's tier when switching models (matching minimal TUI behavior).
2026-02-10 20:22:40 -08:00
William Valentin bf9ca690f3 fix(agent): detect repeated tool call loops and make max_iterations configurable
Local LLMs often get stuck calling the same tool repeatedly because they
lack the sophistication to synthesize results. The agent loop had no
safeguard — it re-executed whatever the model requested up to 10 times.

Add fingerprint-based loop detection: if the same tool+args combination
repeats 3 consecutive times, break the loop and return the last results.
Also add agents.max_iterations to the config schema so the iteration
limit is user-configurable (default: 10).
2026-02-10 19:35:09 -08:00
William Valentin 4ce8e81c01 fix(gmail): sanitize HTML entities and tags in tool output
Gmail API returns snippets with HTML entities (&amp;, &#39;, <br>, etc.)
that leaked into LLM responses as raw HTML. Added shared sanitizeHtml()
utility in src/utils/html.ts and applied it to gmail tool snippets,
HTML body fallback, and gmail watcher snippets.
2026-02-10 16:30:14 -08:00
William Valentin 2644ed269e fix(tui): reset agent state on /reset command in fullscreen mode
Use agent.reset() instead of session.clear() when agent is available,
ensuring tool fingerprint, usage counters, and agent history are all
cleared properly alongside the session.
2026-02-10 13:27:58 -08:00
William Valentin 671ec035e9 fix(tui): show tool activity in fullscreen mode via Ink-compatible callback
Replace process.stdout.write-based onToolUse callback (which corrupts
Ink rendering) with a React state-driven approach that shows tool names,
args, and completion status in the streaming content area.
2026-02-10 13:26:47 -08:00
William Valentin e46e8740a1 fix(tui): enable tool access in fullscreen mode via NativeAgent
Fullscreen TUI was calling modelClient directly, bypassing the NativeAgent
tool loop entirely. Pass the agent through FullscreenTuiConfig → App and
use agent.process() for message handling, which enables the full tool
registry and executor.
2026-02-10 13:21:22 -08:00
William Valentin f892bbe6ca feat(tui): add ASCII art banner on startup 2026-02-10 13:11:32 -08:00
William Valentin f204ff1dd7 feat(tools): add Google Docs, Drive, and Tasks read-only tools
Add three new Google service integrations following the established
Gmail/GCal pattern:

- Google Docs (docs.list, docs.search, docs.read): list, search, and
  read document content as plain text via Docs + Drive APIs
- Google Drive (drive.list, drive.search, drive.read): list, search,
  and read files with export support for Workspace files (Docs→text,
  Sheets→CSV, Slides→text)
- Google Tasks (tasks.lists, tasks.list): list task lists and tasks
  with status, due dates, and notes

Each service has its own config section, OAuth auth command, tool
policy group, and test suite (53 new tests). The setup wizard now
offers to configure all Google services together and run OAuth auth
flows automatically after saving config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:59:15 -08:00
William Valentin 411c6d84a2 feat(tui): persist model tier selection and fix formatting
Persist /model tier choice to ~/.local/share/flynn/preferences.json so
it survives restarts. Decode HTML entities (e.g. &#39;) in markdown
renderer output. Suppress noisy logger.info and punycode deprecation
warnings in TUI startup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:23:12 -08:00
William Valentin 50471d63af feat(tools): add gmail.read tool for full email content
The existing gmail.list and gmail.search tools only return snippets.
gmail.read fetches the full message by ID using format: 'full', decodes
base64url body parts (preferring text/plain, falling back to stripped
HTML), and returns headers + body text. This enables workflows like
searching for invoices and extracting amounts from the full content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 12:01:49 -08:00
William Valentin f6dedf0fbe fix(tui): register Google Calendar tools when gcal is enabled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:45:05 -08:00
William Valentin 55d35c80b4 feat(tui): improve tool use display and register Gmail tools
Format tool names as human-readable labels (e.g. "Gmail: List") and
show args as compact key-value pairs instead of raw JSON. Also register
Gmail tools in the TUI when automation.gmail is enabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:41:35 -08:00
William Valentin 796e143d61 fix(agent): inject tool inventory note when tools change mid-session
Stale session history can cause the model to follow old "I can't do
that" patterns even when new tools are available. NativeAgent now tracks
a tool fingerprint and appends a system prompt note listing current
tools when the inventory changes, resetting on session reset.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:41:31 -08:00
William Valentin 94264e848c feat(tools): add Google Calendar tools and register Gmail/GCal in daemon
Add calendar.today, calendar.list, calendar.search tools mirroring the
Gmail tool pattern. Includes gcal-auth CLI command, config schema, tool
policy entries (messaging/coding profiles + group:gcal), and 17 tests.
Also wires up gmail and gcal tool registration in the daemon and TUI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:40:53 -08:00
William Valentin 4cc29f534a fix(tui): render inline markdown formatting with ANSI codes
Block-level renderer methods (paragraph, heading, blockquote, list) were
using raw token.text instead of this.parser.parseInline(tokens), causing
bold, italic, and inline code to never render. Add table renderer with
aligned columns and box-drawing separators. Remove unused marked-terminal
dependency (incompatible with marked v17).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 11:29:57 -08:00
William Valentin ff03f74404 feat(cli): add gmail-auth command for OAuth2 token setup
Implements `flynn gmail-auth` to complete the OAuth2 flow that
GmailWatcher references but was never built. Supports local callback
server (default) and --manual paste mode. Adds Gmail health check
to `flynn doctor`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 10:33:01 -08:00
William Valentin f4b9c850ab feat(setup): add contextual help text to all wizard flows
Each setup section now explains what's needed before prompting:
- Providers: links to API key consoles (Anthropic, OpenAI, Gemini, etc.)
- Channels: step-by-step bot creation (Telegram @BotFather, Discord dev
  portal, Slack app setup, WhatsApp QR)
- Gmail: Google Cloud Console OAuth setup walkthrough
- Memory: explains what vector search does and key reuse
- Security: describes each option (sandbox, pairing, tool profiles)
- Gateway: explains auth token, Tailscale Serve, lock mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 10:08:44 -08:00
William Valentin 7620616c7c test(setup): add integration tests and update shell completion
Adds comprehensive integration tests for the first-run wizard verifying config
generation for different provider/channel combinations. Updates shell completion
to include the 'setup' command with its options.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:38:53 -08:00
William Valentin f50d7d69fb feat(setup): wire setup command into CLI and start command
- Register setup command in CLI index
- Offer setup wizard when running `flynn start` with no config
- Guard telegram log output since telegram is now optional

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:36:58 -08:00
William Valentin d8b7b08270 feat(setup): add main orchestrator, menu, and CLI command
Implements Task 6 of the setup wizard:
- orchestrator.ts: runMenu() for interactive configuration loop
- orchestrator.ts: runFirstRunWizard() for new user onboarding
- orchestrator.test.ts: test for menu exit behavior
- setup.ts: registerSetupCommand() and runSetup() handler
  - Handles both first-run and existing config scenarios
  - Saves YAML config to disk
  - Optional daemon startup after first-run

All tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:35:32 -08:00
William Valentin 182d86957b feat(setup): add memory, automation, security, and gateway setup flows 2026-02-10 09:34:04 -08:00
William Valentin b673632b0f feat(setup): add channel setup flows
Implement setupChannels function with support for Telegram, Discord, Slack, and WhatsApp.
Includes WebChat gateway configuration and channel choice loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:32:52 -08:00
William Valentin 573cb43534 feat(setup): add model provider setup flows 2026-02-10 09:31:43 -08:00
William Valentin d35ce2beb5 feat(setup): add config builder and summary renderer
Add ConfigBuilder class to accumulate wizard answers into config objects with YAML
serialization, and renderSummary function to display configuration summary. Includes
9 test cases covering provider setup, channel configuration, and feature flags.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:29:56 -08:00
William Valentin 9cc03187b0 feat(setup): add prompt helpers for setup wizard
Created a Prompter interface and factory function for interactive CLI prompts:
- ask(): text input with optional default values
- confirm(): yes/no confirmation with default
- choose(): numbered menu selection with fallback
- password(): text input (no echo planned in TUI)
- println(): simple output helper

All 9 tests pass (ask, confirm, choose, password scenarios).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:28:19 -08:00
William Valentin 213dba855a refactor: make telegram config optional for non-telegram setups
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:27:18 -08:00
William Valentin 64e3716ede test(tui): add /pair command parsing tests 2026-02-09 21:56:27 -08:00
William Valentin 3ea4f64d6b feat(tui): wire /pair command execution with PairingManager 2026-02-09 21:56:27 -08:00
William Valentin 322852917c fix(tui): remove stale readline close listeners to prevent memory leak warning
Clean up the once('close') listener on the readline Interface when
rl.question() resolves normally. Previously, each prompt loop iteration
accumulated a close listener that was never removed, triggering
MaxListenersExceededWarning after 11 prompts.
2026-02-09 21:50:43 -08:00
William Valentin 62331c3a09 feat(daemon): wire PairingStore from SessionStore into PairingManager 2026-02-09 21:49:55 -08:00
William Valentin ecd3aca7c1 feat(session): add pairing_approved table and getPairingStore() 2026-02-09 21:46:51 -08:00
William Valentin 1e1a68924e feat(pairing): add PairingStore interface for persistence injection 2026-02-09 21:45:04 -08:00
William Valentin c3ca3f3776 feat(03-02): extend dashboard with live ops sections
- Core counters: messages processed, sessions, queue depth, uptime, active requests, errors
- Model performance table: recent calls with latency, tokens/sec, provider, status
- Event stream: scrollable log with color-coded levels (error/warn/info)
- Active requests: in-flight request table with session, channel, duration
- Channels grid: existing channel status cards preserved
- Dual timer refresh: 3s for metrics/events/requests, 10s for health/channels
- Targeted DOM updates via getElementById for flicker-free fast updates
2026-02-09 21:34:11 -08:00
William Valentin a0feff9637 feat(03-01): hook metrics recording into agent request flow
- Track active requests with startRequest/endRequest around lane queue work
- Increment messagesProcessed on successful agent.process completion
- Record errors and error events on agent.send failures
- Record tool failure events with tool name and error details
2026-02-09 21:29:14 -08:00
William Valentin bd1880a44c feat(03-01): create MetricsCollector and wire into gateway
- Add MetricsCollector class with counters, model call ring buffer, event ring buffer, and active request tracking
- Add system.metrics, system.events, system.activeRequests RPC handlers
- Add GET /health unauthenticated HTTP endpoint for Docker HEALTHCHECK
- Add totalPending() to LaneQueue for queue depth metrics
- Add 20 tests for MetricsCollector
2026-02-09 21:28:05 -08:00
William Valentin 35f4cab0dc feat: add log-level system to suppress noisy fallback debug output
Replace console.debug/log/warn calls in model router, retry, and daemon
startup with a structured logger that respects a configurable log_level.
Default level is 'info', suppressing verbose fallback debug messages in
the TUI while keeping them available via config when needed.

- Add src/logger.ts with debug/info/warn/error/silent levels
- Wire log_level into config schema (default: 'info')
- Initialize log level in both daemon and TUI startup paths
- Convert all console.debug in router.ts and retry.ts to logger.debug
- Convert console.log/warn in daemon/models.ts to logger.info/warn
2026-02-09 21:23:07 -08:00
William Valentin 6bd372162e feat(02-02): add overlay file validation to flynn doctor
- Import resolveOverlayPath from shared.ts
- Add checkOverlayExists check (skip when no FLYNN_ENV, pass/fail for overlay file)
- Insert after checkConfigExists in allChecks array
- All 1087 tests pass, typecheck clean
2026-02-09 21:00:18 -08:00
William Valentin 29bc18502f feat(02-01): wire FLYNN_ENV resolution into shared.ts with overlay-aware loadConfigSafe
- Add resolveOverlayPath() that maps FLYNN_ENV to {configDir}/{env}.yaml
- Update loadConfigSafe to pass overlay path through to loadConfig
- All CLI commands using loadConfigSafe() automatically get overlay support
- No FLYNN_ENV = exact same behavior as before (backward compatible)
- Full test suite passes (1087 tests, zero regressions)
2026-02-09 20:57:12 -08:00
William Valentin c2cc052694 feat(02-01): implement deepMerge and overlay-aware loadConfig with tests
- Add deepMerge utility for recursive object merging (arrays replace, not concat)
- Extend loadConfig with optional overlayPath parameter
- Merge happens before env var expansion and Zod validation
- Add 6 deepMerge unit tests and 4 overlay integration tests
- Re-export deepMerge from config/index.ts
- All 1087 existing tests still pass
2026-02-09 20:56:29 -08:00
William Valentin 701fcfcaed refactor(01-03): extract services/skills/gateway/mcp into services.ts, reduce index.ts to 140 lines
- Extract initSkills(), initMcp(), loadSystemPrompt(), initPairingManager(), createGateway(), startServices() into services.ts
- daemon/index.ts reduced from 386 to 140 lines (64% reduction, 87% from 1087 baseline)
- Organize imports with section comments (External, Config, Daemon Modules, Infrastructure)
- Add section dividers in startDaemon() (Data & Sessions, Core Services, Model & Prompt, Gateway & Channels, Tier 1 Tools, Lifecycle)
- Convert unused value imports to type-only imports
- DaemonContext interface and re-exports unchanged
2026-02-09 20:22:34 -08:00
William Valentin 35a0061de9 feat(01-02): extract channel adapter registration into src/daemon/channels.ts
- Move Telegram, Discord, Slack, WhatsApp, WebChat adapter setup to channels.ts
- Move CronScheduler, WebhookHandler, GmailWatcher registration to channels.ts
- Clean up index.ts imports (remove unused adapter value imports)
- index.ts calls registerChannels() and receives cronScheduler for tool wiring
2026-02-09 20:14:23 -08:00
William Valentin fb1199a1da refactor(01-01): extract tool registration into src/daemon/tools.ts
- Create initTools() factory encapsulating ToolRegistry, allBuiltinTools, web search tools, ProcessManager, BrowserManager, ToolExecutor, and ToolPolicy
- Replace ~70 lines of inline tool setup in startDaemon() with single initTools() call
- Clean up tool-specific imports from daemon/index.ts (ToolPolicy, allBuiltinTools, createWebSearchTools, createProcessTools, ProcessManager, createBrowserTools)
- Tier 1 agent tools (session, agents list, message send, cron) remain in daemon/index.ts as intended
- daemon/index.ts reduced to 457 lines (from 1088 baseline)
2026-02-09 20:12:46 -08:00
William Valentin efceb38cb6 feat(01-02): extract agent config and sandbox setup into src/daemon/agents.ts
- Create initAgents() function encapsulating AgentConfigRegistry, AgentRouter, SandboxManager init
- Replace ~26 lines in startDaemon() with single initAgents() call
- Lifecycle shutdown handler for sandbox cleanup included in agents.ts
- Zero type errors, routing tests pass
2026-02-09 20:11:32 -08:00