flynn/README.md

# Flynn

Self-hosted personal AI assistant with Telegram and Terminal interfaces.

## Features

- **Multi-Frontend**: Telegram bot + Terminal UI (minimal & fullscreen modes) + Web UI dashboard
- **Multi-Model**: Anthropic Claude, OpenAI, GitHub Copilot, Gemini, Bedrock, Ollama, llama.cpp with intelligent routing
- **Multi-Channel**: Telegram, Discord, Slack, WhatsApp with unified adapter interface
- **Web Dashboard**: SPA control panel with health monitoring, chat, session browser, and settings editor
- **Model Switching**: Switch between cloud/local models on demand
- **Session Persistence**: SQLite-backed conversation history
- **Fallback Chains**: Automatic failover when primary model fails
- **Hook Engine**: Confirmation system for sensitive operations
- **Tool Framework**: Shell, file, web-fetch, web-search, browser control, image analysis, media send
- **Docker Sandboxing**: Per-session container isolation for tool execution
- **Multi-Agent Routing**: Config-driven agent selection per sender/channel with tool profiles
- **Media Pipeline**: Image analysis, outbound attachments, audio transcription across all channels
- **Session Transfer**: Move conversations between frontends
- **CLI**: Full command-line interface (`flynn start`, `send`, `doctor`, etc.)
- **Cron Scheduling**: Automated messages on cron schedules with output routing
- **Inbound Webhooks**: HTTP endpoints that trigger agent processing with HMAC auth and template rendering
- **Heartbeat Monitor**: Periodic health checks (gateway, model, channels, memory, disk) with failure notifications
- **Vector Memory Search**: Hybrid keyword + semantic search with embeddings (OpenAI, Gemini, Ollama, llama.cpp)
- **Docker Deployment**: Multi-stage Dockerfile and docker-compose.yml for production containers
- **Health Diagnostics**: `flynn doctor` validates config, connectivity, and system state
- **MCP Integration**: External tool servers via Model Context Protocol
- **Skills System**: Extensible capability packages (bundled, managed, workspace tiers)

## Quick Start

```bash
# Install dependencies
pnpm install

# Copy and configure
cp config/default.yaml ~/.config/flynn/config.yaml
# Edit config with your API keys and Telegram bot token

# Build and run
pnpm build
flynn start

# Or run without building
pnpm start
```

## CLI Commands

Flynn provides a full CLI via the `flynn` binary (or `npx tsx src/cli/index.ts` during development):

| Command | Description |
|---------|-------------|
| `flynn start` | Start the Flynn daemon (Telegram, WebChat, cron) |
| `flynn tui` | Launch the interactive terminal UI |
| `flynn send <message>` | Send a one-shot message and print the response |
| `flynn sessions` | List active sessions |
| `flynn doctor` | Validate config and check system health |
| `flynn config` | Show resolved configuration (secrets redacted) |

### Examples

```bash
# Start daemon with custom config
flynn start --config ~/my-config.yaml

# One-shot query
flynn send "What's the weather in London?"

# Check system health
flynn doctor --config ~/.config/flynn/config.yaml

# Show current config (secrets masked)
flynn config

# List sessions
flynn sessions
```

## Configuration

Config location: `~/.config/flynn/config.yaml` (or set `FLYNN_CONFIG`)

```yaml
telegram:
  bot_token: "your-telegram-bot-token"
  allowed_chat_ids: [123456789]  # Your Telegram user ID

models:
  default:
    provider: anthropic
    model: claude-opus-4-5-20251101
    api_key: sk-ant-api03-...
  local:
    provider: ollama
    model: qwen2.5:14b
  fallback_chain: [local]

hooks:
  confirm: [shell.*, file.write]
  log: [web.*, file.read]
  silent: [notify]
```

### Model Providers

| Provider | Config |
|----------|--------|
| Anthropic | `provider: anthropic`, `api_key` or `auth_token` |
| OpenAI | `provider: openai`, `api_key`, optional `endpoint` |
| GitHub Copilot | `provider: github`, auto-login via OAuth device flow |
| Gemini | `provider: gemini`, `api_key` |
| Bedrock | `provider: bedrock`, AWS credentials |
| Ollama | `provider: ollama`, `model`, optional `endpoint` |
| llama.cpp | `provider: llamacpp`, `endpoint` |

### Model Tiers

Configure multiple models for different purposes:

```yaml
models:
  fast: { provider: anthropic, model: claude-sonnet-4-... }
  default: { provider: anthropic, model: claude-opus-4-5-... }
  complex: { provider: anthropic, model: claude-opus-4-5-... }
  local: { provider: ollama, model: qwen2.5:14b }
```

## Telegram Commands

| Command | Description |
|---------|-------------|
| `/start` | Initialize bot |
| `/reset` | Clear conversation history |
| `/status` | Show current model and status |
| `/local` | Switch to local model |
| `/cloud` | Switch to cloud model |
| `/model` | Show model info and options |

## Web UI Dashboard

Flynn includes a built-in web control dashboard served by the WebSocket gateway. Access it at `http://localhost:18800` (or your configured gateway port).

### Pages

| Page | Description |
|------|-------------|
| **Dashboard** | System health cards, channel status, usage stats. Auto-refreshes every 10s |
| **Chat** | Session selector, streaming tool events, markdown rendering with syntax highlighting |
| **Sessions** | Browse all sessions, view message history, delete sessions |
| **Settings** | Edit hook patterns (confirm/log/silent), view tools, channels, and redacted config |

The dashboard is a vanilla JS SPA with no build step — hash-based routing, ES modules, and the existing WebSocket JSON-RPC protocol.

## Terminal UI

```bash
# Minimal mode (readline)
pnpm tui

# Fullscreen mode (React/Ink)
pnpm tui:fs
```

### TUI Commands

| Command | Description |
|---------|-------------|
| `/help` | Show help |
| `/reset` | Clear history |
| `/status` | Show session info |
| `/fullscreen` | Switch to fullscreen mode |
| `/transfer telegram` | Transfer session to Telegram |
| `/quit` | Exit |

## Running as Service

```bash
# Create systemd user service
mkdir -p ~/.config/systemd/user

cat > ~/.config/systemd/user/flynn.service << 'EOF'
[Unit]
Description=Flynn Personal AI Assistant
After=network.target ollama.service

[Service]
Type=simple
WorkingDirectory=/path/to/flynn
ExecStart=/usr/bin/pnpm start
Restart=on-failure
RestartSec=5
Environment=NODE_ENV=production

[Install]
WantedBy=default.target
EOF

# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now flynn

# View logs
journalctl --user -u flynn -f
```

## Hook Engine

Control sensitive operations with pattern matching:

```yaml
hooks:
  confirm:    # Requires user approval via Telegram
    - shell.*
    - file.write
  log:        # Logs but doesn't block
    - web.*
    - file.read
  silent:     # Executes without notification
    - notify
```

## Cron Scheduling

Schedule automated messages on cron schedules. Each job fires an inbound message through the agent pipeline and routes the response to a configured output channel.

```yaml
automation:
  cron:
    - name: daily-summary
      schedule: "0 9 * * *"            # 9 AM daily
      message: "Give me a summary of today's tasks"
      output:
        channel: telegram               # Route response to Telegram
        peer: "123456789"               # Chat ID to send to
      timezone: Europe/London           # Optional timezone
      enabled: true

    - name: hourly-check
      schedule: "0 * * * *"            # Every hour
      message: "Check system status"
      output:
        channel: telegram
        peer: "123456789"
      enabled: false                    # Disabled, won't fire
```

### Cron Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `name` | yes | Unique job identifier |
| `schedule` | yes | Cron expression (standard 5-field) |
| `message` | yes | Text sent to the agent when the job fires |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
| `timezone` | no | IANA timezone (defaults to system timezone) |
| `enabled` | no | Whether the job is active (default: `true`) |

## Inbound Webhooks

HTTP endpoints that trigger agent processing. Each webhook accepts POST requests, optionally verifies an HMAC signature, renders a message template, and routes the agent's response to an output channel.

```yaml
automation:
  webhooks:
    - name: github-push
      secret: "whsec_..."                  # HMAC secret for signature verification
      message: "GitHub push to {{json.repository.full_name}}: {{json.head_commit.message}}"
      output:
        channel: telegram
        peer: "123456789"

    - name: alertmanager
      message: "Alert: {{json.alerts.0.annotations.summary}}"
      output:
        channel: discord
        peer: "channel-id"
```

Webhooks are available at `POST /webhooks/:name` on the gateway HTTP server. They bypass gateway token auth and use their own per-webhook HMAC verification instead.

### Webhook Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `name` | yes | Unique webhook identifier (used in URL path) |
| `secret` | no | HMAC secret for `X-Webhook-Signature` header verification (SHA-256) |
| `message` | no | Template for the message sent to the agent (default: `{{body}}`) |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
| `enabled` | no | Whether the webhook is active (default: `true`) |

### Template Variables

| Variable | Description |
|----------|-------------|
| `{{body}}` | Raw request body as string |
| `{{json.field}}` | Extract a field from JSON body (dot notation for nested fields) |

## Heartbeat Monitor

Periodic health checks that validate system components and notify a configured channel on failure.

```yaml
automation:
  heartbeat:
    enabled: true
    interval: "5m"                         # Check every 5 minutes
    checks: [gateway, model, channels, memory, disk]
    notify:
      channel: telegram
      peer: "123456789"
    failure_threshold: 2                   # Notify after 2 consecutive failures
    disk_threshold_mb: 100                 # Warn when <100MB free
```

### Heartbeat Checks

| Check | What it validates |
|-------|-------------------|
| `gateway` | HTTP server is responsive |
| `model` | Model router has an active tier configured |
| `channels` | At least one channel adapter is connected |
| `memory` | Memory directory is readable and writable |
| `disk` | Free disk space exceeds threshold |

The monitor sends a notification when failures reach the configured threshold and a recovery notification when all checks pass again.

### Heartbeat Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable the heartbeat monitor (default: `false`) |
| `interval` | no | Check interval: `60s`, `5m`, `1h` (default: `5m`) |
| `checks` | no | Which checks to run (default: all five) |
| `notify.channel` | no | Channel to send failure/recovery notifications |
| `notify.peer` | no | Peer/chat ID for notifications |
| `failure_threshold` | no | Consecutive failures before notifying (default: `2`) |
| `disk_threshold_mb` | no | Disk space warning threshold in MB (default: `100`) |

## Vector Memory Search

The memory system supports hybrid search combining keyword matching with semantic vector similarity. When embeddings are enabled, `memory.search` uses both approaches and merges results with configurable weighting.

```yaml
memory:
  enabled: true
  auto_extract: true
  max_context_tokens: 2000
  embedding:
    enabled: true
    provider: openai                       # openai, gemini, ollama, llamacpp
    model: text-embedding-3-small
    api_key: "${OPENAI_API_KEY}"
    chunk_size: 512                        # Tokens per chunk
    chunk_overlap: 50                      # Overlap between chunks
    top_k: 5                               # Top results from vector search
    hybrid_weight: 0.7                     # 0.0 = keyword only, 1.0 = vector only
```

### Embedding Providers

| Provider | Config |
|----------|--------|
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
| Gemini | `provider: gemini`, `api_key`, `model` |
| Ollama | `provider: ollama`, `endpoint` (default: localhost:11434), `model` |
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |

Embeddings are indexed in the background — when memory is written, the namespace is marked dirty and re-indexed within 30 seconds. The vector index is stored in `vectors.db` alongside the session database.

When embeddings are disabled or the provider is unreachable, search falls back gracefully to keyword matching.

### Embedding Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable vector search (default: `false`) |
| `provider` | no | Embedding provider (default: `openai`) |
| `model` | no | Embedding model name (default: `text-embedding-3-small`) |
| `endpoint` | no | Provider endpoint (required for `ollama`/`llamacpp`) |
| `api_key` | no | API key (required for `openai`/`gemini`) |
| `dimensions` | no | Vector dimensions (auto-detected from model if not set) |
| `chunk_size` | no | Max tokens per chunk (default: `512`) |
| `chunk_overlap` | no | Token overlap between chunks (default: `50`) |
| `top_k` | no | Number of vector results to return (default: `5`) |
| `hybrid_weight` | no | Vector vs keyword weight, 0.0-1.0 (default: `0.7`) |

## Docker Deployment

Flynn includes a production-ready Dockerfile with multi-stage build.

```bash
# Build the image
docker build -t flynn .

# Run with config and data volumes
docker run -d \
  --name flynn \
  -p 18800:18800 \
  -v ./config.yaml:/config/config.yaml:ro \
  -v flynn-data:/data \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  flynn
```

Or use the included `docker-compose.yml`:

```bash
# Copy your config
cp ~/.config/flynn/config.yaml ./config.yaml

# Start with compose
docker compose up -d

# View logs
docker compose logs -f
```

### Docker Environment Variables

| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Config file path (default: `/config/config.yaml`) |
| `FLYNN_DATA_DIR` | Data directory path (default: `/data`) |
| `ANTHROPIC_API_KEY` | Anthropic API key |
| `OPENAI_API_KEY` | OpenAI API key |
| `TELEGRAM_BOT_TOKEN` | Telegram bot token |

### Volumes

| Mount Point | Purpose |
|-------------|---------|
| `/config/config.yaml` | Configuration file (read-only) |
| `/data` | Persistent data (sessions DB, memory files, vector index) |

## Doctor Diagnostics

`flynn doctor` runs 10 health checks to validate your setup:

```
$ flynn doctor

Flynn Doctor
============

[PASS] Config file exists (/home/user/.config/flynn/config.yaml)
[PASS] Config parses (valid YAML)
[PASS] Config validates (schema valid)
[PASS] Env vars resolved
[PASS] Data directory writable (/home/user/.local/share/flynn)
[PASS] Session DB accessible (sessions.db)
[PASS] Model connectivity (anthropic: claude-sonnet)
[PASS] Telegram bot configured (1 allowed chat(s))
[SKIP] MCP servers configured (none configured)
[PASS] Skills loaded (3 skill(s))

Results: 8 passed, 0 failed, 0 warnings, 1 skipped
```

### Check Details

| Check | What it validates |
|-------|-------------------|
| Config file exists | Config YAML file is present at the expected path |
| Config parses | File is valid YAML syntax |
| Config validates | YAML content passes Zod schema validation |
| Env vars resolved | Any `${VAR}` references in config have values set |
| Data directory writable | Can write to `~/.local/share/flynn/` |
| Session DB accessible | SQLite database opens and queries succeed |
| Model connectivity | Default model provider and model name are configured |
| Telegram bot configured | Bot token is present and reasonable length |
| MCP servers configured | Lists configured MCP tool servers |
| Skills loaded | Discovers and loads skill packages |

Exit code is `1` if any check fails, `0` otherwise. Checks that depend on a valid config are skipped when config is invalid.

## Session Management

- Sessions persist in `~/.local/share/flynn/sessions.db`
- Session ID format: `{frontend}:{userId}` (e.g., `telegram:123456789`)
- History survives restarts
- Transfer sessions between frontends with `/transfer`

## Architecture

```
src/
├── agents/               # Multi-agent routing
├── auth/                 # OAuth flows (GitHub Copilot)
├── backends/native/      # Agent implementation + orchestrator
├── channels/             # Channel adapters (Telegram, Discord, Slack, WhatsApp, WebChat)
├── cli/                  # CLI commands (commander)
├── config/               # YAML config + Zod validation
├── context/              # Token estimation + compaction
├── daemon/               # Lifecycle management + routing
├── frontends/
│   ├── telegram/         # Telegram bot
│   └── tui/              # Terminal UI (minimal + fullscreen)
├── gateway/              # WebSocket gateway + web UI dashboard
│   ├── handlers/         # JSON-RPC method handlers
│   └── ui/               # SPA dashboard (vanilla JS)
│       ├── pages/        # Dashboard, Chat, Sessions, Settings
│       └── lib/          # WebSocket RPC client
├── hooks/                # Confirmation engine
├── mcp/                  # MCP tool server integration
├── memory/               # Persistent memory store + vector search
├── models/               # Model providers + router + media pipeline
├── prompt/               # System prompt templating
├── sandbox/              # Docker sandboxing
├── session/              # SQLite persistence
├── skills/               # Skill packages
├── tools/                # Builtin tools (shell, file, web, browser, process, media)
└── automation/           # Cron scheduler, webhooks, heartbeat monitor
```

## Development

```bash
# Dev mode with watch
pnpm dev          # Daemon
pnpm tui:dev      # TUI

# Type check
pnpm typecheck

# Lint
pnpm lint

# Test
pnpm test
```

## Environment Variables

| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Override config file path |
| `FLYNN_DATA_DIR` | Override data directory (default: `~/.local/share/flynn`) |
| `ANTHROPIC_API_KEY` | Anthropic API key (fallback) |
| `OPENAI_API_KEY` | OpenAI API key (fallback) |

## License

MIT