# Flynn

Self-hosted personal AI assistant with Telegram and Terminal interfaces.

## Features

- **Multi-Frontend**: Telegram bot + Terminal UI (minimal & fullscreen modes) + Web UI dashboard
- **Multi-Model**: Anthropic Claude, OpenAI, GitHub Copilot, Gemini, Bedrock, Zhipu AI (GLM), xAI (Grok), Ollama, llama.cpp with intelligent routing
- **Multi-Channel**: Telegram, Discord, Slack, WhatsApp, Matrix, Signal, Mattermost, Microsoft Teams, Google Chat, LINE, Feishu/Lark, Zalo, and iMessage (BlueBubbles) with unified adapter interface
- **Web Dashboard**: SPA control panel with health monitoring, chat, session browser, usage stats, and settings editor
- **Model Switching**: Switch between cloud/local models on demand
- **Session Persistence**: SQLite-backed conversation history
- **Fallback Chains**: Automatic failover when primary model fails
- **Hook Engine**: Confirmation system for sensitive operations
- **Tool Framework**: Shell, file, file patch, web-fetch, web-search, browser control, image analysis, media send, audio transcribe, system info
- **Docker Sandboxing**: Per-session container isolation for tool execution
- **Multi-Agent Routing**: Config-driven agent selection per sender/channel with tool profiles
- **Media Pipeline**: Image analysis, outbound attachments, audio transcription and native audio passthrough across all channels
- **Talk Mode (Wake Phrase)**: Optional wake-phrase gating (`audio.talk_mode`) with timed conversation windows
- **Capture Tools**: `screen.capture` and `camera.capture` tools for host capture workflows
- **Session Transfer**: Move conversations between frontends
- **CLI**: Full command-line interface (`flynn start`, `send`, `doctor`, `completion`, etc.)
- **Shell Completion**: Auto-generated completions for bash, zsh, and fish with `--install` flag
- **Cron Scheduling**: Automated messages on cron schedules with output routing
- **Inbound Webhooks**: HTTP endpoints that trigger agent processing with HMAC auth and template rendering
- **Heartbeat Monitor**: Periodic health checks (gateway, model, channels, memory, disk) with failure notifications
- **Gmail Pub/Sub Watcher**: Monitor Gmail inbox via Google Cloud Pub/Sub push notifications with polling fallback
- **Vector Memory Search**: Hybrid keyword + semantic search with embeddings (OpenAI, Gemini, Ollama, llama.cpp, Voyage AI)
- **Docker Deployment**: Multi-stage Dockerfile and docker-compose.yml for production containers
- **Health Diagnostics**: `flynn doctor` validates config, connectivity, and system state
- **MCP Integration**: External tool servers via Model Context Protocol
- **Skills System**: Extensible capability packages (bundled, managed, workspace tiers)
- **Gateway Lock**: Single-client mode — reject additional WebSocket connections when one is active
- **Tailscale Serve**: Auto-expose gateway via Tailscale Serve on daemon start with lifecycle management
- **DM Pairing Codes**: Allow unknown senders to pair with the bot via time-limited codes across all channels, with SQLite-backed persistence across restarts
- **Lane Queue**: Per-session FIFO queue serializes concurrent gateway requests
- **Node Capability Negotiation**: Optional companion-node registration and capability discovery over gateway RPC

## Quick Start

```bash
# Install flynn CLI command (build + global link)
make install

# Copy and configure
cp config/default.yaml ~/.config/flynn/config.yaml
# Edit config with your API keys and Telegram bot token

# Run
flynn start

# Or run without building
pnpm start
```

## CLI Commands

Flynn provides a full CLI via the `flynn` binary (or `npx tsx src/cli/index.ts` during development):

| Command | Description |
|---------|-------------|
| `flynn start` | Start the Flynn daemon (Telegram, WebChat, cron, Gmail watcher) |
| `flynn tui` | Launch the interactive terminal UI |
| `flynn send <message>` | Send a one-shot message and print the response |
| `flynn sessions` | List active sessions |
| `flynn doctor` | Validate config and check system health |
| `flynn config` | Show resolved configuration (secrets redacted) |
| `flynn backup` | Create a snapshot backup and optionally upload to MinIO |
| `flynn completion <shell>` | Generate shell completions (bash, zsh, fish) |
| `flynn setup` | Interactive setup wizard |
| `flynn gmail-auth` | Authenticate with Gmail via OAuth2 |
| `flynn gcal-auth` | Authenticate with Google Calendar via OAuth2 |
| `flynn skills` | List/install/manage skills |

### Examples

```bash
# Start daemon with custom config
flynn start --config ~/my-config.yaml

# One-shot query
flynn send "What's the weather in London?"

# Check system health
flynn doctor --config ~/.config/flynn/config.yaml

# Show current config (secrets masked)
flynn config

# List sessions
flynn sessions

# Create backup now (uses config backup + MinIO settings)
flynn backup

# Generate shell completions
flynn completion bash              # Print bash completions to stdout
flynn completion zsh --install     # Install zsh completions to ~/.zshrc
flynn completion fish --install    # Install fish completions to config
```

## Skills Registry

Flynn supports registry-backed skill discovery and install-by-id.

Registry source configuration:

```yaml
skills:
  registry_source: ~/.config/flynn/skills-registry.json
```

You can also set `FLYNN_SKILLS_REGISTRY_SOURCE` instead of `skills.registry_source`.

Common commands:

```bash
# Discover registry entries
flynn skills registry list
flynn skills registry list --search todo --publisher acme --json
flynn skills registry show todoist

# Install by registry id
flynn skills install --registry-id todoist

# For non-local registry sources (git/archive), explicit confirmation is required
flynn skills install --registry-id todoist --confirm
```

Registry metadata (`publisher`, `homepage`, `sha256`) is treated as declared and unverified. Skill scanner checks still run before installation succeeds.

## Configuration

Config location: `~/.config/flynn/config.yaml` (or set `FLYNN_CONFIG`)

```yaml
telegram:
  bot_token: "your-telegram-bot-token"
  allowed_chat_ids: [123456789]  # Your Telegram user ID

# Optional: Matrix
matrix:
  homeserver_url: "https://matrix.example.org"
  access_token: "${MATRIX_ACCESS_TOKEN}"
  allowed_room_ids: ["!room1:example.org"]
  require_mention: true

# Optional: Signal (signal-cli)
signal:
  account: "+15551234567"
  signal_cli_path: "signal-cli"
  allowed_numbers: ["+15550001111"]
  allowed_group_ids: []
  require_mention: true
  mention_name: "flynn"
  poll_interval_ms: 5000
  send_timeout_ms: 15000

# Optional: Microsoft Teams (Bot Framework)
teams:
  app_id: "${TEAMS_APP_ID}"
  app_password: "${TEAMS_APP_PASSWORD}"
  allowed_conversation_ids: []
  require_mention: true

# Bot Framework messaging endpoint should point to:
# POST https://<your-flynn-host>/teams/events

# Optional: Mattermost
mattermost:
  server_url: "${MATTERMOST_SERVER_URL}"
  bot_token: "${MATTERMOST_BOT_TOKEN}"
  allowed_channel_ids: []  # Recommended: explicit channel IDs
  require_mention: true
  mention_name: "flynn"
  poll_interval_ms: 3000

# Optional: Google Chat
google_chat:
  service_account_key_file: "~/.config/flynn/google-chat-service-account.json"
  # or:
  # service_account_json: "${GOOGLE_CHAT_SERVICE_ACCOUNT_JSON}"
  webhook_token: "${GOOGLE_CHAT_WEBHOOK_TOKEN}"
  allowed_space_names: []
  require_mention: true

# Google Chat messaging endpoint should point to:
# POST https://<your-flynn-host>/google-chat/events

# Optional: iMessage via BlueBubbles
bluebubbles:
  endpoint: "http://localhost:1234"
  api_key: "${BLUEBUBBLES_API_KEY}"
  webhook_token: "${BLUEBUBBLES_WEBHOOK_TOKEN}"
  allowed_chat_guids: []
  require_mention: true
  mention_name: "flynn"

# BlueBubbles webhook endpoint should point to:
# POST https://<your-flynn-host>/bluebubbles/events

# Optional: LINE
line:
  channel_access_token: "${LINE_CHANNEL_ACCESS_TOKEN}"
  channel_secret: "${LINE_CHANNEL_SECRET}"
  allowed_source_ids: []   # Empty = allow all users/groups/rooms
  require_mention: true
  mention_name: "flynn"

# LINE webhook endpoint should point to:
# POST https://<your-flynn-host>/line/events

# Optional: Feishu / Lark
feishu:
  app_id: "${FEISHU_APP_ID}"
  app_secret: "${FEISHU_APP_SECRET}"
  webhook_token: "${FEISHU_WEBHOOK_TOKEN}"
  allowed_chat_ids: []
  require_mention: true
  mention_name: "flynn"
  endpoint: "https://open.feishu.cn"

# Feishu webhook endpoint should point to:
# POST https://<your-flynn-host>/feishu/events

# Optional: Zalo
zalo:
  oa_access_token: "${ZALO_OA_ACCESS_TOKEN}"
  webhook_token: "${ZALO_WEBHOOK_TOKEN}"
  allowed_user_ids: []
  require_mention: true
  mention_name: "flynn"
  endpoint: "https://openapi.zalo.me"

# Zalo webhook endpoint should point to:
# POST https://<your-flynn-host>/zalo/events

models:
  default:
    provider: anthropic
    model: claude-opus-4-5-20251101
    api_key: sk-ant-api03-...
  local:
    provider: ollama
    model: qwen2.5:14b
  fallback_chain: [local]

hooks:
  confirm: [shell.*, file.write, file.patch]
  log: [web.*, file.read]
  silent: [notify]
```

## Safety Model

Flynn is designed to be safe-by-default when expanded beyond "chat":

- **Tool policy** restricts which tools are even available to a given context (profiles + allow/deny + per-agent/per-provider overrides).
- **Skills** can declare explicit capabilities (`manifest.json.permissions`) which are enforced at runtime.
- **Sandboxing** can isolate high-risk execution (shell/process) per-session via Docker.
- **Prompt-injection hardening** treats fetched content/tool output as untrusted data and blocks obviously unsafe tool calls when untrusted content is present.
- **Audit logs** record tool usage and approvals with redaction.

Details: `docs/security/SAFE_PERSONAL_AGENT.md`

## Agent-Oriented Architecture Diagram

If you want a fast mental model of where to start as an AI agent / contributor:

- `docs/architecture/AGENT_DIAGRAM.md`
- `docs/architecture/CONTRIBUTOR_MAP.md`
- `docs/architecture/TYPESCRIPT_MAP.md`
- `docs/architecture/SYMBOL_INDEX.md`

### Model Providers

| Provider | Config |
|----------|--------|
| Anthropic | `provider: anthropic`, `api_key` or `auth_token` |
| OpenAI | `provider: openai`, `api_key`, optional `endpoint` |
| Vercel AI Gateway | `provider: vercel`, `api_key` or `AI_GATEWAY_API_KEY`, optional `endpoint` |
| GitHub Copilot | `provider: github`, auto-login via OAuth device flow |
| Gemini | `provider: gemini`, `api_key` |
| Bedrock | `provider: bedrock`, AWS credentials |
| Ollama | `provider: ollama`, `model`, optional `endpoint` |
| Zhipu AI (GLM) | `provider: zhipuai`, `api_key` or `ZHIPUAI_API_KEY`, optional `endpoint` |
| xAI (Grok) | `provider: xai`, `api_key` or `XAI_API_KEY` |
| MiniMax | `provider: minimax`, `api_key` or `MINIMAX_API_KEY`, optional `endpoint` |
| Moonshot (Kimi) | `provider: moonshot`, `api_key` or `MOONSHOT_API_KEY`, optional `endpoint` |
| llama.cpp | `provider: llamacpp`, `endpoint` |

### Model Tiers

Configure multiple models for different purposes:

```yaml
models:
  fast: { provider: anthropic, model: claude-sonnet-4-... }
  default: { provider: anthropic, model: claude-opus-4-5-... }
  complex: { provider: anthropic, model: claude-opus-4-5-... }
  local: { provider: ollama, model: qwen2.5:14b }
```

Each tier can optionally specify `auth_mode` (`auto` | `api_key` | `oauth`) to control whether Flynn uses API keys vs OAuth/token auth for that provider. `use_oauth: true` remains supported as a compatibility alias for `auth_mode: oauth`.

### Native Audio Support

Voice messages from channels can be handled in two ways:

1. **Native passthrough** -- Audio sent directly to models that support audio input (Gemini, OpenAI, GitHub). No transcription step needed.
2. **Whisper transcription** -- Audio transcribed to text via a Whisper-compatible API, then sent as text to models that don't support audio input (Anthropic, Bedrock, Ollama, llama.cpp).

Flynn automatically routes based on the model's capabilities. You can override this per-tier:

```yaml
models:
  default:
    provider: gemini
    model: gemini-2.0-flash
    supports_audio: true     # Force native audio (auto-detected for known providers)
  fast:
    provider: anthropic
    model: claude-sonnet-4
    supports_audio: false    # Force transcription (default for Anthropic)
```

### Audio Transcription

Configure a Whisper-compatible endpoint for models that don't support native audio:

```yaml
audio:
  enabled: true
  provider:
    type: custom                            # openai, groq, ollama, llamacpp, custom
    endpoint: "http://localhost:18801/v1/audio/transcriptions"
    api_key: "${WHISPER_API_KEY}"            # Optional Bearer token
    model: "whisper-1"                       # Model name (default: whisper-1)
```

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable audio transcription (default: `false`) |
| `provider.type` | yes | Provider type: `openai`, `groq`, `ollama`, `llamacpp`, or `custom` |
| `provider.endpoint` | yes | Whisper-compatible API endpoint |
| `provider.api_key` | no | Bearer token for authentication |
| `provider.model` | no | Model name sent in request (default: `whisper-1`) |
| `talk_mode.enabled` | no | Enable wake-phrase talk mode gating (default: `false`) |
| `talk_mode.wake_phrase` | no | Phrase that activates talk mode (default: `hey flynn`) |
| `talk_mode.timeout_ms` | no | Active listen window after wake (default: `120000`) |
| `talk_mode.allow_manual_toggle` | no | Enable `/talk on|off|status` controls (default: `true`) |

Without an `audio` config, voice messages from non-audio-capable models will display an error message to the user. For local transcription, you can run a whisper.cpp server:

```bash
# Option 1: Manual docker run
docker run -d \
  --name whisper-server \
  -p 18801:8080 \
  ghcr.io/ggml-org/whisper.cpp:main \
  --model /app/models/ggml-base.en.bin \
  --host 0.0.0.0 \
  --port 8080 \
  --convert \
  --language en \
  --inference-path /v1/audio/transcriptions

# Option 2: Using docker-compose (uncomment whisper-server service in docker-compose.yml)
# docker compose up -d
```

### Capture Tools

Flynn includes host capture tools:
- `screen.capture` -> captures current screen and returns base64 image payload
- `camera.capture` -> captures one camera frame and returns base64 image payload

Notes:
- These are host-command wrappers and require platform binaries:
  - macOS: `screencapture` (screen), `imagesnap` (camera)
  - Linux: `grim` or ImageMagick `import` (screen), `ffmpeg` (camera)

## Telegram Commands

| Command | Description |
|---------|-------------|
| `/start` | Initialize bot |
| `/reset` | Clear conversation history |
| `/status` | Show current model and status |
| `/local` | Switch to local model |
| `/cloud` | Switch to cloud model |
| `/model` | Show model info and options |

## Web UI Dashboard

Flynn includes a built-in web control dashboard served by the WebSocket gateway. Access it at `http://localhost:18800` (or your configured gateway port).

### Pages

| Page | Description |
|------|-------------|
| **Dashboard** | System health cards, channel status, usage stats. Auto-refreshes every 10s |
| **Chat** | Session selector, streaming tool events, markdown rendering with syntax highlighting |
| **Sessions** | Browse all sessions, view message history, delete sessions |
| **Usage** | Token usage summary cards, per-session breakdown table, auto-refresh |
| **Settings** | Edit hook patterns (confirm/log/silent), view tools, channels, and redacted config |

The dashboard is a vanilla JS SPA with no build step — hash-based routing, ES modules, and the existing WebSocket JSON-RPC protocol.

## Terminal UI

```bash
# Minimal mode (readline)
pnpm tui

# Fullscreen mode (React/Ink)
pnpm tui:fs
```

### TUI Commands

| Command | Description |
|---------|-------------|
| `/help` | Show help |
| `/model` | Show all model tiers and which is active |
| `/model <tier>` | Switch active tier (`local`, `default`, `fast`, `complex`, or aliases `ollama`, `sonnet`, `haiku`, `opus`) |
| `/model <tier> <provider/model>` | Hot-swap a tier's provider and model at runtime |
| `/backend [provider]` | Show or switch local backend (`ollama`, `llamacpp`) |
| `/login [provider]` | Authenticate with GitHub (OAuth device flow) |
| `/reset` | Clear history |
| `/status` | Show session info |
| `/compact` | Compact conversation context |
| `/usage` | Show token usage and cost |
| `/verbose` | Toggle verbose output mode |
| `/pair` | Generate/list/revoke DM pairing codes |
| `/fullscreen` | Switch to fullscreen mode |
| `/transfer <dest>` | Transfer session to another frontend |
| `/quit` | Exit |

#### Runtime Model Switching

Switch providers and models on the fly without editing config or restarting:

```bash
# Show current tiers
/model

# Switch active tier
/model fast
/model complex

# Hot-swap a tier's provider/model
/model default anthropic/claude-sonnet-4
/model default zhipuai/glm-4.7
/model fast github/gpt-4o-mini
/model local ollama/glm-4.7-flash
```

The provider name must match a supported provider (`anthropic`, `openai`, `gemini`, `ollama`, `llamacpp`, `openrouter`, `vercel`, `bedrock`, `github`, `zhipuai`, `xai`, `minimax`, `moonshot`, `synthetic`). Tab completion is available for both tiers and provider names.

For cloud Zhipu models, ensure `ZHIPUAI_API_KEY` is set or `api_key` is configured in the relevant tier.

**Note:** The `/model` command works in the TUI only. WebChat sessions inherit the active tier from the daemon.

## Running as Service

```bash
# Create systemd user service
mkdir -p ~/.config/systemd/user

cat > ~/.config/systemd/user/flynn.service << 'EOF'
[Unit]
Description=Flynn Personal AI Assistant
After=network.target ollama.service

[Service]
Type=simple
WorkingDirectory=/path/to/flynn
ExecStart=/usr/bin/pnpm start
Restart=on-failure
RestartSec=5
Environment=NODE_ENV=production

[Install]
WantedBy=default.target
EOF

# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now flynn

# View logs
journalctl --user -u flynn -f
```

## Hook Engine

Control sensitive operations with pattern matching:

```yaml
hooks:
  confirm:    # Requires user approval via Telegram
    - shell.*
    - file.write
    - file.patch
  log:        # Logs but doesn't block
    - web.*
    - file.read
  silent:     # Executes without notification
    - notify
```

## Cron Scheduling

Schedule automated messages on cron schedules. Each job fires an inbound message through the agent pipeline and routes the response to a configured output channel.

Set `automation.delivery_mode` to control automation session behavior:
- `shared_session` (default): reuse one session per cron job/webhook name.
- `isolated_job`: create a fresh session per cron trigger/webhook request.

```yaml
automation:
  delivery_mode: shared_session
  cron:
    - name: daily-summary
      schedule: "0 9 * * *"            # 9 AM daily
      message: "Give me a summary of today's tasks"
      output:
        channel: telegram               # Route response to Telegram
        peer: "123456789"               # Chat ID to send to
      timezone: Europe/London           # Optional timezone
      enabled: true

    - name: hourly-check
      schedule: "0 * * * *"            # Every hour
      message: "Check system status"
      output:
        channel: telegram
        peer: "123456789"
      enabled: false                    # Disabled, won't fire
      model_tier: fast                  # Use fast tier for quick checks
```

### Cron Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `automation.delivery_mode` | no | Automation session strategy: `shared_session` or `isolated_job` (default: `shared_session`) |
| `name` | yes | Unique job identifier |
| `schedule` | yes | Cron expression (standard 5-field) |
| `message` | yes | Text sent to the agent when the job fires |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
| `timezone` | no | IANA timezone (defaults to system timezone) |
| `enabled` | no | Whether the job is active (default: `true`) |
| `model_tier` | no | Model tier for this job: `fast`, `default`, `complex`, or `local` |

## Inbound Webhooks

HTTP endpoints that trigger agent processing. Each webhook accepts POST requests, optionally verifies an HMAC signature, renders a message template, and routes the agent's response to an output channel.

```yaml
automation:
  delivery_mode: shared_session
  webhooks:
    - name: github-push
      secret: "whsec_..."                  # HMAC secret for signature verification
      message: "GitHub push to {{json.repository.full_name}}: {{json.head_commit.message}}"
      output:
        channel: telegram
        peer: "123456789"

    - name: alertmanager
      message: "Alert: {{json.alerts.0.annotations.summary}}"
      output:
        channel: discord
        peer: "channel-id"
```

Webhooks are available at `POST /webhooks/:name` on the gateway HTTP server. They bypass gateway token auth and use their own per-webhook HMAC verification instead.

### Webhook Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `automation.delivery_mode` | no | Automation session strategy: `shared_session` or `isolated_job` (default: `shared_session`) |
| `name` | yes | Unique webhook identifier (used in URL path) |
| `secret` | no | HMAC secret for `X-Webhook-Signature` header verification (SHA-256) |
| `message` | no | Template for the message sent to the agent (default: `{{body}}`) |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
| `enabled` | no | Whether the webhook is active (default: `true`) |

### Template Variables

| Variable | Description |
|----------|-------------|
| `{{body}}` | Raw request body as string |
| `{{json.field}}` | Extract a field from JSON body (dot notation for nested fields) |

## Heartbeat Monitor

Periodic health checks that validate system components and notify a configured channel on failure.

```yaml
automation:
  heartbeat:
    enabled: true
    interval: "5m"                         # Check every 5 minutes
    checks: [gateway, model, channels, memory, disk]
    notify:
      channel: telegram
      peer: "123456789"
    failure_threshold: 2                   # Notify after 2 consecutive failures
    disk_threshold_mb: 100                 # Warn when <100MB free
```

### Heartbeat Checks

| Check | What it validates |
|-------|-------------------|
| `gateway` | HTTP server is responsive |
| `model` | Model router has an active tier configured |
| `channels` | At least one channel adapter is connected |
| `memory` | Memory directory is readable and writable |
| `disk` | Free disk space exceeds threshold |

The monitor sends a notification when failures reach the configured threshold and a recovery notification when all checks pass again.

### Heartbeat Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable the heartbeat monitor (default: `false`) |
| `interval` | no | Check interval: `60s`, `5m`, `1h` (default: `5m`) |
| `checks` | no | Which checks to run (default: all five) |
| `notify.channel` | no | Channel to send failure/recovery notifications |
| `notify.peer` | no | Peer/chat ID for notifications |
| `failure_threshold` | no | Consecutive failures before notifying (default: `2`) |
| `disk_threshold_mb` | no | Disk space warning threshold in MB (default: `100`) |

## Gmail Pub/Sub Watcher

Monitor a Gmail inbox and forward new messages into the agent pipeline.

Supported delivery modes:
- **Push** (Gmail watch → Pub/Sub topic → HTTP push subscription → `POST /gmail/push`)
- **Pull** (Pub/Sub pull subscription → Flynn periodically pulls messages; no inbound webhook)
- **Polling** (Gmail History API polling fallback)

### Prerequisites

1. Create a Google Cloud project with the Gmail API enabled
2. Create OAuth2 credentials (Desktop application type) and download the JSON file
3. Run `flynn gmail-auth` to complete the OAuth2 flow and store the refresh token

For Pub/Sub delivery (push/pull), also enable the Pub/Sub API and create:
- A topic (e.g. `projects/your-project/topics/gmail-push`)
- A subscription (push and/or pull)

### Configuration

```yaml
automation:
  gmail:
    enabled: true
    credentials_file: ~/.config/flynn/gmail-credentials.json
    token_file: ~/.config/flynn/gmail-token.json   # Default location

    # Push mode (optional)
    pubsub_topic: projects/your-project/topics/gmail-push
    disable_push: false

    # Pull mode (optional; no inbound webhook required)
    pubsub_subscription_id: projects/your-project/subscriptions/gmail-pull
    pubsub_pull_interval: "60s"
    pubsub_max_messages: 10

    watch_labels: [INBOX]                           # Labels to watch
    poll_interval: "60s"                            # Polling fallback interval
    message: "New email from {{from}}: {{subject}}\n\n{{snippet}}"
    output:
      channel: telegram
      peer: "123456789"
```

Push notifications arrive at `POST /gmail/push` on the gateway HTTP server (bypasses gateway auth).

Pull mode uses Application Default Credentials (e.g. `GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json`) to access Pub/Sub.

### Gmail Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable the Gmail watcher (default: `false`) |
| `credentials_file` | yes | Path to Google OAuth2 credentials JSON |
| `token_file` | no | Path to stored OAuth2 refresh token (default: `~/.config/flynn/gmail-token.json`) |
| `pubsub_topic` | no | Pub/Sub topic for Gmail watch push notifications (`projects/<project>/topics/<topic>`) |
| `disable_push` | no | Disable watch registration even if `pubsub_topic` is set (default: `false`) |
| `pubsub_subscription_id` | no | Pub/Sub pull subscription (`projects/<project>/subscriptions/<sub>`) |
| `pubsub_pull_interval` | no | How often to pull subscription messages (default: `60s`) |
| `pubsub_max_messages` | no | Max messages per pull cycle (default: `10`) |
| `watch_labels` | no | Gmail labels to watch (default: `[INBOX]`) |
| `poll_interval` | no | Polling fallback interval: `60s`, `5m` (default: `300s`) |
| `history_start` | no | ISO date string — only process emails received after this date |
| `message` | no | Template for the agent message (default: `New email from {{from}}: {{subject}}\n\n{{snippet}}`) |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |

### Template Variables

| Variable | Description |
|----------|-------------|
| `{{from}}` | Sender address |
| `{{to}}` | Recipient address |
| `{{subject}}` | Email subject line |
| `{{snippet}}` | Gmail-provided message snippet |
| `{{date}}` | Email date |
| `{{id}}` | Gmail message ID |
| `{{labels}}` | Comma-separated label names |

## Google Calendar Tools

Query Google Calendar events from within conversations. Provides three tools: `calendar.today` (today's agenda), `calendar.list` (date range), and `calendar.search` (full-text search).

### Prerequisites

1. A Google Cloud project with the **Calendar API** enabled
2. OAuth2 credentials (Desktop application type) — the same credentials file used for Gmail works
3. Run `flynn gcal-auth` to complete the OAuth2 flow and store the refresh token

### Configuration

```yaml
automation:
  gcal:
    enabled: true
    credentials_file: ~/.config/flynn/gmail-credentials.json
    token_file: ~/.config/flynn/gcal-token.json   # Default location
    calendar_ids: [primary]                        # Calendar IDs to query
```

### Google Calendar Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable the calendar tools (default: `false`) |
| `credentials_file` | yes | Path to Google OAuth2 credentials JSON |
| `token_file` | no | Path to stored OAuth2 refresh token (default: `~/.config/flynn/gcal-token.json`) |
| `calendar_ids` | no | Calendar IDs available for queries (default: `[primary]`) |

## Vector Memory Search

The memory system supports hybrid search combining keyword matching with semantic vector similarity. When embeddings are enabled, `memory.search` uses both approaches and merges results with configurable weighting.

Memory persistence is hybrid:
- Manual writes via `memory.write`
- Automatic fact extraction during context compaction when `memory.auto_extract: true`

```yaml
memory:
  enabled: true
  auto_extract: true
  max_context_tokens: 2000
  embedding:
    enabled: true
    provider: openai                       # openai, gemini, ollama, llamacpp
    model: text-embedding-3-small
    api_key: "${OPENAI_API_KEY}"
    chunk_size: 512                        # Tokens per chunk
    chunk_overlap: 50                      # Overlap between chunks
    top_k: 5                               # Top results from vector search
    hybrid_weight: 0.7                     # 0.0 = keyword only, 1.0 = vector only
  qmd:
    enabled: false                         # Experimental markdown-native search backend
    top_k: 8                               # Max QMD results
    min_score: 0.15                        # Minimum match score (0-1)
```

### Embedding Providers

| Provider | Config |
|----------|--------|
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
| Gemini | `provider: gemini`, `api_key`, `model` |
| Ollama | `provider: ollama`, `endpoint` (default: localhost:11434), `model` |
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |
| Voyage AI | `provider: voyageai`, `api_key` or `VOYAGE_API_KEY`, `model` (default: `voyage-3-large`) |

Embeddings are indexed in the background — when memory is written, the namespace is marked dirty and re-indexed within 30 seconds. The vector index is stored in `vectors.db` alongside the session database.

Search backend selection:

- `memory.embedding.enabled: true` -> hybrid keyword+vector backend
- `memory.embedding.enabled: false` and `memory.qmd.enabled: true` -> QMD markdown backend
- otherwise -> keyword-only fallback

When the selected backend is unavailable (for example embedding provider errors), search falls back gracefully to keyword matching.

`memory.auto_extract` controls whether compaction appends extracted durable facts to `global` memory.

### Embedding Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable vector search (default: `false`) |
| `provider` | no | Embedding provider (default: `openai`) |
| `model` | no | Embedding model name (default: `text-embedding-3-small`) |
| `endpoint` | no | Provider endpoint (required for `ollama`/`llamacpp`) |
| `api_key` | no | API key (required for `openai`/`gemini`/`voyageai`) |
| `dimensions` | no | Vector dimensions (auto-detected from model if not set) |
| `chunk_size` | no | Max tokens per chunk (default: `512`) |
| `chunk_overlap` | no | Token overlap between chunks (default: `50`) |
| `top_k` | no | Number of vector results to return (default: `5`) |
| `hybrid_weight` | no | Vector vs keyword weight, 0.0-1.0 (default: `0.7`) |

### QMD Config Fields

| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable experimental markdown-native QMD backend (default: `false`) |
| `top_k` | no | Max QMD results returned by `memory.search` (default: `8`) |
| `min_score` | no | Minimum relevance score (0.0-1.0) for QMD matches (default: `0.15`) |

## Session End Summary

Optionally summarize conversations when a WebSocket session ends and append the summary to memory.

```yaml
sessions:
  end_summary:
    enabled: true
    tier: fast
    memory_namespace: session/summaries
```

## Audit Trail

Flynn writes structured audit events to `audit.path`, including tool execution, session lifecycle, and user actions (`user.action`) from both channel and gateway requests.

## Gateway Lock

Single-client mode for the WebSocket gateway. When enabled, only one WebSocket connection is allowed at a time. Additional connections are rejected with close code `4003`.

```yaml
server:
  lock: true
```

The web UI detects the locked state and disables auto-reconnect when rejected.

## Gateway WebSocket Rate Limit

Per-connection ingress throttling for WebSocket requests. Excess bursts are rejected; repeated violations close the connection (`4008`).

```yaml
server:
  ws_rate_limit:
    enabled: true
    capacity: 30
    refill_per_sec: 15
    max_violations: 8
    violation_window_ms: 10000
```

## Gateway Lane Queue Policy

Per-session FIFO queue policy for concurrent gateway requests (`agent.send`).

```yaml
server:
  queue:
    mode: collect      # collect | followup | steer | steer_backlog | interrupt
    cap: 50            # max pending requests per session lane
    overflow: drop_old # drop_old | drop_new
    debounce_ms: 0     # delay before running next queued item
    summarize_overflow: true
    overrides:
      channels:
        ws:
          mode: followup
          cap: 10
      sessions:
        ws:vip-user:
          mode: interrupt
          overflow: drop_new
          debounce_ms: 100
```

Notes:
- `collect` keeps all queued requests (subject to `cap`).
- `followup` keeps at most one pending item while a request is active; newer followups replace older pending items.
- `steer` and `steer_backlog` replace pending backlog with the newest request while one is active.
- `interrupt` uses steer-backlog queueing behavior; active work still requires `agent.cancel` for best-effort cancellation.
- `interrupt` currently does not force-stop already running work; use `agent.cancel` for active cancellation.
- `debounce_ms` delays the next queued execution, helping collapse bursty same-session traffic.
- `summarize_overflow` enables richer overflow error messages and payload metadata.
- On overflow, `drop_old` evicts the oldest pending request, `drop_new` rejects the new request.
- Override precedence: exact `sessions` match first, then `channels`.

Runtime session controls from chat commands:
- `/queue` shows effective session queue policy.
- `/queue set <mode|cap|overflow|debounce_ms|summarize_overflow> <value>` sets a per-session override.
- `/queue reset` clears per-session queue overrides.

## Gateway Node Capability Negotiation

Optional gateway surface for companion clients and node-role negotiation:

```yaml
server:
  nodes:
    enabled: true
    allowed_roles: [companion]
    feature_gates:
      ui.canvas: true
    location:
      enabled: true
    push:
      enabled: true
```

Methods:
- `node.register` registers role + declared capabilities for the current connection.
- `node.capabilities.get` returns negotiated protocol version and enabled capabilities.
- `node.location.set` updates the node's last-known location (when `server.nodes.location.enabled` is true).
- `node.location.get` returns the node's stored location payload.
- `node.status.set` publishes companion status/heartbeat fields (`platform`, `appVersion`, `batteryPct`, etc.).
- `node.push_token.set` registers node push tokens (APNs for iOS/macOS, FCM for Android) when `server.nodes.push.enabled` is true.
- `system.location` provides an operator view of registered node locations.
- `system.nodes` returns registered node snapshots (role, capabilities, identity, location/status).
- `system.capabilities` returns gateway protocol and node policy snapshot.

Companion runtime helper:
- `src/companion/runtimeClient.ts` provides a typed Node/WebSocket client for companion runtimes (macOS/iOS/Android workers) with wrappers for `node.register`, `node.capabilities.get`, `node.location.set/get`, `node.status.set`, `node.push_token.set`, `system.capabilities`, and `system.nodes`.

## Canvas / A2UI Foundation

Gateway provides a session-scoped canvas artifact API for companion/UI surfaces:
- `canvas.put` upserts an artifact (`artifactId`, `type`, `content`, optional `title`/`metadata`).
- `canvas.get` retrieves a single artifact.
- `canvas.list` lists artifacts for a session (most recently updated first).
- `canvas.delete` removes one artifact.
- `canvas.clear` removes all artifacts for a session.

This foundation is currently in-memory (runtime ephemeral) and intended as the first step for richer visual workspace flows.

## Gateway Request Body Limit

Cap inbound HTTP POST body size (webhooks and Gmail push) to reduce memory-DoS risk.

```yaml
server:
  max_request_body_bytes: 1048576  # 1 MiB
```

## Tailscale Serve

Automatically expose the gateway via Tailscale Serve when the daemon starts. Requires Tailscale to be installed and authenticated on the host.

```yaml
server:
  tailscale:
    serve: true
```

When enabled, Flynn runs `tailscale serve` on startup to expose the gateway port over your tailnet, and cleans up on shutdown. The `flynn doctor` command includes a Tailscale availability check.

## Bonjour / mDNS Discovery

Optionally advertise the gateway on your local network so LAN clients can discover Flynn without manual host entry.

```yaml
server:
  localhost: false
  discovery:
    enabled: true
    service_name: flynn-gateway
    service_type: _flynn._tcp
    txt:
      env: home-lab
```

Notes:
- Discovery is disabled by default.
- `server.localhost` must be `false` for LAN clients to connect.
- Flynn advertises non-secret metadata only (instance/version + optional `txt` keys you provide).
- Runtime uses host tools (`avahi-publish-service` on Linux, `dns-sd` on macOS) when available.

## DM Pairing Codes

Allow unknown senders to authenticate with the bot via time-limited pairing codes. Works across all channel adapters (Telegram, Discord, Slack, WhatsApp).

```yaml
pairing:
  enabled: true
  code_ttl: "10m"       # Code expiry time (default: 10 minutes)
  code_length: 6        # Code length (default: 6 digits)
```

### How it works

1. Generate a code via the TUI (`/pair generate`), gateway API (`pairing.generate`), or web dashboard
2. Share the code with the user
3. The user sends the code as their first DM to the bot
4. If valid, the user's sender ID is permanently approved for that channel (persisted in SQLite, survives daemon restarts)
5. Approved users can be listed (`/pair list`) and revoked (`/pair revoke <channel> <id>`)

### TUI Commands

| Command | Description |
|---------|-------------|
| `/pair` | Generate a new pairing code |
| `/pair generate [label]` | Generate a code with optional label |
| `/pair list` | List pending codes and approved senders |
| `/pair revoke <channel> <id>` | Revoke an approved sender |

## WhatsApp Chromium Sandbox

WhatsApp adapter now launches Chromium in sandboxed mode by default.

If you must disable Chromium sandboxing in a high-trust/containerized environment:

```yaml
whatsapp:
  no_sandbox: true
```

### Gateway API

| Method | Description |
|--------|-------------|
| `pairing.generate` | Generate a new pairing code (optional `label` param) |
| `pairing.list` | List pending codes and approved senders |
| `pairing.revoke` | Revoke an approved sender (`channel` + `senderId` params) |
| `system.presence` | List observed sender presence with online/offline status inferred from recent inbound activity |

## Shell Completion

Generate shell completions for bash, zsh, or fish:

```bash
# Print completions to stdout
flynn completion bash
flynn completion zsh
flynn completion fish

# Install directly to shell config
flynn completion bash --install    # Appends to ~/.bashrc
flynn completion zsh --install     # Appends to ~/.zshrc
flynn completion fish --install    # Writes to ~/.config/fish/completions/flynn.fish
```

## Docker Deployment

Flynn includes a production-ready Dockerfile with multi-stage build.

```bash
# Build the image
docker build -t flynn .

# Run with config and data volumes
docker run -d \
  --name flynn \
  -p 18800:18800 \
  -v ./config.yaml:/config/config.yaml:ro \
  -v flynn-data:/data \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  flynn
```

Or use the included `docker-compose.yml`:

```bash
# Copy your config
cp ~/.config/flynn/config.yaml ./config.yaml

# Start with compose
docker compose up -d

# View logs
docker compose logs -f
```

### Docker Environment Variables

| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Config file path (default: `/config/config.yaml`) |
| `FLYNN_DATA_DIR` | Data directory path (default: `/data`) |
| `ANTHROPIC_API_KEY` | Anthropic API key |
| `OPENAI_API_KEY` | OpenAI API key |
| `TELEGRAM_BOT_TOKEN` | Telegram bot token |

### Volumes

| Mount Point | Purpose |
|-------------|---------|
| `/config/config.yaml` | Configuration file (read-only) |
| `/data` | Persistent data (sessions DB, memory files, vector index) |

## PaaS Deployment (Fly.io / Railway / Render)

See `docs/deployment/PAAS.md` and the templates under `deploy/`.

## Nix Deployment

See `docs/deployment/NIX.md` for the flake (package + dev shell + optional NixOS module).

## Doctor Diagnostics

`flynn doctor` runs health checks to validate your setup:

```
$ flynn doctor

Flynn Doctor
============

[PASS] Config file exists (/home/user/.config/flynn/config.yaml)
[PASS] Config parses (valid YAML)
[PASS] Config validates (schema valid)
[PASS] Env vars resolved
[PASS] Data directory writable (/home/user/.local/share/flynn)
[PASS] Session DB accessible (sessions.db)
[PASS] Model connectivity (anthropic: claude-sonnet)
[PASS] Telegram bot configured (1 allowed chat(s))
[SKIP] MCP servers configured (none configured)
[PASS] Skills loaded (3 skill(s))

Results: 8 passed, 0 failed, 0 warnings, 1 skipped
```

### Check Details

| Check | What it validates |
|-------|-------------------|
| Config file exists | Config YAML file is present at the expected path |
| Config parses | File is valid YAML syntax |
| Config validates | YAML content passes Zod schema validation |
| Env vars resolved | Any `${VAR}` references in config have values set |
| Data directory writable | Can write to `~/.local/share/flynn/` |
| Session DB accessible | SQLite database opens and queries succeed |
| Model connectivity | Default model provider and model name are configured |
| Telegram bot configured | Bot token is present and reasonable length |
| MCP servers configured | Lists configured MCP tool servers |
| Skills loaded | Discovers and loads skill packages |
| Skills registry | Verifies registry source is configured and catalog is reachable/parsible |

Exit code is `1` if any check fails, `0` otherwise. Checks that depend on a valid config are skipped when config is invalid.

## Session Management

- Sessions persist in `~/.local/share/flynn/sessions.db`
- Session ID format: `{frontend}:{userId}` (e.g., `telegram:123456789`)
- History survives restarts
- Transfer sessions between frontends with `/transfer`

## Architecture

```
src/
├── agents/               # Multi-agent routing
├── auth/                 # OAuth flows (GitHub Copilot)
├── backends/native/      # Agent implementation + orchestrator
├── channels/             # Channel adapters (Telegram, Discord, Slack, WhatsApp, Matrix, WebChat)
│   └── pairing.ts        # DM pairing code manager
├── cli/                  # CLI commands (commander)
│   └── completion.ts     # Shell completion generator (bash/zsh/fish)
├── config/               # YAML config + Zod validation
├── context/              # Token estimation + compaction
├── daemon/               # Lifecycle management + routing
├── frontends/
│   ├── telegram/         # Telegram bot
│   └── tui/              # Terminal UI (minimal + fullscreen)
├── gateway/              # WebSocket gateway + web UI dashboard
│   ├── handlers/         # JSON-RPC method handlers (agent, sessions, system, pairing, tools, config)
│   ├── tailscale.ts      # Tailscale Serve lifecycle management
│   ├── lane-queue.ts     # Per-session FIFO request queue
│   └── ui/               # SPA dashboard (vanilla JS)
│       ├── pages/        # Dashboard, Chat, Sessions, Usage, Settings
│       └── lib/          # WebSocket RPC client
├── hooks/                # Confirmation engine
├── mcp/                  # MCP tool server integration
├── memory/               # Persistent memory store + vector search
├── models/               # Model providers + router + media pipeline + audio capabilities
├── prompt/               # System prompt templating (auto-injects current date/time)
├── sandbox/              # Docker sandboxing
├── session/              # SQLite persistence
├── skills/               # Skill packages
├── tools/                # Builtin tools (shell, file, web, browser, process, media, audio, system.info)
└── automation/           # Cron scheduler, webhooks, heartbeat monitor, Gmail watcher
```

## System Prompt

Flynn assembles its system prompt from layered template files (`SOUL.md`, `AGENTS.md`, `IDENTITY.md`, `USER.md`, `TOOLS.md`) searched in configurable directories. The first match per file wins.

A **Runtime Context** section is automatically appended to every system prompt with the current date and time, so the model always knows when "now" is without needing a tool call.

The `system.info` tool provides on-demand access to more detailed runtime information: current date/time, hostname, platform, architecture, OS release, uptime, Node.js version, memory usage, and working directory.

## Development

```bash
# Dev mode with watch
pnpm dev          # Daemon
pnpm tui:dev      # TUI

# Type check
pnpm typecheck

# Lint
pnpm lint

# Test
pnpm test
```

## Environment Variables

| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Override config file path |
| `FLYNN_DATA_DIR` | Override data directory (default: `~/.local/share/flynn`) |
| `ANTHROPIC_API_KEY` | Anthropic API key (fallback) |
| `OPENAI_API_KEY` | OpenAI API key (fallback) |

## License

MIT