Flynn

Self-hosted personal AI assistant with Telegram and Terminal interfaces.

Features

• Multi-Frontend: Telegram bot + Terminal UI (minimal & fullscreen modes) + Web UI dashboard
• Multi-Model: Anthropic Claude, OpenAI, GitHub Copilot, Gemini, Bedrock, Ollama, llama.cpp with intelligent routing
• Multi-Channel: Telegram, Discord, Slack, WhatsApp with unified adapter interface
• Web Dashboard: SPA control panel with health monitoring, chat, session browser, and settings editor
• Model Switching: Switch between cloud/local models on demand
• Session Persistence: SQLite-backed conversation history
• Fallback Chains: Automatic failover when primary model fails
• Hook Engine: Confirmation system for sensitive operations
• Tool Framework: Shell, file, file patch, web-fetch, web-search, browser control, image analysis, media send, system info
• Docker Sandboxing: Per-session container isolation for tool execution
• Multi-Agent Routing: Config-driven agent selection per sender/channel with tool profiles
• Media Pipeline: Image analysis, outbound attachments, audio transcription across all channels
• Session Transfer: Move conversations between frontends
• CLI: Full command-line interface (flynn start, send, doctor, etc.)
• Cron Scheduling: Automated messages on cron schedules with output routing
• Inbound Webhooks: HTTP endpoints that trigger agent processing with HMAC auth and template rendering
• Heartbeat Monitor: Periodic health checks (gateway, model, channels, memory, disk) with failure notifications
• Gmail Pub/Sub Watcher: Monitor Gmail inbox via Google Cloud Pub/Sub push notifications with polling fallback
• Vector Memory Search: Hybrid keyword + semantic search with embeddings (OpenAI, Gemini, Ollama, llama.cpp)
• Docker Deployment: Multi-stage Dockerfile and docker-compose.yml for production containers
• Health Diagnostics: flynn doctor validates config, connectivity, and system state
• MCP Integration: External tool servers via Model Context Protocol
• Skills System: Extensible capability packages (bundled, managed, workspace tiers)

Quick Start

# Install dependencies
pnpm install

# Copy and configure
cp config/default.yaml ~/.config/flynn/config.yaml
# Edit config with your API keys and Telegram bot token

# Build and run
pnpm build
flynn start

# Or run without building
pnpm start

CLI Commands

Flynn provides a full CLI via the flynn binary (or npx tsx src/cli/index.ts during development):

Command	Description
flynn start	Start the Flynn daemon (Telegram, WebChat, cron, Gmail watcher)
flynn tui	Launch the interactive terminal UI
flynn send <message>	Send a one-shot message and print the response
flynn sessions	List active sessions
flynn doctor	Validate config and check system health
flynn config	Show resolved configuration (secrets redacted)

Examples

# Start daemon with custom config
flynn start --config ~/my-config.yaml

# One-shot query
flynn send "What's the weather in London?"

# Check system health
flynn doctor --config ~/.config/flynn/config.yaml

# Show current config (secrets masked)
flynn config

# List sessions
flynn sessions

Configuration

Config location: ~/.config/flynn/config.yaml (or set FLYNN_CONFIG)

telegram:
  bot_token: "your-telegram-bot-token"
  allowed_chat_ids: [123456789]  # Your Telegram user ID

models:
  default:
    provider: anthropic
    model: claude-opus-4-5-20251101
    api_key: sk-ant-api03-...
  local:
    provider: ollama
    model: qwen2.5:14b
  fallback_chain: [local]

hooks:
  confirm: [shell.*, file.write, file.patch]
  log: [web.*, file.read]
  silent: [notify]

Model Providers

Provider	Config
Anthropic	provider: anthropic, api_key or auth_token
OpenAI	provider: openai, api_key, optional endpoint
GitHub Copilot	provider: github, auto-login via OAuth device flow
Gemini	provider: gemini, api_key
Bedrock	provider: bedrock, AWS credentials
Ollama	provider: ollama, model, optional endpoint
llama.cpp	provider: llamacpp, endpoint

Model Tiers

Configure multiple models for different purposes:

models:
  fast: { provider: anthropic, model: claude-sonnet-4-... }
  default: { provider: anthropic, model: claude-opus-4-5-... }
  complex: { provider: anthropic, model: claude-opus-4-5-... }
  local: { provider: ollama, model: qwen2.5:14b }

Telegram Commands

Command	Description
/start	Initialize bot
/reset	Clear conversation history
/status	Show current model and status
/local	Switch to local model
/cloud	Switch to cloud model
/model	Show model info and options

Web UI Dashboard

Flynn includes a built-in web control dashboard served by the WebSocket gateway. Access it at http://localhost:18800 (or your configured gateway port).

Pages

Page	Description
Dashboard	System health cards, channel status, usage stats. Auto-refreshes every 10s
Chat	Session selector, streaming tool events, markdown rendering with syntax highlighting
Sessions	Browse all sessions, view message history, delete sessions
Settings	Edit hook patterns (confirm/log/silent), view tools, channels, and redacted config

The dashboard is a vanilla JS SPA with no build step — hash-based routing, ES modules, and the existing WebSocket JSON-RPC protocol.

Terminal UI

# Minimal mode (readline)
pnpm tui

# Fullscreen mode (React/Ink)
pnpm tui:fs

TUI Commands

Command	Description
/help	Show help
/reset	Clear history
/status	Show session info
/fullscreen	Switch to fullscreen mode
/transfer telegram	Transfer session to Telegram
/quit	Exit

Running as Service

# Create systemd user service
mkdir -p ~/.config/systemd/user

cat > ~/.config/systemd/user/flynn.service << 'EOF'
[Unit]
Description=Flynn Personal AI Assistant
After=network.target ollama.service

[Service]
Type=simple
WorkingDirectory=/path/to/flynn
ExecStart=/usr/bin/pnpm start
Restart=on-failure
RestartSec=5
Environment=NODE_ENV=production

[Install]
WantedBy=default.target
EOF

# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now flynn

# View logs
journalctl --user -u flynn -f

Hook Engine

Control sensitive operations with pattern matching:

hooks:
  confirm:    # Requires user approval via Telegram
    - shell.*
    - file.write
    - file.patch
  log:        # Logs but doesn't block
    - web.*
    - file.read
  silent:     # Executes without notification
    - notify

Cron Scheduling

Schedule automated messages on cron schedules. Each job fires an inbound message through the agent pipeline and routes the response to a configured output channel.

automation:
  cron:
    - name: daily-summary
      schedule: "0 9 * * *"            # 9 AM daily
      message: "Give me a summary of today's tasks"
      output:
        channel: telegram               # Route response to Telegram
        peer: "123456789"               # Chat ID to send to
      timezone: Europe/London           # Optional timezone
      enabled: true

    - name: hourly-check
      schedule: "0 * * * *"            # Every hour
      message: "Check system status"
      output:
        channel: telegram
        peer: "123456789"
      enabled: false                    # Disabled, won't fire

Cron Config Fields

Field	Required	Description
name	yes	Unique job identifier
schedule	yes	Cron expression (standard 5-field)
message	yes	Text sent to the agent when the job fires
output.channel	yes	Channel name to route the response (e.g. telegram)
output.peer	yes	Peer/chat ID on the output channel
timezone	no	IANA timezone (defaults to system timezone)
enabled	no	Whether the job is active (default: true)

Inbound Webhooks

HTTP endpoints that trigger agent processing. Each webhook accepts POST requests, optionally verifies an HMAC signature, renders a message template, and routes the agent's response to an output channel.

automation:
  webhooks:
    - name: github-push
      secret: "whsec_..."                  # HMAC secret for signature verification
      message: "GitHub push to {{json.repository.full_name}}: {{json.head_commit.message}}"
      output:
        channel: telegram
        peer: "123456789"

    - name: alertmanager
      message: "Alert: {{json.alerts.0.annotations.summary}}"
      output:
        channel: discord
        peer: "channel-id"

Webhooks are available at POST /webhooks/:name on the gateway HTTP server. They bypass gateway token auth and use their own per-webhook HMAC verification instead.

Webhook Config Fields

Field	Required	Description
name	yes	Unique webhook identifier (used in URL path)
secret	no	HMAC secret for X-Webhook-Signature header verification (SHA-256)
message	no	Template for the message sent to the agent (default: {{body}})
output.channel	yes	Channel name to route the response (e.g. telegram)
output.peer	yes	Peer/chat ID on the output channel
enabled	no	Whether the webhook is active (default: true)

Template Variables

Variable	Description
{{body}}	Raw request body as string
{{json.field}}	Extract a field from JSON body (dot notation for nested fields)

Heartbeat Monitor

Periodic health checks that validate system components and notify a configured channel on failure.

automation:
  heartbeat:
    enabled: true
    interval: "5m"                         # Check every 5 minutes
    checks: [gateway, model, channels, memory, disk]
    notify:
      channel: telegram
      peer: "123456789"
    failure_threshold: 2                   # Notify after 2 consecutive failures
    disk_threshold_mb: 100                 # Warn when <100MB free

Heartbeat Checks

Check	What it validates
gateway	HTTP server is responsive
model	Model router has an active tier configured
channels	At least one channel adapter is connected
memory	Memory directory is readable and writable
disk	Free disk space exceeds threshold

The monitor sends a notification when failures reach the configured threshold and a recovery notification when all checks pass again.

Heartbeat Config Fields

Field	Required	Description
enabled	no	Enable the heartbeat monitor (default: false)
interval	no	Check interval: 60s, 5m, 1h (default: 5m)
checks	no	Which checks to run (default: all five)
notify.channel	no	Channel to send failure/recovery notifications
notify.peer	no	Peer/chat ID for notifications
failure_threshold	no	Consecutive failures before notifying (default: 2)
disk_threshold_mb	no	Disk space warning threshold in MB (default: 100)

Gmail Pub/Sub Watcher

Monitor a Gmail inbox via Google Cloud Pub/Sub push notifications. New emails trigger the agent pipeline and route responses to a configured output channel. Falls back to polling when push notifications are unavailable.

Prerequisites

1. Create a Google Cloud project with the Gmail API and Pub/Sub API enabled
2. Create OAuth2 credentials (Desktop application type) and download the JSON file
3. Create a Pub/Sub topic (e.g. projects/your-project/topics/gmail-push)
4. Run flynn gmail-auth to complete the OAuth2 flow and store the refresh token

Configuration

automation:
  gmail:
    enabled: true
    credentials_file: ~/.config/flynn/gmail-credentials.json
    token_file: ~/.config/flynn/gmail-token.json   # Default location
    watch_labels: [INBOX]                           # Labels to watch
    poll_interval: "60s"                            # Polling fallback interval
    message: "New email from {{from}}: {{subject}}\n\n{{snippet}}"
    output:
      channel: telegram
      peer: "123456789"

Push notifications arrive at POST /gmail/push on the gateway HTTP server (bypasses gateway auth).

Gmail Config Fields

Field	Required	Description
enabled	no	Enable the Gmail watcher (default: false)
credentials_file	yes	Path to Google OAuth2 credentials JSON
token_file	no	Path to stored OAuth2 refresh token (default: ~/.config/flynn/gmail-token.json)
watch_labels	no	Gmail labels to watch (default: [INBOX])
poll_interval	no	Polling fallback interval: 60s, 5m (default: 60s)
history_start	no	ISO date string — only process emails received after this date
message	no	Template for the agent message (default: New email from {{from}}: {{subject}}\n\n{{snippet}})
output.channel	yes	Channel name to route the response (e.g. telegram)
output.peer	yes	Peer/chat ID on the output channel

Template Variables

Variable	Description
{{from}}	Sender address
{{to}}	Recipient address
{{subject}}	Email subject line
{{snippet}}	Gmail-provided message snippet
{{date}}	Email date
{{id}}	Gmail message ID
{{labels}}	Comma-separated label names

Vector Memory Search

The memory system supports hybrid search combining keyword matching with semantic vector similarity. When embeddings are enabled, memory.search uses both approaches and merges results with configurable weighting.

memory:
  enabled: true
  auto_extract: true
  max_context_tokens: 2000
  embedding:
    enabled: true
    provider: openai                       # openai, gemini, ollama, llamacpp
    model: text-embedding-3-small
    api_key: "${OPENAI_API_KEY}"
    chunk_size: 512                        # Tokens per chunk
    chunk_overlap: 50                      # Overlap between chunks
    top_k: 5                               # Top results from vector search
    hybrid_weight: 0.7                     # 0.0 = keyword only, 1.0 = vector only

Embedding Providers

Provider	Config
OpenAI	provider: openai, api_key, model (default: text-embedding-3-small)
Gemini	provider: gemini, api_key, model
Ollama	provider: ollama, endpoint (default: localhost:11434), model
llama.cpp	provider: llamacpp, endpoint, optional model

Embeddings are indexed in the background — when memory is written, the namespace is marked dirty and re-indexed within 30 seconds. The vector index is stored in vectors.db alongside the session database.

When embeddings are disabled or the provider is unreachable, search falls back gracefully to keyword matching.

Embedding Config Fields

Field	Required	Description
enabled	no	Enable vector search (default: false)
provider	no	Embedding provider (default: openai)
model	no	Embedding model name (default: text-embedding-3-small)
endpoint	no	Provider endpoint (required for ollama/llamacpp)
api_key	no	API key (required for openai/gemini)
dimensions	no	Vector dimensions (auto-detected from model if not set)
chunk_size	no	Max tokens per chunk (default: 512)
chunk_overlap	no	Token overlap between chunks (default: 50)
top_k	no	Number of vector results to return (default: 5)
hybrid_weight	no	Vector vs keyword weight, 0.0-1.0 (default: 0.7)

Docker Deployment

Flynn includes a production-ready Dockerfile with multi-stage build.

# Build the image
docker build -t flynn .

# Run with config and data volumes
docker run -d \
  --name flynn \
  -p 18800:18800 \
  -v ./config.yaml:/config/config.yaml:ro \
  -v flynn-data:/data \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  flynn

Or use the included docker-compose.yml:

# Copy your config
cp ~/.config/flynn/config.yaml ./config.yaml

# Start with compose
docker compose up -d

# View logs
docker compose logs -f

Docker Environment Variables

Variable	Description
FLYNN_CONFIG	Config file path (default: /config/config.yaml)
FLYNN_DATA_DIR	Data directory path (default: /data)
ANTHROPIC_API_KEY	Anthropic API key
OPENAI_API_KEY	OpenAI API key
TELEGRAM_BOT_TOKEN	Telegram bot token

Volumes

Mount Point	Purpose
/config/config.yaml	Configuration file (read-only)
/data	Persistent data (sessions DB, memory files, vector index)

Doctor Diagnostics

flynn doctor runs 10 health checks to validate your setup:

$ flynn doctor

Flynn Doctor
============

[PASS] Config file exists (/home/user/.config/flynn/config.yaml)
[PASS] Config parses (valid YAML)
[PASS] Config validates (schema valid)
[PASS] Env vars resolved
[PASS] Data directory writable (/home/user/.local/share/flynn)
[PASS] Session DB accessible (sessions.db)
[PASS] Model connectivity (anthropic: claude-sonnet)
[PASS] Telegram bot configured (1 allowed chat(s))
[SKIP] MCP servers configured (none configured)
[PASS] Skills loaded (3 skill(s))

Results: 8 passed, 0 failed, 0 warnings, 1 skipped

Check Details

Check	What it validates
Config file exists	Config YAML file is present at the expected path
Config parses	File is valid YAML syntax
Config validates	YAML content passes Zod schema validation
Env vars resolved	Any ${VAR} references in config have values set
Data directory writable	Can write to ~/.local/share/flynn/
Session DB accessible	SQLite database opens and queries succeed
Model connectivity	Default model provider and model name are configured
Telegram bot configured	Bot token is present and reasonable length
MCP servers configured	Lists configured MCP tool servers
Skills loaded	Discovers and loads skill packages

Exit code is 1 if any check fails, 0 otherwise. Checks that depend on a valid config are skipped when config is invalid.

Session Management

• Sessions persist in ~/.local/share/flynn/sessions.db
• Session ID format: {frontend}:{userId} (e.g., telegram:123456789)
• History survives restarts
• Transfer sessions between frontends with /transfer

Architecture

src/
├── agents/               # Multi-agent routing
├── auth/                 # OAuth flows (GitHub Copilot)
├── backends/native/      # Agent implementation + orchestrator
├── channels/             # Channel adapters (Telegram, Discord, Slack, WhatsApp, WebChat)
├── cli/                  # CLI commands (commander)
├── config/               # YAML config + Zod validation
├── context/              # Token estimation + compaction
├── daemon/               # Lifecycle management + routing
├── frontends/
│   ├── telegram/         # Telegram bot
│   └── tui/              # Terminal UI (minimal + fullscreen)
├── gateway/              # WebSocket gateway + web UI dashboard
│   ├── handlers/         # JSON-RPC method handlers
│   └── ui/               # SPA dashboard (vanilla JS)
│       ├── pages/        # Dashboard, Chat, Sessions, Settings
│       └── lib/          # WebSocket RPC client
├── hooks/                # Confirmation engine
├── mcp/                  # MCP tool server integration
├── memory/               # Persistent memory store + vector search
├── models/               # Model providers + router + media pipeline
├── prompt/               # System prompt templating (auto-injects current date/time)
├── sandbox/              # Docker sandboxing
├── session/              # SQLite persistence
├── skills/               # Skill packages
├── tools/                # Builtin tools (shell, file, web, browser, process, media, system.info)
└── automation/           # Cron scheduler, webhooks, heartbeat monitor, Gmail watcher

System Prompt

Flynn assembles its system prompt from layered template files (SOUL.md, AGENTS.md, IDENTITY.md, USER.md, TOOLS.md) searched in configurable directories. The first match per file wins.

A Runtime Context section is automatically appended to every system prompt with the current date and time, so the model always knows when "now" is without needing a tool call.

The system.info tool provides on-demand access to more detailed runtime information: current date/time, hostname, platform, architecture, OS release, uptime, Node.js version, memory usage, and working directory.

Development

# Dev mode with watch
pnpm dev          # Daemon
pnpm tui:dev      # TUI

# Type check
pnpm typecheck

# Lint
pnpm lint

# Test
pnpm test

Environment Variables

Variable	Description
FLYNN_CONFIG	Override config file path
FLYNN_DATA_DIR	Override data directory (default: ~/.local/share/flynn)
ANTHROPIC_API_KEY	Anthropic API key (fallback)
OPENAI_API_KEY	OpenAI API key (fallback)

License

MIT