will/flynn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8bf88049bf968bc94b36fb6070dc12495ea55e0c
flynn/README.md
T
William Valentin 33f6f142bc docs: add file.patch and Gmail Pub/Sub watcher documentation 
- README: features list, Gmail config section with field reference and
  template variables, hook examples, architecture tree, CLI description
- CHANGELOG: new Added entries for both features
- SOUL.md: file.patch in capabilities list
- config/default.yaml: file.patch in confirm hooks, commented-out
  automation section with cron, webhooks, gmail, heartbeat examples
2026-02-07 15:45:41 -08:00

604 lines
21 KiB
Markdown
Raw Blame History

# Flynn
Self-hosted personal AI assistant with Telegram and Terminal interfaces.
## Features
- **Multi-Frontend**: Telegram bot + Terminal UI (minimal & fullscreen modes) + Web UI dashboard
- **Multi-Model**: Anthropic Claude, OpenAI, GitHub Copilot, Gemini, Bedrock, Ollama, llama.cpp with intelligent routing
- **Multi-Channel**: Telegram, Discord, Slack, WhatsApp with unified adapter interface
- **Web Dashboard**: SPA control panel with health monitoring, chat, session browser, and settings editor
- **Model Switching**: Switch between cloud/local models on demand
- **Session Persistence**: SQLite-backed conversation history
- **Fallback Chains**: Automatic failover when primary model fails
- **Hook Engine**: Confirmation system for sensitive operations
- **Tool Framework**: Shell, file, file patch, web-fetch, web-search, browser control, image analysis, media send
- **Docker Sandboxing**: Per-session container isolation for tool execution
- **Multi-Agent Routing**: Config-driven agent selection per sender/channel with tool profiles
- **Media Pipeline**: Image analysis, outbound attachments, audio transcription across all channels
- **Session Transfer**: Move conversations between frontends
- **CLI**: Full command-line interface (`flynn start`, `send`, `doctor`, etc.)
- **Cron Scheduling**: Automated messages on cron schedules with output routing
- **Inbound Webhooks**: HTTP endpoints that trigger agent processing with HMAC auth and template rendering
- **Heartbeat Monitor**: Periodic health checks (gateway, model, channels, memory, disk) with failure notifications
- **Gmail Pub/Sub Watcher**: Monitor Gmail inbox via Google Cloud Pub/Sub push notifications with polling fallback
- **Vector Memory Search**: Hybrid keyword + semantic search with embeddings (OpenAI, Gemini, Ollama, llama.cpp)
- **Docker Deployment**: Multi-stage Dockerfile and docker-compose.yml for production containers
- **Health Diagnostics**: `flynn doctor` validates config, connectivity, and system state
- **MCP Integration**: External tool servers via Model Context Protocol
- **Skills System**: Extensible capability packages (bundled, managed, workspace tiers)
## Quick Start
```bash
# Install dependencies
pnpm install
# Copy and configure
cp config/default.yaml ~/.config/flynn/config.yaml
# Edit config with your API keys and Telegram bot token
# Build and run
pnpm build
flynn start
# Or run without building
pnpm start
```
## CLI Commands
Flynn provides a full CLI via the `flynn` binary (or `npx tsx src/cli/index.ts` during development):
| Command | Description |
|---------|-------------|
| `flynn start` | Start the Flynn daemon (Telegram, WebChat, cron, Gmail watcher) |
| `flynn tui` | Launch the interactive terminal UI |
| `flynn send <message>` | Send a one-shot message and print the response |
| `flynn sessions` | List active sessions |
| `flynn doctor` | Validate config and check system health |
| `flynn config` | Show resolved configuration (secrets redacted) |
### Examples
```bash
# Start daemon with custom config
flynn start --config ~/my-config.yaml
# One-shot query
flynn send "What's the weather in London?"
# Check system health
flynn doctor --config ~/.config/flynn/config.yaml
# Show current config (secrets masked)
flynn config
# List sessions
flynn sessions
```
## Configuration
Config location: `~/.config/flynn/config.yaml` (or set `FLYNN_CONFIG`)
```yaml
telegram:
bot_token: "your-telegram-bot-token"
allowed_chat_ids: [123456789] # Your Telegram user ID
models:
default:
provider: anthropic
model: claude-opus-4-5-20251101
api_key: sk-ant-api03-...
local:
provider: ollama
model: qwen2.5:14b
fallback_chain: [local]
hooks:
confirm: [shell.*, file.write, file.patch]
log: [web.*, file.read]
silent: [notify]
```
### Model Providers
| Provider | Config |
|----------|--------|
| Anthropic | `provider: anthropic`, `api_key` or `auth_token` |
| OpenAI | `provider: openai`, `api_key`, optional `endpoint` |
| GitHub Copilot | `provider: github`, auto-login via OAuth device flow |
| Gemini | `provider: gemini`, `api_key` |
| Bedrock | `provider: bedrock`, AWS credentials |
| Ollama | `provider: ollama`, `model`, optional `endpoint` |
| llama.cpp | `provider: llamacpp`, `endpoint` |
### Model Tiers
Configure multiple models for different purposes:
```yaml
models:
fast: { provider: anthropic, model: claude-sonnet-4-... }
default: { provider: anthropic, model: claude-opus-4-5-... }
complex: { provider: anthropic, model: claude-opus-4-5-... }
local: { provider: ollama, model: qwen2.5:14b }
```
## Telegram Commands
| Command | Description |
|---------|-------------|
| `/start` | Initialize bot |
| `/reset` | Clear conversation history |
| `/status` | Show current model and status |
| `/local` | Switch to local model |
| `/cloud` | Switch to cloud model |
| `/model` | Show model info and options |
## Web UI Dashboard
Flynn includes a built-in web control dashboard served by the WebSocket gateway. Access it at `http://localhost:18800` (or your configured gateway port).
### Pages
| Page | Description |
|------|-------------|
| **Dashboard** | System health cards, channel status, usage stats. Auto-refreshes every 10s |
| **Chat** | Session selector, streaming tool events, markdown rendering with syntax highlighting |
| **Sessions** | Browse all sessions, view message history, delete sessions |
| **Settings** | Edit hook patterns (confirm/log/silent), view tools, channels, and redacted config |
The dashboard is a vanilla JS SPA with no build step — hash-based routing, ES modules, and the existing WebSocket JSON-RPC protocol.
## Terminal UI
```bash
# Minimal mode (readline)
pnpm tui
# Fullscreen mode (React/Ink)
pnpm tui:fs
```
### TUI Commands
| Command | Description |
|---------|-------------|
| `/help` | Show help |
| `/reset` | Clear history |
| `/status` | Show session info |
| `/fullscreen` | Switch to fullscreen mode |
| `/transfer telegram` | Transfer session to Telegram |
| `/quit` | Exit |
## Running as Service
```bash
# Create systemd user service
mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/flynn.service << 'EOF'
[Unit]
Description=Flynn Personal AI Assistant
After=network.target ollama.service
[Service]
Type=simple
WorkingDirectory=/path/to/flynn
ExecStart=/usr/bin/pnpm start
Restart=on-failure
RestartSec=5
Environment=NODE_ENV=production
[Install]
WantedBy=default.target
EOF
# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now flynn
# View logs
journalctl --user -u flynn -f
```
## Hook Engine
Control sensitive operations with pattern matching:
```yaml
hooks:
confirm: # Requires user approval via Telegram
- shell.*
- file.write
- file.patch
log: # Logs but doesn't block
- web.*
- file.read
silent: # Executes without notification
- notify
```
## Cron Scheduling
Schedule automated messages on cron schedules. Each job fires an inbound message through the agent pipeline and routes the response to a configured output channel.
```yaml
automation:
cron:
- name: daily-summary
schedule: "0 9 * * *" # 9 AM daily
message: "Give me a summary of today's tasks"
output:
channel: telegram # Route response to Telegram
peer: "123456789" # Chat ID to send to
timezone: Europe/London # Optional timezone
enabled: true
- name: hourly-check
schedule: "0 * * * *" # Every hour
message: "Check system status"
output:
channel: telegram
peer: "123456789"
enabled: false # Disabled, won't fire
```
### Cron Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `name` | yes | Unique job identifier |
| `schedule` | yes | Cron expression (standard 5-field) |
| `message` | yes | Text sent to the agent when the job fires |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
| `timezone` | no | IANA timezone (defaults to system timezone) |
| `enabled` | no | Whether the job is active (default: `true`) |
## Inbound Webhooks
HTTP endpoints that trigger agent processing. Each webhook accepts POST requests, optionally verifies an HMAC signature, renders a message template, and routes the agent's response to an output channel.
```yaml
automation:
webhooks:
- name: github-push
secret: "whsec_..." # HMAC secret for signature verification
message: "GitHub push to {{json.repository.full_name}}: {{json.head_commit.message}}"
output:
channel: telegram
peer: "123456789"
- name: alertmanager
message: "Alert: {{json.alerts.0.annotations.summary}}"
output:
channel: discord
peer: "channel-id"
```
Webhooks are available at `POST /webhooks/:name` on the gateway HTTP server. They bypass gateway token auth and use their own per-webhook HMAC verification instead.
### Webhook Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `name` | yes | Unique webhook identifier (used in URL path) |
| `secret` | no | HMAC secret for `X-Webhook-Signature` header verification (SHA-256) |
| `message` | no | Template for the message sent to the agent (default: `{{body}}`) |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
| `enabled` | no | Whether the webhook is active (default: `true`) |
### Template Variables
| Variable | Description |
|----------|-------------|
| `{{body}}` | Raw request body as string |
| `{{json.field}}` | Extract a field from JSON body (dot notation for nested fields) |
## Heartbeat Monitor
Periodic health checks that validate system components and notify a configured channel on failure.
```yaml
automation:
heartbeat:
enabled: true
interval: "5m" # Check every 5 minutes
checks: [gateway, model, channels, memory, disk]
notify:
channel: telegram
peer: "123456789"
failure_threshold: 2 # Notify after 2 consecutive failures
disk_threshold_mb: 100 # Warn when <100MB free
```
### Heartbeat Checks
| Check | What it validates |
|-------|-------------------|
| `gateway` | HTTP server is responsive |
| `model` | Model router has an active tier configured |
| `channels` | At least one channel adapter is connected |
| `memory` | Memory directory is readable and writable |
| `disk` | Free disk space exceeds threshold |
The monitor sends a notification when failures reach the configured threshold and a recovery notification when all checks pass again.
### Heartbeat Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable the heartbeat monitor (default: `false`) |
| `interval` | no | Check interval: `60s`, `5m`, `1h` (default: `5m`) |
| `checks` | no | Which checks to run (default: all five) |
| `notify.channel` | no | Channel to send failure/recovery notifications |
| `notify.peer` | no | Peer/chat ID for notifications |
| `failure_threshold` | no | Consecutive failures before notifying (default: `2`) |
| `disk_threshold_mb` | no | Disk space warning threshold in MB (default: `100`) |
## Gmail Pub/Sub Watcher
Monitor a Gmail inbox via Google Cloud Pub/Sub push notifications. New emails trigger the agent pipeline and route responses to a configured output channel. Falls back to polling when push notifications are unavailable.
### Prerequisites
1. Create a Google Cloud project with the Gmail API and Pub/Sub API enabled
2. Create OAuth2 credentials (Desktop application type) and download the JSON file
3. Create a Pub/Sub topic (e.g. `projects/your-project/topics/gmail-push`)
4. Run `flynn gmail-auth` to complete the OAuth2 flow and store the refresh token
### Configuration
```yaml
automation:
gmail:
enabled: true
credentials_file: ~/.config/flynn/gmail-credentials.json
token_file: ~/.config/flynn/gmail-token.json # Default location
watch_labels: [INBOX] # Labels to watch
poll_interval: "60s" # Polling fallback interval
message: "New email from {{from}}: {{subject}}\n\n{{snippet}}"
output:
channel: telegram
peer: "123456789"
```
Push notifications arrive at `POST /gmail/push` on the gateway HTTP server (bypasses gateway auth).
### Gmail Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable the Gmail watcher (default: `false`) |
| `credentials_file` | yes | Path to Google OAuth2 credentials JSON |
| `token_file` | no | Path to stored OAuth2 refresh token (default: `~/.config/flynn/gmail-token.json`) |
| `watch_labels` | no | Gmail labels to watch (default: `[INBOX]`) |
| `poll_interval` | no | Polling fallback interval: `60s`, `5m` (default: `60s`) |
| `history_start` | no | ISO date string — only process emails received after this date |
| `message` | no | Template for the agent message (default: `New email from {{from}}: {{subject}}\n\n{{snippet}}`) |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
### Template Variables
| Variable | Description |
|----------|-------------|
| `{{from}}` | Sender address |
| `{{to}}` | Recipient address |
| `{{subject}}` | Email subject line |
| `{{snippet}}` | Gmail-provided message snippet |
| `{{date}}` | Email date |
| `{{id}}` | Gmail message ID |
| `{{labels}}` | Comma-separated label names |
## Vector Memory Search
The memory system supports hybrid search combining keyword matching with semantic vector similarity. When embeddings are enabled, `memory.search` uses both approaches and merges results with configurable weighting.
```yaml
memory:
enabled: true
auto_extract: true
max_context_tokens: 2000
embedding:
enabled: true
provider: openai # openai, gemini, ollama, llamacpp
model: text-embedding-3-small
api_key: "${OPENAI_API_KEY}"
chunk_size: 512 # Tokens per chunk
chunk_overlap: 50 # Overlap between chunks
top_k: 5 # Top results from vector search
hybrid_weight: 0.7 # 0.0 = keyword only, 1.0 = vector only
```
### Embedding Providers
| Provider | Config |
|----------|--------|
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
| Gemini | `provider: gemini`, `api_key`, `model` |
| Ollama | `provider: ollama`, `endpoint` (default: localhost:11434), `model` |
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |
Embeddings are indexed in the background — when memory is written, the namespace is marked dirty and re-indexed within 30 seconds. The vector index is stored in `vectors.db` alongside the session database.
When embeddings are disabled or the provider is unreachable, search falls back gracefully to keyword matching.
### Embedding Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable vector search (default: `false`) |
| `provider` | no | Embedding provider (default: `openai`) |
| `model` | no | Embedding model name (default: `text-embedding-3-small`) |
| `endpoint` | no | Provider endpoint (required for `ollama`/`llamacpp`) |
| `api_key` | no | API key (required for `openai`/`gemini`) |
| `dimensions` | no | Vector dimensions (auto-detected from model if not set) |
| `chunk_size` | no | Max tokens per chunk (default: `512`) |
| `chunk_overlap` | no | Token overlap between chunks (default: `50`) |
| `top_k` | no | Number of vector results to return (default: `5`) |
| `hybrid_weight` | no | Vector vs keyword weight, 0.0-1.0 (default: `0.7`) |
## Docker Deployment
Flynn includes a production-ready Dockerfile with multi-stage build.
```bash
# Build the image
docker build -t flynn .
# Run with config and data volumes
docker run -d \
--name flynn \
-p 18800:18800 \
-v ./config.yaml:/config/config.yaml:ro \
-v flynn-data:/data \
-e ANTHROPIC_API_KEY=sk-ant-... \
flynn
```
Or use the included `docker-compose.yml`:
```bash
# Copy your config
cp ~/.config/flynn/config.yaml ./config.yaml
# Start with compose
docker compose up -d
# View logs
docker compose logs -f
```
### Docker Environment Variables
| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Config file path (default: `/config/config.yaml`) |
| `FLYNN_DATA_DIR` | Data directory path (default: `/data`) |
| `ANTHROPIC_API_KEY` | Anthropic API key |
| `OPENAI_API_KEY` | OpenAI API key |
| `TELEGRAM_BOT_TOKEN` | Telegram bot token |
### Volumes
| Mount Point | Purpose |
|-------------|---------|
| `/config/config.yaml` | Configuration file (read-only) |
| `/data` | Persistent data (sessions DB, memory files, vector index) |
## Doctor Diagnostics
`flynn doctor` runs 10 health checks to validate your setup:
```
$ flynn doctor
Flynn Doctor
============
[PASS] Config file exists (/home/user/.config/flynn/config.yaml)
[PASS] Config parses (valid YAML)
[PASS] Config validates (schema valid)
[PASS] Env vars resolved
[PASS] Data directory writable (/home/user/.local/share/flynn)
[PASS] Session DB accessible (sessions.db)
[PASS] Model connectivity (anthropic: claude-sonnet)
[PASS] Telegram bot configured (1 allowed chat(s))
[SKIP] MCP servers configured (none configured)
[PASS] Skills loaded (3 skill(s))
Results: 8 passed, 0 failed, 0 warnings, 1 skipped
```
### Check Details
| Check | What it validates |
|-------|-------------------|
| Config file exists | Config YAML file is present at the expected path |
| Config parses | File is valid YAML syntax |
| Config validates | YAML content passes Zod schema validation |
| Env vars resolved | Any `${VAR}` references in config have values set |
| Data directory writable | Can write to `~/.local/share/flynn/` |
| Session DB accessible | SQLite database opens and queries succeed |
| Model connectivity | Default model provider and model name are configured |
| Telegram bot configured | Bot token is present and reasonable length |
| MCP servers configured | Lists configured MCP tool servers |
| Skills loaded | Discovers and loads skill packages |
Exit code is `1` if any check fails, `0` otherwise. Checks that depend on a valid config are skipped when config is invalid.
## Session Management
- Sessions persist in `~/.local/share/flynn/sessions.db`
- Session ID format: `{frontend}:{userId}` (e.g., `telegram:123456789`)
- History survives restarts
- Transfer sessions between frontends with `/transfer`
## Architecture
```
src/
├── agents/ # Multi-agent routing
├── auth/ # OAuth flows (GitHub Copilot)
├── backends/native/ # Agent implementation + orchestrator
├── channels/ # Channel adapters (Telegram, Discord, Slack, WhatsApp, WebChat)
├── cli/ # CLI commands (commander)
├── config/ # YAML config + Zod validation
├── context/ # Token estimation + compaction
├── daemon/ # Lifecycle management + routing
├── frontends/
│ ├── telegram/ # Telegram bot
│ └── tui/ # Terminal UI (minimal + fullscreen)
├── gateway/ # WebSocket gateway + web UI dashboard
│ ├── handlers/ # JSON-RPC method handlers
│ └── ui/ # SPA dashboard (vanilla JS)
│ ├── pages/ # Dashboard, Chat, Sessions, Settings
│ └── lib/ # WebSocket RPC client
├── hooks/ # Confirmation engine
├── mcp/ # MCP tool server integration
├── memory/ # Persistent memory store + vector search
├── models/ # Model providers + router + media pipeline
├── prompt/ # System prompt templating
├── sandbox/ # Docker sandboxing
├── session/ # SQLite persistence
├── skills/ # Skill packages
├── tools/ # Builtin tools (shell, file, web, browser, process, media)
└── automation/ # Cron scheduler, webhooks, heartbeat monitor, Gmail watcher
```
## Development
```bash
# Dev mode with watch
pnpm dev # Daemon
pnpm tui:dev # TUI
# Type check
pnpm typecheck
# Lint
pnpm lint
# Test
pnpm test
```
## Environment Variables
| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Override config file path |
| `FLYNN_DATA_DIR` | Override data directory (default: `~/.local/share/flynn`) |
| `ANTHROPIC_API_KEY` | Anthropic API key (fallback) |
| `OPENAI_API_KEY` | OpenAI API key (fallback) |
## License
MIT
Reference in New Issue View Git Blame Copy Permalink