docs: add documentation for Tier 2 features

- README: webhooks, heartbeat, vector memory, Docker sections with config examples
- README: updated features list, architecture tree, environment variables
- CHANGELOG: entries for all 4 Tier 2 features
This commit is contained in:
William Valentin
2026-02-07 14:52:02 -08:00
parent 93c0d64e8d
commit d0ebbff608
2 changed files with 203 additions and 3 deletions
+186 -3
View File
@@ -19,6 +19,10 @@ Self-hosted personal AI assistant with Telegram and Terminal interfaces.
- **Session Transfer**: Move conversations between frontends
- **CLI**: Full command-line interface (`flynn start`, `send`, `doctor`, etc.)
- **Cron Scheduling**: Automated messages on cron schedules with output routing
- **Inbound Webhooks**: HTTP endpoints that trigger agent processing with HMAC auth and template rendering
- **Heartbeat Monitor**: Periodic health checks (gateway, model, channels, memory, disk) with failure notifications
- **Vector Memory Search**: Hybrid keyword + semantic search with embeddings (OpenAI, Gemini, Ollama, llama.cpp)
- **Docker Deployment**: Multi-stage Dockerfile and docker-compose.yml for production containers
- **Health Diagnostics**: `flynn doctor` validates config, connectivity, and system state
- **MCP Integration**: External tool servers via Model Context Protocol
- **Skills System**: Extensible capability packages (bundled, managed, workspace tiers)
@@ -253,6 +257,184 @@ automation:
| `timezone` | no | IANA timezone (defaults to system timezone) |
| `enabled` | no | Whether the job is active (default: `true`) |
## Inbound Webhooks
HTTP endpoints that trigger agent processing. Each webhook accepts POST requests, optionally verifies an HMAC signature, renders a message template, and routes the agent's response to an output channel.
```yaml
automation:
webhooks:
- name: github-push
secret: "whsec_..." # HMAC secret for signature verification
message: "GitHub push to {{json.repository.full_name}}: {{json.head_commit.message}}"
output:
channel: telegram
peer: "123456789"
- name: alertmanager
message: "Alert: {{json.alerts.0.annotations.summary}}"
output:
channel: discord
peer: "channel-id"
```
Webhooks are available at `POST /webhooks/:name` on the gateway HTTP server. They bypass gateway token auth and use their own per-webhook HMAC verification instead.
### Webhook Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `name` | yes | Unique webhook identifier (used in URL path) |
| `secret` | no | HMAC secret for `X-Webhook-Signature` header verification (SHA-256) |
| `message` | no | Template for the message sent to the agent (default: `{{body}}`) |
| `output.channel` | yes | Channel name to route the response (e.g. `telegram`) |
| `output.peer` | yes | Peer/chat ID on the output channel |
| `enabled` | no | Whether the webhook is active (default: `true`) |
### Template Variables
| Variable | Description |
|----------|-------------|
| `{{body}}` | Raw request body as string |
| `{{json.field}}` | Extract a field from JSON body (dot notation for nested fields) |
## Heartbeat Monitor
Periodic health checks that validate system components and notify a configured channel on failure.
```yaml
automation:
heartbeat:
enabled: true
interval: "5m" # Check every 5 minutes
checks: [gateway, model, channels, memory, disk]
notify:
channel: telegram
peer: "123456789"
failure_threshold: 2 # Notify after 2 consecutive failures
disk_threshold_mb: 100 # Warn when <100MB free
```
### Heartbeat Checks
| Check | What it validates |
|-------|-------------------|
| `gateway` | HTTP server is responsive |
| `model` | Model router has an active tier configured |
| `channels` | At least one channel adapter is connected |
| `memory` | Memory directory is readable and writable |
| `disk` | Free disk space exceeds threshold |
The monitor sends a notification when failures reach the configured threshold and a recovery notification when all checks pass again.
### Heartbeat Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable the heartbeat monitor (default: `false`) |
| `interval` | no | Check interval: `60s`, `5m`, `1h` (default: `5m`) |
| `checks` | no | Which checks to run (default: all five) |
| `notify.channel` | no | Channel to send failure/recovery notifications |
| `notify.peer` | no | Peer/chat ID for notifications |
| `failure_threshold` | no | Consecutive failures before notifying (default: `2`) |
| `disk_threshold_mb` | no | Disk space warning threshold in MB (default: `100`) |
## Vector Memory Search
The memory system supports hybrid search combining keyword matching with semantic vector similarity. When embeddings are enabled, `memory.search` uses both approaches and merges results with configurable weighting.
```yaml
memory:
enabled: true
auto_extract: true
max_context_tokens: 2000
embedding:
enabled: true
provider: openai # openai, gemini, ollama, llamacpp
model: text-embedding-3-small
api_key: "${OPENAI_API_KEY}"
chunk_size: 512 # Tokens per chunk
chunk_overlap: 50 # Overlap between chunks
top_k: 5 # Top results from vector search
hybrid_weight: 0.7 # 0.0 = keyword only, 1.0 = vector only
```
### Embedding Providers
| Provider | Config |
|----------|--------|
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
| Gemini | `provider: gemini`, `api_key`, `model` |
| Ollama | `provider: ollama`, `endpoint` (default: localhost:11434), `model` |
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |
Embeddings are indexed in the background — when memory is written, the namespace is marked dirty and re-indexed within 30 seconds. The vector index is stored in `vectors.db` alongside the session database.
When embeddings are disabled or the provider is unreachable, search falls back gracefully to keyword matching.
### Embedding Config Fields
| Field | Required | Description |
|-------|----------|-------------|
| `enabled` | no | Enable vector search (default: `false`) |
| `provider` | no | Embedding provider (default: `openai`) |
| `model` | no | Embedding model name (default: `text-embedding-3-small`) |
| `endpoint` | no | Provider endpoint (required for `ollama`/`llamacpp`) |
| `api_key` | no | API key (required for `openai`/`gemini`) |
| `dimensions` | no | Vector dimensions (auto-detected from model if not set) |
| `chunk_size` | no | Max tokens per chunk (default: `512`) |
| `chunk_overlap` | no | Token overlap between chunks (default: `50`) |
| `top_k` | no | Number of vector results to return (default: `5`) |
| `hybrid_weight` | no | Vector vs keyword weight, 0.0-1.0 (default: `0.7`) |
## Docker Deployment
Flynn includes a production-ready Dockerfile with multi-stage build.
```bash
# Build the image
docker build -t flynn .
# Run with config and data volumes
docker run -d \
--name flynn \
-p 18800:18800 \
-v ./config.yaml:/config/config.yaml:ro \
-v flynn-data:/data \
-e ANTHROPIC_API_KEY=sk-ant-... \
flynn
```
Or use the included `docker-compose.yml`:
```bash
# Copy your config
cp ~/.config/flynn/config.yaml ./config.yaml
# Start with compose
docker compose up -d
# View logs
docker compose logs -f
```
### Docker Environment Variables
| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Config file path (default: `/config/config.yaml`) |
| `FLYNN_DATA_DIR` | Data directory path (default: `/data`) |
| `ANTHROPIC_API_KEY` | Anthropic API key |
| `OPENAI_API_KEY` | OpenAI API key |
| `TELEGRAM_BOT_TOKEN` | Telegram bot token |
### Volumes
| Mount Point | Purpose |
|-------------|---------|
| `/config/config.yaml` | Configuration file (read-only) |
| `/data` | Persistent data (sessions DB, memory files, vector index) |
## Doctor Diagnostics
`flynn doctor` runs 10 health checks to validate your setup:
@@ -323,14 +505,14 @@ src/
│ └── lib/ # WebSocket RPC client
├── hooks/ # Confirmation engine
├── mcp/ # MCP tool server integration
├── memory/ # Persistent memory store
├── memory/ # Persistent memory store + vector search
├── models/ # Model providers + router + media pipeline
├── prompt/ # System prompt templating
├── sandbox/ # Docker sandboxing
├── session/ # SQLite persistence
├── skills/ # Skill packages
├── tools/ # Builtin tools (shell, file, web, browser, process, media)
└── automation/ # Cron scheduler
└── automation/ # Cron scheduler, webhooks, heartbeat monitor
```
## Development
@@ -354,7 +536,8 @@ pnpm test
| Variable | Description |
|----------|-------------|
| `FLYNN_CONFIG` | Override config path |
| `FLYNN_CONFIG` | Override config file path |
| `FLYNN_DATA_DIR` | Override data directory (default: `~/.local/share/flynn`) |
| `ANTHROPIC_API_KEY` | Anthropic API key (fallback) |
| `OPENAI_API_KEY` | OpenAI API key (fallback) |