docs: Add comprehensive documentation for production deployment and contribution
This commit adds 6 new documentation files to fill critical gaps: - CONTRIBUTING.md: Developer onboarding guide with setup, workflow, code style, testing, and adding features - TROUBLESHOOTING.md: Common issues and solutions for errors, model issues, tool issues, channel issues, gateway issues, configuration issues, and memory/database issues - docs/api/PROTOCOL.md: Gateway JSON-RPC protocol documentation with connection, authentication, message format, methods, events, error codes, and example client implementation - docs/api/TOOLS.md: Tools API documentation covering tool interface, input schema format, result format, tool patterns, tool registration, tool policy, execution flow, and builtin tools reference - docs/deployment/PRODUCTION.md: Production deployment guide covering Docker deployment, systemd service, security, configuration, monitoring, backup & recovery, and performance tuning - docs/performance/TUNING.md: Performance optimization guide covering context management, model routing, tool execution, memory & embeddings, session management, database performance, gateway performance, and resource usage These files complement the existing excellent documentation (README.md, AGENTS.md, ARCHITECTURE.md, STRUCTURE.md, CONVENTIONS.md) to provide complete coverage for users, developers, and operators.
This commit is contained in:
William Valentin
File diff suppressed because it is too large
Load Diff
+1193
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,913 @@
|
||||
# Production Deployment Guide
|
||||
|
||||
This guide covers deploying Flynn in a production environment.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Docker Deployment](#docker-deployment)
|
||||
- [Systemd Service](#systemd-service)
|
||||
- [Security](#security)
|
||||
- [Configuration](#configuration)
|
||||
- [Monitoring](#monitoring)
|
||||
- [Backup & Recovery](#backup--recovery)
|
||||
- [Performance Tuning](#performance-tuning)
|
||||
- [Scaling Considerations](#scaling-considerations)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### System Requirements
|
||||
|
||||
- **OS**: Linux (Ubuntu 22.04+ recommended) or macOS
|
||||
- **Node.js**: >= 22.0.0
|
||||
- **Memory**: Minimum 2GB, 4GB+ recommended
|
||||
- **Disk**: 10GB+ for sessions, memory, and vectors
|
||||
- **Docker**: Required for sandbox features (optional)
|
||||
|
||||
### Network Requirements
|
||||
|
||||
- Public IP or VPN (Tailscale recommended) for remote access
|
||||
- Open ports: 18800 (gateway), optional 443 (Tailscale Serve)
|
||||
- Outbound HTTPS access for model providers and web tools
|
||||
|
||||
### External Services (Optional)
|
||||
|
||||
- **Model Providers**: Anthropic, OpenAI, GitHub Models, etc. (API keys required)
|
||||
- **Email**: SMTP server for email notifications
|
||||
- **Object Storage**: MinIO or S3 for backups (optional)
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
### Quick Start
|
||||
|
||||
Using the provided `docker-compose.yml`:
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone <repo-url>
|
||||
cd flynn
|
||||
|
||||
# Create config
|
||||
cp config/default.yaml config/production.yaml
|
||||
# Edit config/production.yaml with your settings
|
||||
|
||||
# Start services
|
||||
docker-compose up -d
|
||||
|
||||
# View logs
|
||||
docker-compose logs -f
|
||||
```
|
||||
|
||||
### Dockerfile
|
||||
|
||||
The multi-stage Dockerfile:
|
||||
|
||||
```dockerfile
|
||||
# Stage 1: Build
|
||||
FROM node:22-alpine AS builder
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm ci --only=production
|
||||
COPY . .
|
||||
RUN npm run build
|
||||
|
||||
# Stage 2: Runtime
|
||||
FROM node:22-alpine
|
||||
WORKDIR /app
|
||||
COPY --from=builder /app/dist ./dist
|
||||
COPY --from=builder /app/node_modules ./node_modules
|
||||
COPY config ./config
|
||||
COPY src/gateway/ui ./dist/gateway/ui
|
||||
|
||||
# Create data directory
|
||||
RUN mkdir -p /data
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD node -e "require('http').get('http://localhost:18800/health', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"
|
||||
|
||||
# Expose gateway port
|
||||
EXPOSE 18800
|
||||
|
||||
# Run
|
||||
CMD ["node", "dist/cli/index.js", "start"]
|
||||
```
|
||||
|
||||
### Docker Compose Configuration
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
flynn:
|
||||
build: .
|
||||
container_name: flynn
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "18800:18800"
|
||||
volumes:
|
||||
- ./config/production.yaml:/flynn/config.yaml:ro
|
||||
- flynn_data:/data
|
||||
- /var/run/docker.sock:/var/run/docker.sock # For sandbox
|
||||
environment:
|
||||
- NODE_ENV=production
|
||||
- FLYNN_CONFIG=/flynn/config.yaml
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "--spider", "-q", "http://localhost:18800/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 5s
|
||||
|
||||
whisper:
|
||||
image: openai/whisper-server:latest
|
||||
container_name: whisper-server
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- whisper_cache:/cache
|
||||
environment:
|
||||
- WHISPER_MODEL=base
|
||||
- WHISPER_HTTP_PORT=8080
|
||||
|
||||
volumes:
|
||||
flynn_data:
|
||||
whisper_cache:
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Node environment
|
||||
export NODE_ENV=production
|
||||
|
||||
# Config path
|
||||
export FLYNN_CONFIG=/path/to/config.yaml
|
||||
|
||||
# Data directory (default: ~/.local/share/flynn)
|
||||
export FLYNN_DATA_DIR=/var/lib/flynn
|
||||
|
||||
# Optional: Override model provider credentials
|
||||
export ANTHROPIC_API_KEY=sk-...
|
||||
export OPENAI_API_KEY=sk-...
|
||||
```
|
||||
|
||||
## Systemd Service
|
||||
|
||||
### Service File
|
||||
|
||||
Create `/etc/systemd/system/flynn.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Flynn AI Assistant Daemon
|
||||
After=network.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=flynn
|
||||
Group=flynn
|
||||
WorkingDirectory=/opt/flynn
|
||||
Environment="NODE_ENV=production"
|
||||
Environment="FLYNN_CONFIG=/etc/flynn/config.yaml"
|
||||
Environment="FLYNN_DATA_DIR=/var/lib/flynn"
|
||||
ExecStart=/usr/local/bin/node /opt/flynn/dist/cli/index.js start
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
SyslogIdentifier=flynn
|
||||
|
||||
# Security hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/lib/flynn /var/log/flynn /var/run
|
||||
|
||||
# Resource limits
|
||||
MemoryLimit=2G
|
||||
MemorySwap=0
|
||||
CPUQuota=200%
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
### Create Flynn User
|
||||
|
||||
```bash
|
||||
# Create user and group
|
||||
sudo useradd --system --home /var/lib/flynn --shell /usr/sbin/nologin flynn
|
||||
sudo groupadd flynn
|
||||
|
||||
# Create directories
|
||||
sudo mkdir -p /opt/flynn /etc/flynn /var/lib/flynn /var/log/flynn
|
||||
sudo chown -R flynn:flynn /opt/flynn /var/lib/flynn /var/log/flynn
|
||||
|
||||
# Copy binaries and config
|
||||
sudo cp -r dist/* /opt/flynn/
|
||||
sudo cp config/production.yaml /etc/flynn/config.yaml
|
||||
sudo chown -R root:root /opt/flynn /etc/flynn
|
||||
sudo chmod 644 /etc/flynn/config.yaml
|
||||
```
|
||||
|
||||
### Enable and Start Service
|
||||
|
||||
```bash
|
||||
# Reload systemd
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
# Enable service (start on boot)
|
||||
sudo systemctl enable flynn
|
||||
|
||||
# Start service
|
||||
sudo systemctl start flynn
|
||||
|
||||
# Check status
|
||||
sudo systemctl status flynn
|
||||
|
||||
# View logs
|
||||
sudo journalctl -u flynn -f
|
||||
|
||||
# Restart service
|
||||
sudo systemctl restart flynn
|
||||
```
|
||||
|
||||
### Service Management
|
||||
|
||||
```bash
|
||||
# Stop service
|
||||
sudo systemctl stop flynn
|
||||
|
||||
# Reload config (requires restart)
|
||||
sudo systemctl restart flynn
|
||||
|
||||
# Check if running
|
||||
sudo systemctl is-active flynn
|
||||
|
||||
# View recent logs
|
||||
sudo journalctl -u flynn -n 100 --no-pager
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
### Secrets Management
|
||||
|
||||
Never commit secrets to version control. Use one of these approaches:
|
||||
|
||||
#### Environment Variables
|
||||
|
||||
```yaml
|
||||
# config/production.yaml
|
||||
models:
|
||||
default:
|
||||
anthropic:
|
||||
apiKey: '${ANTHROPIC_API_KEY}'
|
||||
```
|
||||
|
||||
Set in `/etc/flynn/.env` or systemd service file:
|
||||
```ini
|
||||
Environment="ANTHROPIC_API_KEY=sk-..."
|
||||
```
|
||||
|
||||
#### HashiCorp Vault (Advanced)
|
||||
|
||||
Use a secrets manager and inject at runtime:
|
||||
|
||||
```bash
|
||||
vault kv get -field=api_key secret/anthropic > /tmp/anthropic_key.txt
|
||||
export ANTHROPIC_API_KEY=$(cat /tmp/anthropic_key.txt)
|
||||
rm /tmp/anthropic_key.txt
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
#### Gateway Auth
|
||||
|
||||
```yaml
|
||||
# config/production.yaml
|
||||
gateway:
|
||||
enabled: true
|
||||
auth:
|
||||
token: 'your-random-token-here' # Generate with: openssl rand -hex 32
|
||||
trustTailscaleIdentity: true
|
||||
applyToHttp: true
|
||||
```
|
||||
|
||||
Generate a secure token:
|
||||
```bash
|
||||
openssl rand -hex 32
|
||||
```
|
||||
|
||||
#### Channel Whitelists
|
||||
|
||||
Restrict who can interact with Flynn:
|
||||
|
||||
```yaml
|
||||
channels:
|
||||
telegram:
|
||||
allowedChatIds: ['123456789'] # Your Telegram chat ID
|
||||
discord:
|
||||
allowedGuildIds: ['987654321098765432']
|
||||
allowedChannelIds: ['123456789012345678']
|
||||
slack:
|
||||
allowedChannelIds: ['C12345678']
|
||||
signingSecret: '${SLACK_SIGNING_SECRET}'
|
||||
```
|
||||
|
||||
### Network Security
|
||||
|
||||
#### Firewall
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian (ufw)
|
||||
sudo ufw allow 22/tcp # SSH
|
||||
sudo ufw allow 18800/tcp # Flynn gateway
|
||||
sudo ufw enable
|
||||
|
||||
# CentOS/RHEL (firewalld)
|
||||
sudo firewall-cmd --permanent --add-port=18800/tcp
|
||||
sudo firewall-cmd --reload
|
||||
```
|
||||
|
||||
#### Reverse Proxy (Nginx)
|
||||
|
||||
Place Flynn behind Nginx for TLS:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name flynn.example.com;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/flynn.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/flynn.example.com/privkey.pem;
|
||||
|
||||
# WebSocket upgrade
|
||||
location / {
|
||||
proxy_pass http://localhost:18800;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Timeouts
|
||||
proxy_connect_timeout 60s;
|
||||
proxy_send_timeout 60s;
|
||||
proxy_read_timeout 60s;
|
||||
}
|
||||
|
||||
# Health check endpoint (no auth required)
|
||||
location /health {
|
||||
proxy_pass http://localhost:18800/health;
|
||||
access_log off;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Obtain TLS certificate with Let's Encrypt:
|
||||
```bash
|
||||
sudo certbot --nginx -d flynn.example.com
|
||||
```
|
||||
|
||||
### File Permissions
|
||||
|
||||
```bash
|
||||
# Data directory
|
||||
sudo chmod 750 /var/lib/flynn
|
||||
sudo chown flynn:flynn /var/lib/flynn
|
||||
|
||||
# Config file
|
||||
sudo chmod 640 /etc/flynn/config.yaml
|
||||
sudo chown root:flynn /etc/flynn/config.yaml
|
||||
|
||||
# Logs
|
||||
sudo chmod 750 /var/log/flynn
|
||||
sudo chown flynn:flynn /var/log/flynn
|
||||
```
|
||||
|
||||
### Sandbox Security
|
||||
|
||||
Docker sandbox adds isolation but requires careful configuration:
|
||||
|
||||
```yaml
|
||||
# config/production.yaml
|
||||
sandbox:
|
||||
enabled: true
|
||||
image: 'node:22-alpine'
|
||||
dockerSocket: '/var/run/docker.sock'
|
||||
resourceLimits:
|
||||
memory: '512m'
|
||||
cpus: '0.5'
|
||||
timeoutSec: 60
|
||||
networkMode: 'none' # No network access
|
||||
```
|
||||
|
||||
Ensure Docker is secured:
|
||||
```bash
|
||||
# Run Docker as Flynn user
|
||||
sudo usermod -aG docker flynn
|
||||
|
||||
# Configure Docker daemon security
|
||||
sudo vim /etc/docker/daemon.json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"log-driver": "json-file",
|
||||
"log-opts": {
|
||||
"max-size": "10m",
|
||||
"max-file": "3"
|
||||
},
|
||||
"live-restore": true,
|
||||
"userland-proxy": false
|
||||
}
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Production Config Template
|
||||
|
||||
```yaml
|
||||
# config/production.yaml
|
||||
# Base config for production deployment
|
||||
|
||||
# ── Gateway ───────────────────────────────────────────────────────────────
|
||||
gateway:
|
||||
enabled: true
|
||||
port: 18800
|
||||
auth:
|
||||
token: '${GATEWAY_TOKEN}'
|
||||
trustTailscaleIdentity: true
|
||||
applyToHttp: true
|
||||
lock:
|
||||
enabled: true
|
||||
tailscaleServe:
|
||||
enabled: false # Set to true to expose via Tailscale
|
||||
hostname: 'flynn'
|
||||
port: 443
|
||||
|
||||
# ── Models ─────────────────────────────────────────────────────────────────
|
||||
models:
|
||||
default:
|
||||
anthropic:
|
||||
apiKey: '${ANTHROPIC_API_KEY}'
|
||||
model: 'claude-sonnet-4-20250514'
|
||||
maxTokens: 4096
|
||||
|
||||
router:
|
||||
tiers:
|
||||
default: 'anthropic:claude-sonnet-4-20250514'
|
||||
fast: 'anthropic:claude-haiku-4-20250514'
|
||||
complex: 'anthropic:claude-opus-4-20250514'
|
||||
local: 'ollama:llama3'
|
||||
|
||||
fallbackChain:
|
||||
- 'github:claude-sonnet-4-5'
|
||||
- 'local:ollama:llama3'
|
||||
|
||||
retry:
|
||||
maxAttempts: 3
|
||||
initialDelayMs: 1000
|
||||
multiplier: 2
|
||||
maxDelayMs: 30000
|
||||
|
||||
# ── Channels ───────────────────────────────────────────────────────────────
|
||||
channels:
|
||||
telegram:
|
||||
enabled: true
|
||||
token: '${TELEGRAM_BOT_TOKEN}'
|
||||
allowedChatIds: ['123456789']
|
||||
|
||||
discord:
|
||||
enabled: false
|
||||
|
||||
slack:
|
||||
enabled: false
|
||||
|
||||
whatsapp:
|
||||
enabled: false
|
||||
|
||||
# ── Sessions ───────────────────────────────────────────────────────────────
|
||||
sessions:
|
||||
ttl: '7d'
|
||||
maxSessions: 100
|
||||
|
||||
# ── Memory ────────────────────────────────────────────────────────────────
|
||||
memory:
|
||||
enabled: true
|
||||
embeddings:
|
||||
provider: 'openai'
|
||||
openai:
|
||||
apiKey: '${OPENAI_API_KEY}'
|
||||
model: 'text-embedding-3-small'
|
||||
|
||||
# ── Tools ─────────────────────────────────────────────────────────────────
|
||||
tools:
|
||||
policy: 'coding' # Restrict tool access
|
||||
|
||||
executor:
|
||||
defaultTimeoutMs: 30000
|
||||
maxOutputBytes: 51200
|
||||
|
||||
sandbox:
|
||||
enabled: false # Enable if using Docker
|
||||
|
||||
# ── Agents ────────────────────────────────────────────────────────────────
|
||||
agents:
|
||||
default:
|
||||
modelTier: 'default'
|
||||
toolPolicy: 'coding'
|
||||
compaction:
|
||||
thresholdPct: 80
|
||||
keepTurns: 4
|
||||
summaryMaxTokens: 1024
|
||||
|
||||
# ── Automation ────────────────────────────────────────────────────────────
|
||||
automation:
|
||||
cron:
|
||||
enabled: false
|
||||
|
||||
webhooks:
|
||||
enabled: false
|
||||
|
||||
heartbeat:
|
||||
enabled: true
|
||||
interval: '5m'
|
||||
checks:
|
||||
- 'gateway'
|
||||
- 'model'
|
||||
- 'channels'
|
||||
- 'memory'
|
||||
- 'disk'
|
||||
notifications:
|
||||
- type: 'telegram'
|
||||
chatId: '123456789'
|
||||
|
||||
# ── Logging ───────────────────────────────────────────────────────────────
|
||||
logging:
|
||||
level: 'info' # debug, info, warn, error
|
||||
```
|
||||
|
||||
### Config Validation
|
||||
|
||||
Validate config before starting:
|
||||
|
||||
```bash
|
||||
flynn doctor --config /etc/flynn/config.yaml
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Health Checks
|
||||
|
||||
Flynn provides a health check endpoint:
|
||||
|
||||
```bash
|
||||
# HTTP health check
|
||||
curl http://localhost:18800/health
|
||||
|
||||
# Response
|
||||
{
|
||||
"status": "ok",
|
||||
"version": "0.1.0",
|
||||
"uptime": 12345
|
||||
}
|
||||
```
|
||||
|
||||
### Logs
|
||||
|
||||
#### Journalctl (systemd)
|
||||
|
||||
```bash
|
||||
# Follow logs
|
||||
sudo journalctl -u flynn -f
|
||||
|
||||
# View last 100 lines
|
||||
sudo journalctl -u flynn -n 100 --no-pager
|
||||
|
||||
# View logs since yesterday
|
||||
sudo journalctl -u flynn --since yesterday
|
||||
|
||||
# Search for errors
|
||||
sudo journalctl -u flynn | grep -i error
|
||||
```
|
||||
|
||||
#### Log Rotation
|
||||
|
||||
Configure logrotate for systemd journal:
|
||||
|
||||
```bash
|
||||
sudo vim /etc/systemd/journald.conf
|
||||
```
|
||||
|
||||
```
|
||||
[Journal]
|
||||
SystemMaxUse=100M
|
||||
MaxRetentionSec=7day
|
||||
```
|
||||
|
||||
Restart systemd:
|
||||
```bash
|
||||
sudo systemctl restart systemd-journald
|
||||
```
|
||||
|
||||
### Heartbeat Monitor
|
||||
|
||||
Enable built-in heartbeat monitoring:
|
||||
|
||||
```yaml
|
||||
automation:
|
||||
heartbeat:
|
||||
enabled: true
|
||||
interval: '5m'
|
||||
checks:
|
||||
- 'gateway'
|
||||
- 'model'
|
||||
- 'channels'
|
||||
- 'memory'
|
||||
- 'disk'
|
||||
notifications:
|
||||
- type: 'telegram'
|
||||
chatId: '123456789'
|
||||
- type: 'webhook'
|
||||
url: 'https://hooks.slack.com/services/...'
|
||||
```
|
||||
|
||||
### External Monitoring
|
||||
|
||||
#### Prometheus (Optional)
|
||||
|
||||
Use Node.js prom-client for metrics (not currently implemented):
|
||||
|
||||
```yaml
|
||||
# Future feature
|
||||
monitoring:
|
||||
prometheus:
|
||||
enabled: true
|
||||
port: 9090
|
||||
```
|
||||
|
||||
#### Uptime Monitoring
|
||||
|
||||
Use external services:
|
||||
- UptimeRobot
|
||||
- Pingdom
|
||||
- Better Uptime
|
||||
|
||||
Monitor:
|
||||
- Gateway HTTP health endpoint
|
||||
- WebSocket connection
|
||||
- Response time
|
||||
|
||||
## Backup & Recovery
|
||||
|
||||
### What to Backup
|
||||
|
||||
1. **Configuration**: `/etc/flynn/config.yaml`
|
||||
2. **Sessions**: SQLite database at `~/.local/share/flynn/sessions.db`
|
||||
3. **Memory Files**: `~/.local/share/flynn/memory/`
|
||||
4. **Vectors**: SQLite database at `~/.local/share/flynn/vectors.db`
|
||||
5. **Pairing Codes**: SQLite table within sessions.db
|
||||
|
||||
### Backup Script
|
||||
|
||||
Create `/usr/local/bin/flynn-backup.sh`:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
BACKUP_DIR="/var/backups/flynn"
|
||||
DATA_DIR="/var/lib/flynn"
|
||||
CONFIG_DIR="/etc/flynn"
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
BACKUP_FILE="$BACKUP_DIR/flynn_$DATE.tar.gz"
|
||||
|
||||
# Create backup directory
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
# Stop Flynn
|
||||
sudo systemctl stop flynn
|
||||
|
||||
# Create backup
|
||||
tar -czf "$BACKUP_FILE" \
|
||||
"$CONFIG_DIR/config.yaml" \
|
||||
"$DATA_DIR/sessions.db" \
|
||||
"$DATA_DIR/vectors.db" \
|
||||
"$DATA_DIR/memory/"
|
||||
|
||||
# Compress old backups (keep last 7 daily, 4 weekly, 12 monthly)
|
||||
find "$BACKUP_DIR" -name "flynn_*.tar.gz" -mtime +90 -delete
|
||||
|
||||
# Restart Flynn
|
||||
sudo systemctl start flynn
|
||||
|
||||
echo "Backup created: $BACKUP_FILE"
|
||||
```
|
||||
|
||||
Make executable:
|
||||
```bash
|
||||
sudo chmod +x /usr/local/bin/flynn-backup.sh
|
||||
```
|
||||
|
||||
### Cron Job
|
||||
|
||||
Add to root crontab:
|
||||
|
||||
```bash
|
||||
sudo crontab -e
|
||||
```
|
||||
|
||||
```
|
||||
# Daily backup at 2 AM
|
||||
0 2 * * * /usr/local/bin/flynn-backup.sh >> /var/log/flynn-backup.log 2>&1
|
||||
```
|
||||
|
||||
### Restore
|
||||
|
||||
```bash
|
||||
# Stop Flynn
|
||||
sudo systemctl stop flynn
|
||||
|
||||
# Extract backup
|
||||
sudo tar -xzf /var/backups/flynn/flynn_20250213_020000.tar.gz -C /
|
||||
|
||||
# Start Flynn
|
||||
sudo systemctl start flynn
|
||||
```
|
||||
|
||||
### Database Maintenance
|
||||
|
||||
Run SQLite vacuum periodically:
|
||||
|
||||
```bash
|
||||
sqlite3 /var/lib/flynn/sessions.db "VACUUM;"
|
||||
sqlite3 /var/lib/flynn/vectors.db "VACUUM;"
|
||||
```
|
||||
|
||||
Add to crontab (monthly):
|
||||
```
|
||||
0 0 1 * * sqlite3 /var/lib/flynn/sessions.db "VACUUM;" >> /var/log/flynn-maintenance.log 2>&1
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### Node.js Tuning
|
||||
|
||||
Set Node.js options for production:
|
||||
|
||||
```bash
|
||||
# In systemd service
|
||||
Environment="NODE_OPTIONS=--max-old-space-size=2048"
|
||||
|
||||
# Or via environment variable
|
||||
export NODE_OPTIONS="--max-old-space-size=2048"
|
||||
```
|
||||
|
||||
### Context Management
|
||||
|
||||
Optimize compaction settings:
|
||||
|
||||
```yaml
|
||||
agents:
|
||||
default:
|
||||
compaction:
|
||||
thresholdPct: 75 # Trigger earlier
|
||||
keepTurns: 6 # Keep more context
|
||||
summaryMaxTokens: 2048 # Better summaries
|
||||
```
|
||||
|
||||
### SQLite Performance
|
||||
|
||||
Enable WAL mode:
|
||||
|
||||
```bash
|
||||
sqlite3 /var/lib/flynn/sessions.db "PRAGMA journal_mode=WAL;"
|
||||
sqlite3 /var/lib/flynn/sessions.db "PRAGMA synchronous=NORMAL;"
|
||||
sqlite3 /var/lib/flynn/sessions.db "PRAGMA cache_size=-64000;" # 64MB
|
||||
```
|
||||
|
||||
### Model Routing
|
||||
|
||||
Configure tiers for optimal cost/latency:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
router:
|
||||
tiers:
|
||||
fast: 'anthropic:claude-haiku-4-20250514' # Quick tasks
|
||||
default: 'anthropic:claude-sonnet-4-20250514' # General use
|
||||
complex: 'anthropic:claude-opus-4-20250514' # Complex reasoning
|
||||
local: 'ollama:llama3' # Fallback
|
||||
```
|
||||
|
||||
### Caching (Future)
|
||||
|
||||
Consider adding caching for:
|
||||
- Repeated tool calls
|
||||
- Memory search results
|
||||
- Model responses for common queries
|
||||
|
||||
## Scaling Considerations
|
||||
|
||||
### Single-Operator Scope
|
||||
|
||||
Flynn is designed for a single operator with multiple concurrent users. Limitations:
|
||||
|
||||
- **Max Concurrent Sessions**: ~100 (depends on model rate limits)
|
||||
- **Throughput**: ~10-20 messages/second (varies by model)
|
||||
- **Memory Usage**: 2-4GB for moderate usage
|
||||
|
||||
### When to Scale Up
|
||||
|
||||
Consider scaling if:
|
||||
- Consistent CPU usage > 80%
|
||||
- Memory usage > 4GB
|
||||
- Frequent rate limiting from model providers
|
||||
- Slow response times > 30 seconds
|
||||
|
||||
### Scaling Strategies
|
||||
|
||||
1. **Horizontal Scaling**: Deploy multiple Flynn instances behind a load balancer (not currently supported - sessions are stateful)
|
||||
|
||||
2. **Vertical Scaling**: Increase server resources (CPU, memory)
|
||||
|
||||
3. **Multi-Instance Architecture** (future):
|
||||
- Shared session storage (PostgreSQL/Redis)
|
||||
- Message queue for request distribution
|
||||
- Session affinity for stateful connections
|
||||
|
||||
### Cost Optimization
|
||||
|
||||
- Use local models for non-critical tasks
|
||||
- Cache embeddings
|
||||
- Optimize compaction to reduce token usage
|
||||
- Use efficient models for delegated tasks
|
||||
|
||||
## Troubleshooting Production Issues
|
||||
|
||||
### Service Won't Start
|
||||
|
||||
```bash
|
||||
# Check status
|
||||
sudo systemctl status flynn
|
||||
|
||||
# View logs
|
||||
sudo journalctl -u flynn -n 50 --no-pager
|
||||
|
||||
# Validate config
|
||||
flynn doctor --config /etc/flynn/config.yaml
|
||||
```
|
||||
|
||||
### High Memory Usage
|
||||
|
||||
```bash
|
||||
# Check memory
|
||||
free -h
|
||||
|
||||
# Check process memory
|
||||
ps aux | grep flynn
|
||||
|
||||
# Restart service
|
||||
sudo systemctl restart flynn
|
||||
```
|
||||
|
||||
### Gateway Connection Issues
|
||||
|
||||
```bash
|
||||
# Check if port is listening
|
||||
sudo ss -tlnp | grep 18800
|
||||
|
||||
# Check firewall
|
||||
sudo ufw status
|
||||
|
||||
# Test connectivity
|
||||
curl http://localhost:18800/health
|
||||
```
|
||||
|
||||
### Slow Response Times
|
||||
|
||||
```bash
|
||||
# Check CPU usage
|
||||
top
|
||||
|
||||
# Check model provider status
|
||||
# Verify API keys are valid
|
||||
# Check network latency
|
||||
|
||||
# Enable debug logging
|
||||
DEBUG='*' sudo systemctl restart flynn
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
For additional help, see:
|
||||
- [TROUBLESHOOTING.md](../../TROUBLESHOOTING.md)
|
||||
- [README.md](../../README.md)
|
||||
- GitHub Issues
|
||||
@@ -0,0 +1,846 @@
|
||||
# Performance Tuning Guide
|
||||
|
||||
This guide covers performance optimization techniques for Flynn in production environments.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Context Management](#context-management)
|
||||
- [Model Routing](#model-routing)
|
||||
- [Tool Execution](#tool-execution)
|
||||
- [Memory & Embeddings](#memory--embeddings)
|
||||
- [Session Management](#session-management)
|
||||
- [Database Performance](#database-performance)
|
||||
- [Gateway Performance](#gateway-performance)
|
||||
- [Resource Usage](#resource-usage)
|
||||
- [Monitoring & Profiling](#monitoring--profiling)
|
||||
|
||||
## Overview
|
||||
|
||||
Flynn's performance depends on several factors:
|
||||
|
||||
1. **Context window efficiency**: How efficiently tokens are used
|
||||
2. **Model selection**: Choosing the right model for each task
|
||||
3. **Tool execution**: Fast, reliable tool responses
|
||||
4. **I/O operations**: Database and file system access
|
||||
5. **Concurrency**: Handling multiple simultaneous requests
|
||||
|
||||
### Performance Goals
|
||||
|
||||
- **Response time**: < 5 seconds for simple queries
|
||||
- **Context efficiency**: > 80% token utilization
|
||||
- **Throughput**: 10-20 concurrent conversations
|
||||
- **Resource usage**: < 2GB memory, < 50% CPU
|
||||
|
||||
## Context Management
|
||||
|
||||
### Compaction Settings
|
||||
|
||||
Context compaction prevents conversations from exceeding model context windows.
|
||||
|
||||
```yaml
|
||||
agents:
|
||||
default:
|
||||
compaction:
|
||||
# Trigger compaction at 75% of context window
|
||||
thresholdPct: 75
|
||||
|
||||
# Keep last 6 turns (user + assistant pairs)
|
||||
keepTurns: 6
|
||||
|
||||
# Allow 2048 tokens for summary
|
||||
summaryMaxTokens: 2048
|
||||
|
||||
# Preserve high-importance messages
|
||||
importanceThreshold: 0.8
|
||||
```
|
||||
|
||||
### Tuning Guidelines
|
||||
|
||||
**For fast interactions:**
|
||||
```yaml
|
||||
thresholdPct: 60 # Compact early
|
||||
keepTurns: 2 # Minimal history
|
||||
summaryMaxTokens: 512 # Short summaries
|
||||
```
|
||||
|
||||
**For complex reasoning:**
|
||||
```yaml
|
||||
thresholdPct: 85 # Maximize context
|
||||
keepTurns: 10 # More history
|
||||
summaryMaxTokens: 4096 # Detailed summaries
|
||||
```
|
||||
|
||||
### Context Depth Levels
|
||||
|
||||
Control how much context is injected into the system prompt:
|
||||
|
||||
```yaml
|
||||
prompt:
|
||||
contextDepth: 'normal' # minimal | normal | detailed | debug
|
||||
```
|
||||
|
||||
- `minimal`: Only basic system prompt
|
||||
- `normal`: System prompt + basic memory
|
||||
- `detailed`: Full memory + tool descriptions
|
||||
- `debug`: Verbose context (development only)
|
||||
|
||||
### Token Counting
|
||||
|
||||
Flynn uses rule-based token estimation (fast but approximate).
|
||||
|
||||
**Enable tokenizer for accuracy (slower):**
|
||||
|
||||
```typescript
|
||||
// Currently not implemented
|
||||
// Future: Use tiktoken or similar for exact token counts
|
||||
```
|
||||
|
||||
## Model Routing
|
||||
|
||||
### Tier Configuration
|
||||
|
||||
Optimize model tiers for cost and latency:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
router:
|
||||
tiers:
|
||||
# Fast, cheap: Quick tasks, delegated calls
|
||||
fast: 'anthropic:claude-haiku-4-20250514'
|
||||
|
||||
# Default: General conversation
|
||||
default: 'anthropic:claude-sonnet-4-20250514'
|
||||
|
||||
# Complex: Deep reasoning, analysis
|
||||
complex: 'anthropic:claude-opus-4-20250514'
|
||||
|
||||
# Fallback: Local models when cloud fails
|
||||
local: 'ollama:llama3'
|
||||
```
|
||||
|
||||
### Delegation Tasks
|
||||
|
||||
Map delegation tasks to appropriate tiers:
|
||||
|
||||
```yaml
|
||||
agents:
|
||||
default:
|
||||
delegation:
|
||||
tiers:
|
||||
compaction: 'fast' # Summarize history
|
||||
memoryExtraction: 'fast' # Extract facts
|
||||
classification: 'default' # Classify intent
|
||||
toolSummarization: 'default' # Summarize tool results
|
||||
complexReasoning: 'complex' # Deep analysis
|
||||
```
|
||||
|
||||
### Fallback Chains
|
||||
|
||||
Configure fallback chains for resilience:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
router:
|
||||
# Try same model on different provider
|
||||
tierFallbacks:
|
||||
default:
|
||||
- 'github:claude-sonnet-4-5'
|
||||
- 'openai:gpt-4o-mini'
|
||||
|
||||
# Global fallback when all tiers fail
|
||||
fallbackChain:
|
||||
- 'github:claude-sonnet-4-5'
|
||||
- 'local:ollama:llama3'
|
||||
```
|
||||
|
||||
### Retry Configuration
|
||||
|
||||
Optimize retry behavior for different scenarios:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
router:
|
||||
retry:
|
||||
# More retries for transient failures
|
||||
maxAttempts: 3
|
||||
|
||||
# Start with 1s delay
|
||||
initialDelayMs: 1000
|
||||
|
||||
# Exponential backoff
|
||||
multiplier: 2
|
||||
|
||||
# Max 30s between retries
|
||||
maxDelayMs: 30000
|
||||
|
||||
# Don't retry auth errors
|
||||
nonRetryablePatterns:
|
||||
- 'invalid_api_key'
|
||||
- 'permission_denied'
|
||||
- 'rate_limit_exceeded'
|
||||
```
|
||||
|
||||
**For production reliability:**
|
||||
```yaml
|
||||
maxAttempts: 5
|
||||
initialDelayMs: 500
|
||||
multiplier: 1.5
|
||||
maxDelayMs: 60000
|
||||
```
|
||||
|
||||
### Cost Estimation
|
||||
|
||||
Monitor token usage and costs:
|
||||
|
||||
```typescript
|
||||
// Model costs (examples)
|
||||
const MODEL_COSTS = {
|
||||
'anthropic:claude-sonnet-4-20250514': {
|
||||
input: 3.0, // $3 per 1M input tokens
|
||||
output: 15.0 // $15 per 1M output tokens
|
||||
},
|
||||
'anthropic:claude-haiku-4-20250514': {
|
||||
input: 0.25,
|
||||
output: 1.25
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
Track usage with `AgentOrchestrator.getUsageStats()`.
|
||||
|
||||
## Tool Execution
|
||||
|
||||
### Timeout Configuration
|
||||
|
||||
Set appropriate timeouts for different tool types:
|
||||
|
||||
```yaml
|
||||
tools:
|
||||
executor:
|
||||
# Default 30s timeout
|
||||
defaultTimeoutMs: 30000
|
||||
|
||||
# Max 50KB output
|
||||
maxOutputBytes: 51200
|
||||
```
|
||||
|
||||
**For long-running tools:**
|
||||
```yaml
|
||||
tools:
|
||||
executor:
|
||||
defaultTimeoutMs: 60000 # 60s
|
||||
```
|
||||
|
||||
**For fast tools:**
|
||||
```yaml
|
||||
tools:
|
||||
executor:
|
||||
defaultTimeoutMs: 10000 # 10s
|
||||
```
|
||||
|
||||
### Caching (Future)
|
||||
|
||||
Implement caching for repeated operations:
|
||||
|
||||
```yaml
|
||||
# Not yet implemented
|
||||
tools:
|
||||
cache:
|
||||
enabled: true
|
||||
ttl: 300 # 5 minutes
|
||||
maxSize: 1000
|
||||
excludePatterns:
|
||||
- 'shell.exec'
|
||||
- 'process.*'
|
||||
```
|
||||
|
||||
### Sandbox Performance
|
||||
|
||||
Docker sandbox adds overhead. Optimize:
|
||||
|
||||
```yaml
|
||||
sandbox:
|
||||
enabled: true
|
||||
image: 'node:22-alpine'
|
||||
|
||||
# Resource limits
|
||||
resourceLimits:
|
||||
memory: '512m'
|
||||
cpus: '0.5'
|
||||
timeoutSec: 60
|
||||
|
||||
# Use host networking if safe
|
||||
networkMode: 'host' # Faster than bridge mode
|
||||
```
|
||||
|
||||
**For best performance:**
|
||||
```yaml
|
||||
sandbox:
|
||||
enabled: false # Disable if not needed
|
||||
```
|
||||
|
||||
### Parallel Tool Execution
|
||||
|
||||
Flynn executes tools sequentially. For parallel execution:
|
||||
|
||||
```typescript
|
||||
// Future enhancement
|
||||
const results = await Promise.all([
|
||||
toolRegistry.execute('tool1', args1),
|
||||
toolRegistry.execute('tool2', args2),
|
||||
toolRegistry.execute('tool3', args3)
|
||||
]);
|
||||
```
|
||||
|
||||
## Memory & Embeddings
|
||||
|
||||
### Embedding Provider Selection
|
||||
|
||||
Choose embedding provider based on latency and cost:
|
||||
|
||||
```yaml
|
||||
memory:
|
||||
embeddings:
|
||||
provider: 'openai' # openai | gemini | ollama | llamacpp | voyage
|
||||
|
||||
openai:
|
||||
apiKey: '${OPENAI_API_KEY}'
|
||||
model: 'text-embedding-3-small' # Fastest
|
||||
|
||||
# Alternative: Local embeddings
|
||||
ollama:
|
||||
host: 'localhost:11434'
|
||||
model: 'nomic-embed-text'
|
||||
```
|
||||
|
||||
**Latency comparison:**
|
||||
- OpenAI `text-embedding-3-small`: ~100ms
|
||||
- Gemini: ~200ms
|
||||
- Ollama `nomic-embed-text`: ~500ms (local)
|
||||
- llama.cpp: ~300ms (local)
|
||||
|
||||
### Text Chunking
|
||||
|
||||
Optimize chunking for better search:
|
||||
|
||||
```yaml
|
||||
memory:
|
||||
embeddings:
|
||||
chunking:
|
||||
# Smaller chunks for precision
|
||||
maxChunkSize: 512
|
||||
|
||||
# Overlap for context preservation
|
||||
chunkOverlap: 50
|
||||
|
||||
# Don't chunk small documents
|
||||
minChunkSize: 128
|
||||
```
|
||||
|
||||
**For fast indexing:**
|
||||
```yaml
|
||||
maxChunkSize: 1024
|
||||
chunkOverlap: 100
|
||||
```
|
||||
|
||||
**For precise search:**
|
||||
```yaml
|
||||
maxChunkSize: 256
|
||||
chunkOverlap: 25
|
||||
```
|
||||
|
||||
### Hybrid Search Tuning
|
||||
|
||||
Balance keyword and vector search:
|
||||
|
||||
```yaml
|
||||
memory:
|
||||
search:
|
||||
# Weight vector search higher
|
||||
vectorWeight: 0.7
|
||||
keywordWeight: 0.3
|
||||
|
||||
# Return top results
|
||||
limit: 10
|
||||
|
||||
# Minimum relevance threshold
|
||||
threshold: 0.5
|
||||
```
|
||||
|
||||
**For keyword-heavy queries:**
|
||||
```yaml
|
||||
vectorWeight: 0.4
|
||||
keywordWeight: 0.6
|
||||
```
|
||||
|
||||
**For semantic queries:**
|
||||
```yaml
|
||||
vectorWeight: 0.8
|
||||
keywordWeight: 0.2
|
||||
```
|
||||
|
||||
### Embedding Caching
|
||||
|
||||
Cache embeddings to avoid recomputation:
|
||||
|
||||
```yaml
|
||||
memory:
|
||||
embeddings:
|
||||
cache:
|
||||
enabled: true
|
||||
ttl: 86400 # 24 hours
|
||||
```
|
||||
|
||||
## Session Management
|
||||
|
||||
### TTL Configuration
|
||||
|
||||
Set appropriate session TTLs:
|
||||
|
||||
```yaml
|
||||
sessions:
|
||||
ttl: '7d' # Keep sessions for 7 days
|
||||
|
||||
# Maximum concurrent sessions
|
||||
maxSessions: 100
|
||||
```
|
||||
|
||||
**For memory efficiency:**
|
||||
```yaml
|
||||
ttl: '1d'
|
||||
maxSessions: 50
|
||||
```
|
||||
|
||||
**For long-term memory:**
|
||||
```yaml
|
||||
ttl: '30d'
|
||||
maxSessions: 200
|
||||
```
|
||||
|
||||
### Session Pruning
|
||||
|
||||
Prune old sessions regularly:
|
||||
|
||||
```yaml
|
||||
automation:
|
||||
sessionPruner:
|
||||
enabled: true
|
||||
interval: '1h' # Run every hour
|
||||
|
||||
# Prune sessions older than TTL
|
||||
pruneOlderThan: '7d'
|
||||
```
|
||||
|
||||
### Session Indexing
|
||||
|
||||
Optimize session search with indexes:
|
||||
|
||||
```sql
|
||||
-- SQLite indexes
|
||||
CREATE INDEX idx_sessions_created_at ON sessions(created_at);
|
||||
CREATE INDEX idx_sessions_last_active ON sessions(last_active_at);
|
||||
CREATE INDEX idx_messages_session_id ON messages(session_id);
|
||||
```
|
||||
|
||||
## Database Performance
|
||||
|
||||
### SQLite Configuration
|
||||
|
||||
Optimize SQLite for Flynn's workload:
|
||||
|
||||
```bash
|
||||
# In SQLite connection setup
|
||||
PRAGMA journal_mode = WAL; -- Better concurrency
|
||||
PRAGMA synchronous = NORMAL; -- Faster writes
|
||||
PRAGMA cache_size = -64000; -- 64MB cache
|
||||
PRAGMA temp_store = MEMORY; -- Store temp data in memory
|
||||
PRAGMA mmap_size = 268435456; -- 256MB mmap
|
||||
PRAGMA page_size = 4096; -- Default page size
|
||||
```
|
||||
|
||||
### Connection Pooling
|
||||
|
||||
Flynn uses single SQLite connection per database. For high concurrency, consider:
|
||||
|
||||
```typescript
|
||||
// Future: Connection pool
|
||||
import Database from 'better-sqlite3';
|
||||
|
||||
const pool = new ConnectionPool({
|
||||
filename: '/path/to/database.db',
|
||||
maxConnections: 10
|
||||
});
|
||||
```
|
||||
|
||||
### Query Optimization
|
||||
|
||||
Use indexed columns in queries:
|
||||
|
||||
```typescript
|
||||
// Good: Uses index
|
||||
const sessions = db.prepare(`
|
||||
SELECT * FROM sessions
|
||||
WHERE last_active_at > ?
|
||||
ORDER BY last_active_at DESC
|
||||
LIMIT 10
|
||||
`).all(threshold);
|
||||
|
||||
// Bad: Full table scan
|
||||
const sessions = db.prepare(`
|
||||
SELECT * FROM sessions
|
||||
WHERE message_count > ?
|
||||
`).all(threshold);
|
||||
```
|
||||
|
||||
### Vacuum and Analyze
|
||||
|
||||
Regular maintenance improves performance:
|
||||
|
||||
```bash
|
||||
# Vacuum to reclaim space
|
||||
sqlite3 sessions.db "VACUUM;"
|
||||
|
||||
# Analyze for query optimization
|
||||
sqlite3 sessions.db "ANALYZE;"
|
||||
|
||||
# Rebuild indexes
|
||||
sqlite3 sessions.db "REINDEX;"
|
||||
```
|
||||
|
||||
Add to crontab (monthly):
|
||||
```
|
||||
0 0 1 * * sqlite3 /var/lib/flynn/sessions.db "VACUUM; ANALYZE;" >> /var/log/flynn-maintenance.log 2>&1
|
||||
```
|
||||
|
||||
## Gateway Performance
|
||||
|
||||
### Connection Limits
|
||||
|
||||
Limit concurrent connections:
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
enabled: true
|
||||
port: 18800
|
||||
|
||||
# Maximum concurrent WebSocket connections
|
||||
maxConnections: 50
|
||||
|
||||
# Single-client lock
|
||||
lock:
|
||||
enabled: true # Only one client at a time
|
||||
```
|
||||
|
||||
**For multiple users:**
|
||||
```yaml
|
||||
gateway:
|
||||
maxConnections: 100
|
||||
lock:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
### Lane Queue
|
||||
|
||||
The lane queue serializes requests per session:
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
laneQueue:
|
||||
# Max requests per session
|
||||
maxDepth: 10
|
||||
|
||||
# Request timeout
|
||||
requestTimeoutMs: 30000
|
||||
```
|
||||
|
||||
### WebSocket Optimization
|
||||
|
||||
Configure WebSocket for performance:
|
||||
|
||||
```typescript
|
||||
// Gateway server WebSocket options
|
||||
const wsOptions = {
|
||||
// Enable compression
|
||||
perMessageDeflate: {
|
||||
threshold: 1024
|
||||
},
|
||||
|
||||
// Ping interval (heartbeat)
|
||||
clientTracking: true,
|
||||
|
||||
// Maximum message size
|
||||
maxPayload: 16 * 1024 * 1024 // 16MB
|
||||
};
|
||||
```
|
||||
|
||||
### HTTP Server
|
||||
|
||||
Optimize HTTP server for static files:
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
static:
|
||||
# Enable gzip compression
|
||||
gzip: true
|
||||
|
||||
# Cache static assets
|
||||
cacheControl: 'public, max-age=3600'
|
||||
|
||||
# Serve index.html for SPA routes
|
||||
spa: true
|
||||
```
|
||||
|
||||
## Resource Usage
|
||||
|
||||
### Node.js Options
|
||||
|
||||
Tune Node.js for production:
|
||||
|
||||
```bash
|
||||
# Increase memory limit
|
||||
export NODE_OPTIONS="--max-old-space-size=4096"
|
||||
|
||||
# Enable optimizations
|
||||
export NODE_OPTIONS="--max-old-space-size=4096 --optimize-for-size --gc-interval=100"
|
||||
```
|
||||
|
||||
In systemd service:
|
||||
```ini
|
||||
Environment="NODE_OPTIONS=--max-old-space-size=4096"
|
||||
```
|
||||
|
||||
### Process Limits
|
||||
|
||||
Set appropriate limits:
|
||||
|
||||
```ini
|
||||
[Service]
|
||||
# Memory limit (2GB)
|
||||
MemoryLimit=2G
|
||||
MemorySwap=0
|
||||
|
||||
# CPU quota (200% = 2 cores)
|
||||
CPUQuota=200%
|
||||
|
||||
# File descriptors
|
||||
LimitNOFILE=65536
|
||||
```
|
||||
|
||||
### Docker Resource Limits
|
||||
|
||||
Constrain Docker container:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
flynn:
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
cpus: '2.0'
|
||||
memory: 2G
|
||||
reservations:
|
||||
cpus: '1.0'
|
||||
memory: 1G
|
||||
```
|
||||
|
||||
### Memory Monitoring
|
||||
|
||||
Monitor memory usage:
|
||||
|
||||
```bash
|
||||
# Check Flynn memory
|
||||
ps aux | grep flynn
|
||||
|
||||
# System memory
|
||||
free -h
|
||||
|
||||
# Node.js heap stats (add to code)
|
||||
console.log('Heap used:', process.memoryUsage().heapUsed / 1024 / 1024, 'MB');
|
||||
```
|
||||
|
||||
## Monitoring & Profiling
|
||||
|
||||
### Health Checks
|
||||
|
||||
Enable gateway health endpoint:
|
||||
|
||||
```yaml
|
||||
automation:
|
||||
heartbeat:
|
||||
enabled: true
|
||||
interval: '5m'
|
||||
checks:
|
||||
- 'gateway'
|
||||
- 'model'
|
||||
- 'channels'
|
||||
- 'memory'
|
||||
- 'disk'
|
||||
```
|
||||
|
||||
Check health:
|
||||
```bash
|
||||
curl http://localhost:18800/health
|
||||
```
|
||||
|
||||
### Logging Levels
|
||||
|
||||
Configure logging appropriately:
|
||||
|
||||
```yaml
|
||||
logging:
|
||||
level: 'info' # debug | info | warn | error
|
||||
```
|
||||
|
||||
**Development:** `debug` - All messages
|
||||
**Production:** `info` - Normal operation
|
||||
**Minimal:** `warn` - Only warnings and errors
|
||||
|
||||
### Performance Metrics
|
||||
|
||||
Track key metrics:
|
||||
|
||||
```typescript
|
||||
// Future: Metrics collection
|
||||
interface Metrics {
|
||||
// Response times
|
||||
avgResponseTime: number;
|
||||
p95ResponseTime: number;
|
||||
p99ResponseTime: number;
|
||||
|
||||
// Throughput
|
||||
requestsPerSecond: number;
|
||||
concurrentSessions: number;
|
||||
|
||||
// Token usage
|
||||
avgInputTokens: number;
|
||||
avgOutputTokens: number;
|
||||
totalTokens: number;
|
||||
|
||||
// Errors
|
||||
errorRate: number;
|
||||
timeoutRate: number;
|
||||
}
|
||||
```
|
||||
|
||||
### Profiling
|
||||
|
||||
Profile Node.js execution:
|
||||
|
||||
```bash
|
||||
# Generate CPU profile
|
||||
node --prof dist/cli/index.js start
|
||||
|
||||
# Process profile
|
||||
node --prof-process isolate-*.log > profile.txt
|
||||
|
||||
# Analyze with Chrome DevTools
|
||||
# Open chrome://inspect and load profile
|
||||
```
|
||||
|
||||
### Flamegraphs
|
||||
|
||||
Generate flamegraphs for bottleneck analysis:
|
||||
|
||||
```bash
|
||||
# Install 0x
|
||||
npm install -g 0x
|
||||
|
||||
# Run with profiler
|
||||
0x dist/cli/index.js start
|
||||
```
|
||||
|
||||
## Common Performance Issues
|
||||
|
||||
### High Memory Usage
|
||||
|
||||
**Symptoms:**
|
||||
- OOM errors
|
||||
- Slow garbage collection
|
||||
- System swapping
|
||||
|
||||
**Solutions:**
|
||||
1. Reduce `keepTurns` in compaction
|
||||
2. Decrease session TTL
|
||||
3. Prune old sessions
|
||||
4. Increase Node.js memory limit
|
||||
5. Check for memory leaks
|
||||
|
||||
### Slow Response Times
|
||||
|
||||
**Symptoms:**
|
||||
- Responses > 10 seconds
|
||||
- Timeouts
|
||||
- Poor user experience
|
||||
|
||||
**Solutions:**
|
||||
1. Switch to faster model tier
|
||||
2. Enable compaction
|
||||
3. Use local fallbacks
|
||||
4. Optimize tool timeouts
|
||||
5. Check network latency
|
||||
|
||||
### High CPU Usage
|
||||
|
||||
**Symptoms:**
|
||||
- CPU > 80%
|
||||
- Slow system
|
||||
- High latency
|
||||
|
||||
**Solutions:**
|
||||
1. Reduce concurrent sessions
|
||||
2. Optimize database queries
|
||||
3. Use efficient embeddings
|
||||
4. Disable unnecessary features
|
||||
5. Scale vertically (more CPU)
|
||||
|
||||
### Database Locks
|
||||
|
||||
**Symptoms:**
|
||||
- SQLite database locked errors
|
||||
- Slow writes
|
||||
- Concurrent access issues
|
||||
|
||||
**Solutions:**
|
||||
1. Enable WAL mode
|
||||
2. Reduce write frequency
|
||||
3. Use connection pooling
|
||||
4. Add appropriate indexes
|
||||
|
||||
### Model Rate Limits
|
||||
|
||||
**Symptoms:**
|
||||
- 429 Too Many Requests errors
|
||||
- Frequent fallbacks
|
||||
- Increased latency
|
||||
|
||||
**Solutions:**
|
||||
1. Configure retry with exponential backoff
|
||||
2. Use faster models for delegated tasks
|
||||
3. Implement request queuing
|
||||
4. Add local model fallbacks
|
||||
|
||||
## Performance Checklist
|
||||
|
||||
Before deploying to production, verify:
|
||||
|
||||
- [ ] Compaction configured with appropriate threshold
|
||||
- [ ] Model tiers configured for cost/latency
|
||||
- [ ] Fallback chains configured
|
||||
- [ ] Tool timeouts set appropriately
|
||||
- [ ] Session TTL reasonable for use case
|
||||
- [ ] SQLite optimized (WAL mode, cache size)
|
||||
- [ ] Database indexes created
|
||||
- [ ] Gateway connection limits set
|
||||
- [ ] Memory limits configured
|
||||
- [ ] Monitoring enabled
|
||||
- [ ] Logging level set to `info` or `warn`
|
||||
- [ ] Health checks working
|
||||
- [ ] Backup/restore tested
|
||||
|
||||
---
|
||||
|
||||
For more information:
|
||||
- [TROUBLESHOOTING.md](../../TROUBLESHOOTING.md)
|
||||
- [PRODUCTION.md](../deployment/PRODUCTION.md)
|
||||
- [ARCHITECTURE.md](../../.planning/codebase/ARCHITECTURE.md)
|
||||
Reference in New Issue
Block a user