This commit adds 6 new documentation files to fill critical gaps: - CONTRIBUTING.md: Developer onboarding guide with setup, workflow, code style, testing, and adding features - TROUBLESHOOTING.md: Common issues and solutions for errors, model issues, tool issues, channel issues, gateway issues, configuration issues, and memory/database issues - docs/api/PROTOCOL.md: Gateway JSON-RPC protocol documentation with connection, authentication, message format, methods, events, error codes, and example client implementation - docs/api/TOOLS.md: Tools API documentation covering tool interface, input schema format, result format, tool patterns, tool registration, tool policy, execution flow, and builtin tools reference - docs/deployment/PRODUCTION.md: Production deployment guide covering Docker deployment, systemd service, security, configuration, monitoring, backup & recovery, and performance tuning - docs/performance/TUNING.md: Performance optimization guide covering context management, model routing, tool execution, memory & embeddings, session management, database performance, gateway performance, and resource usage These files complement the existing excellent documentation (README.md, AGENTS.md, ARCHITECTURE.md, STRUCTURE.md, CONVENTIONS.md) to provide complete coverage for users, developers, and operators.
13 KiB
Troubleshooting Flynn
This guide covers common issues, error messages, and debugging techniques for Flynn.
Table of Contents
- Common Errors
- Model Issues
- Tool Issues
- Channel Issues
- Gateway Issues
- Configuration Issues
- Memory & Database Issues
- Debug Mode
- Getting Help
Common Errors
Error: Environment variable FLYNN_CONFIG is not set
Flynn can't find your configuration file.
Solution:
# Create config from template
cp config/default.yaml ~/.config/flynn/config.yaml
# Or specify config file explicitly
flynn start --config ~/my-config.yaml
Error: Failed to load config: ...
Configuration validation failed. Check your YAML syntax.
Solution:
# Validate your config
flynn doctor --config ~/.config/flynn/config.yaml
# Check YAML syntax
cat ~/.config/flynn/config.yaml | yamllint
Common issues:
- Missing quotes around special characters
- Incorrect indentation (YAML is space-sensitive, 2 spaces)
- Invalid boolean values (use
true/false, notyes/no)
Error: Tool 'xxx' is not allowed by tool policy
Tool execution blocked by policy configuration.
Solution:
-
Check tool policy in config:
tools: policy: 'full' # or 'coding', 'messaging', 'minimal' profiles: full: allow: ['*'] deny: [] -
Check agent-specific tool overrides:
agents: my-agent: toolPolicy: 'full' -
Verify tool is registered (check
src/daemon/index.ts)
Error: Session not found
Gateway tried to access a non-existent session.
Solution:
# List active sessions
flynn sessions
# Sessions are auto-created on first message
# Ensure you're sending to a valid channel/sender combination
Model Issues
Model refuses to answer / "I can't help with that"
The model may be rejecting requests due to content filters or safety constraints.
Solution:
- Try rephrasing your request
- Check if the model has specific content policies
- Try a different model tier:
# In TUI, switch models /model complex
Rate Limit Errors / Too Many Requests
API rate limits exceeded.
Solution:
-
Reduce request frequency
-
Add retry configuration in config:
models: default: anthropic: apiKey: 'sk-...' retry: maxAttempts: 3 initialDelayMs: 1000 maxDelayMs: 30000 multiplier: 2 -
Switch to a different provider or tier
Model Fallback Not Working
Fallback chain not triggering on errors.
Solution:
-
Check model router config:
models: router: tiers: default: 'anthropic:claude-sonnet-4-20250514' fallbackChain: - 'github:claude-sonnet-4-5' - 'local:ollama:llama3' -
Check error patterns in retry config (errors matching these patterns won't retry):
retry: nonRetryablePatterns: - 'invalid_api_key' - 'permission_denied' -
Enable debug logging to see fallback decisions:
DEBUG='*' flynn start
Context Window Exceeded
Conversation too long for model's context window.
Solution:
-
Check compaction settings:
agents: default: compaction: thresholdPct: 80 keepTurns: 4 summaryMaxTokens: 1024 -
Increase
keepTurnsto preserve more recent history -
Increase
thresholdPctto trigger compaction earlier
Tool Issues
Tool Execution Timeout
Tool took too long to complete.
Solution:
-
Check timeout config:
tools: executor: defaultTimeoutMs: 30000 -
Increase timeout for specific tools (if supported):
shell.exec --timeout 60000 "long-running-command" -
Check if tool is hanging (stuck process, network issue)
Tool Permission Denied
Tool can't access file/execute command.
Solution:
-
Check file permissions:
ls -la /path/to/file chmod +x /path/to/executable -
Check Docker sandbox permissions (if enabled):
# Ensure Docker daemon is running docker ps # Check Flynn's Docker access groups $USER | grep docker -
Verify hook configuration (hooks may block tools):
flynn config | grep -A 20 hooks
Tool Output Truncated
Output exceeds maximum size limit.
Solution:
Check max output config:
tools:
executor:
maxOutputBytes: 51200 # 50KB default
Increase limit if needed, or use file operations to handle large outputs.
Tool Not Found
Tool doesn't exist or isn't registered.
Solution:
# List available tools
flynn doctor --config ~/.config/flynn/config.yaml
# Check if tool is in builtin list
grep -r "name: 'your.tool'" src/tools/builtin/
If you added a custom tool, ensure it's registered in src/daemon/index.ts.
Channel Issues
Telegram Bot Not Responding
Telegram bot not receiving or processing messages.
Solution:
-
Check bot token in config:
channels: telegram: enabled: true token: '123456789:ABCdefGHIjklMNOpqrSTUvwxYZ' -
Verify bot is running:
# Check Flynn logs for telegram startup flynn start 2>&1 | grep telegram -
Test bot via Telegram API:
curl https://api.telegram.org/bot<YOUR_TOKEN>/getMe -
Check
allowed_chat_idswhitelist:channels: telegram: allowedChatIds: ['123456789'] # Your chat ID
Discord Bot Not Joining Channels
Discord bot permissions issues.
Solution:
-
Check bot token:
channels: discord: enabled: true token: 'MTIzNDU2Nzg5O...ABcDefGhIjKlMnOpQrStUvWxYz' -
Verify guild/channel IDs in whitelist:
channels: discord: allowedGuildIds: ['123456789012345678'] allowedChannelIds: ['123456789012345678'] -
Check bot permissions in Discord server (need
Read Messages,Send Messages,Embed Links)
Slack Webhooks Not Receiving
Slack event subscription issues.
Solution:
-
Verify signing secret in config:
channels: slack: enabled: true signingSecret: 'a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6' -
Check Slack app permissions and event subscriptions
-
Verify ngrok/tunnel if testing locally
WhatsApp Bot Not Connecting
WhatsApp Web.js connection issues.
Solution:
-
Check phone number whitelist:
channels: whatsapp: enabled: true allowedNumbers: ['+1234567890'] -
WhatsApp requires QR code scan on first run - run in foreground:
flynn start # Watch console for QR code -
If running in Docker, ensure it can display QR code or use saved session
Gateway Issues
WebSocket Connection Refused
Can't connect to gateway WebSocket.
Solution:
-
Check gateway is running:
curl http://localhost:18800/health -
Check port configuration:
gateway: enabled: true port: 18800 -
Check firewall rules:
sudo ufw allow 18800 -
Check auth token if configured:
gateway: auth: token: 'your-secret-token'
Gateway Lock Active
Only one WebSocket client allowed at a time.
Solution:
gateway:
lock:
enabled: false # Disable lock
Or disconnect existing client first.
Tailscale Serve Not Exposing
Tailscale integration not working.
Solution:
-
Ensure Tailscale is installed and running:
tailscale status -
Check Tailscale config:
gateway: tailscaleServe: enabled: true hostname: 'my-flynn' port: 443 -
Check Flynn logs for Tailscale errors:
flynn start 2>&1 | grep -i tailscale
Configuration Issues
Invalid YAML Syntax
Configuration file has YAML syntax errors.
Solution:
# Validate with YAML linter
pip install yamllint
yamllint ~/.config/flynn/config.yaml
# Or use online YAML validator
# https://www.yamllint.com/
Common mistakes:
- Using tabs instead of spaces (YAML requires spaces)
- Incorrect indentation (must be consistent)
- Missing quotes around special characters (
:,{,},[,]) - Unclosed brackets or quotes
Environment Variable Expansion Not Working
${ENV_VAR} in config not expanding.
Solution:
-
Ensure variable is set:
echo $MY_API_KEY -
Check variable syntax in config:
models: default: anthropic: apiKey: '${ANTHROPIC_API_KEY}' # Correct # Not: $ANTHROPIC_API_KEY or "${ANTHROPIC_API_KEY}" -
Ensure no quotes around the entire value:
# Wrong apiKey: '${ANTHROPIC_API_KEY}' # Correct apiKey: '${ANTHROPIC_API_KEY}'
Secrets Showing in Logs
API keys or sensitive data visible in logs.
Solution:
Flynn automatically redacts secrets when showing config with flynn config.
If you see secrets in output, check:
- Don't log config manually (use the built-in redaction)
- Don't commit config files to git (add to
.gitignore) - Use
FLYNN_CONFIGenv var for sensitive paths
Memory & Database Issues
SQLite Database Locked
Can't write to session or vector database.
Solution:
-
Check if another Flynn instance is running:
ps aux | grep flynn -
Kill existing instances:
pkill flynn -
Check database permissions:
ls -la ~/.local/share/flynn/*.db chmod 644 ~/.local/share/flynn/*.db
Memory Not Saving
Memory writes not persisting.
Solution:
-
Check memory directory:
ls -la ~/.local/share/flynn/memory/ -
Check namespace format (must be valid filename):
# Good memory.write --namespace 'my-knowledge' # Bad memory.write --namespace 'my/namespace' # Creates directory -
Check disk space:
df -h ~/.local/share/flynn/
Vector Search Returns No Results
Hybrid search not finding matches.
Solution:
-
Ensure embeddings are generated:
# Check vector database sqlite3 ~/.local/share/flynn/vectors.db "SELECT COUNT(*) FROM embeddings;" -
Check embedding provider config:
memory: embeddings: provider: 'openai' # or 'gemini', 'ollama', 'llamacpp' openai: apiKey: '${OPENAI_API_KEY}' model: 'text-embedding-3-small' -
Try keyword search only:
memory.search --query 'my query' --keyword-only
Debug Mode
Enable Debug Logging
Flynn uses console logging. Enable verbose output:
# Set DEBUG environment variable (if using debug packages)
DEBUG='*' flynn start
# Or redirect all output to file
flynn start > /tmp/flynn.log 2>&1
# Monitor logs in real-time
tail -f /tmp/flynn.log
Run Doctor Command
Use the built-in diagnostic tool:
# Full check
flynn doctor
# Check specific component
flynn doctor --config ~/.config/flynn/config.yaml
# Check connectivity
flynn doctor --check connectivity
The doctor checks:
- Config syntax and validation
- Model provider connectivity
- Channel adapter status
- Memory and database integrity
- Gateway health
Test Components Individually
# Test model connectivity
flynn send "Hello, world!"
# Test specific tool (in TUI)
/file.read /path/to/file
# Test channel
# Send message via platform (Telegram, Discord, etc.)
# Test gateway
curl http://localhost:18800/health
Inspect Session Data
# View sessions
flynn sessions
# Inspect session database directly
sqlite3 ~/.local/share/flynn/sessions.db "SELECT * FROM sessions LIMIT 5;"
# View session messages
sqlite3 ~/.local/share/flynn/sessions.db "SELECT * FROM messages WHERE session_id = '...';"
Enable Heartbeat Monitoring
Flynn includes a heartbeat monitor that checks system health:
automation:
heartbeat:
enabled: true
interval: '5m'
checks:
- 'gateway'
- 'model'
- 'channels'
- 'memory'
- 'disk'
Failures will be logged and notifications sent (if configured).
Getting Help
Log Your Issue
When reporting issues, include:
- Error message (full stack trace if available)
- Flynn version:
flynn --version - Configuration (redact secrets):
flynn config - Node version:
node --version - OS:
uname -a - Steps to reproduce
Check Existing Issues
Search GitHub issues for similar problems:
Ask for Help
- GitHub Issues: Bug reports and feature requests
- GitHub Discussions: Questions and community help
- Documentation: Check README.md, AGENTS.md, and this file
Provide Diagnostic Output
# Run doctor and save output
flynn doctor > /tmp/flynn-doctor.txt 2>&1
# Get config (redacted)
flynn config > /tmp/flynn-config.txt
# Get logs
journalctl -u flynn -n 100 --no-pager > /tmp/flynn-logs.txt
# Attach these files to your issue
Still having issues? Open a GitHub issue with the diagnostic output above.