docs: improve local backend reliability and documentation

- Switch local endpoints from localhost to 127.0.0.1 to avoid IPv6 resolution issues\n- Add recommended Ollama user systemd service example with keep-alive and CUDA support
This commit is contained in:
William Valentin
2026-02-26 12:19:28 -08:00
parent d07e05d4cc
commit adcef2249b
2 changed files with 40 additions and 9 deletions
+36 -5
View File
@@ -448,7 +448,7 @@ audio:
enabled: true
provider:
type: custom # openai, groq, ollama, llamacpp, custom
endpoint: "http://localhost:18801/v1/audio/transcriptions"
endpoint: "http://127.0.0.1:18801/v1/audio/transcriptions"
api_key: "${WHISPER_API_KEY}" # Optional Bearer token
model: "whisper-1" # Model name (default: whisper-1)
```
@@ -549,7 +549,7 @@ docker compose --profile search up -d searxng
Local-first + homelab fallback pattern:
- If Flynn runs in Docker Compose, set `endpoint: "http://searxng:8080"`.
- If Flynn runs on the host (`pnpm dev` / `pnpm start`), set `endpoint: "http://localhost:18803"`.
- If Flynn runs on the host (`pnpm dev` / `pnpm start`), set `endpoint: "http://127.0.0.1:18803"`.
- Set `fallback_endpoint` to your existing homelab SearXNG URL.
Example for host-run Flynn with local compose SearXNG and homelab backup:
@@ -557,7 +557,7 @@ Example for host-run Flynn with local compose SearXNG and homelab backup:
```yaml
web_search:
provider: searxng
endpoint: "http://localhost:18803"
endpoint: "http://127.0.0.1:18803"
fallback_endpoint: "https://searxng.homelab.local"
max_results: 5
```
@@ -795,6 +795,37 @@ Or via tools:
## Running as Service
### Ollama (Recommended)
If running Ollama locally, use a user-level service to ensure it stays alive and reachable:
```bash
cat > ~/.config/systemd/user/ollama.service << 'EOF'
[Unit]
Description=Ollama Service (User)
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/ollama serve
Restart=always
RestartSec=3
Environment="HOME=%h"
Environment="OLLAMA_MODELS=%h/.ollama"
Environment="LD_LIBRARY_PATH=/usr/lib:/opt/cuda/lib64"
Environment="CUDA_VISIBLE_DEVICES=0"
Environment="OLLAMA_KEEP_ALIVE=-1"
[Install]
WantedBy=default.target
EOF
systemctl --user daemon-reload
systemctl --user enable --now ollama
```
### Flynn daemon
```bash
# Create systemd user service
mkdir -p ~/.config/systemd/user
@@ -1020,7 +1051,7 @@ backup:
include_vectors: true
minio:
enabled: true
endpoint: localhost:9000
endpoint: 127.0.0.1:9000
access_key: "${MINIO_ACCESS_KEY}"
secret_key: "${MINIO_SECRET_KEY}"
bucket: flynn-backups
@@ -1411,7 +1442,7 @@ memory:
|----------|--------|
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
| Gemini | `provider: gemini`, `api_key`, `model` |
| Ollama | `provider: ollama`, `endpoint` (default: localhost:11434), `model` |
| Ollama | `provider: ollama`, `endpoint` (default: 127.0.0.1:11434), `model` |
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |
| Voyage AI | `provider: voyageai`, `api_key` or `VOYAGE_API_KEY`, `model` (default: `voyage-3-large`) |
+4 -4
View File
@@ -186,11 +186,11 @@ models:
ollama:
provider: ollama
model: glm-4.7-flash
endpoint: http://localhost:11434
endpoint: http://127.0.0.1:11434
llamacpp:
provider: llamacpp
model: gpt-oss-20b
endpoint: http://localhost:8080
endpoint: http://127.0.0.1:8080
#
# Then reference them in fallback_chain:
# fallback_chain: [ollama, llamacpp, local]
@@ -311,7 +311,7 @@ memory:
enabled: true
provider: ollama
model: nomic-embed-text
endpoint: http://localhost:11434
endpoint: http://127.0.0.1:11434
chunk_size: 512
chunk_overlap: 50
top_k: 5
@@ -565,7 +565,7 @@ automation:
# enabled: true
# provider:
# type: custom # openai, groq, ollama, llamacpp, custom
# endpoint: "http://localhost:18801/v1/audio/transcriptions"
# endpoint: "http://127.0.0.1:18801/v1/audio/transcriptions"
# api_key: "${WHISPER_API_KEY}" # Optional Bearer token
# model: "whisper-1" # Model name (default: whisper-1)
# talk_mode: