docs: improve local backend reliability and documentation

- Switch local endpoints from localhost to 127.0.0.1 to avoid IPv6 resolution issues\n- Add recommended Ollama user systemd service example with keep-alive and CUDA support
This commit is contained in:
William Valentin
2026-02-26 12:19:28 -08:00
parent d07e05d4cc
commit adcef2249b
2 changed files with 40 additions and 9 deletions
+36 -5
View File
@@ -448,7 +448,7 @@ audio:
enabled: true enabled: true
provider: provider:
type: custom # openai, groq, ollama, llamacpp, custom type: custom # openai, groq, ollama, llamacpp, custom
endpoint: "http://localhost:18801/v1/audio/transcriptions" endpoint: "http://127.0.0.1:18801/v1/audio/transcriptions"
api_key: "${WHISPER_API_KEY}" # Optional Bearer token api_key: "${WHISPER_API_KEY}" # Optional Bearer token
model: "whisper-1" # Model name (default: whisper-1) model: "whisper-1" # Model name (default: whisper-1)
``` ```
@@ -549,7 +549,7 @@ docker compose --profile search up -d searxng
Local-first + homelab fallback pattern: Local-first + homelab fallback pattern:
- If Flynn runs in Docker Compose, set `endpoint: "http://searxng:8080"`. - If Flynn runs in Docker Compose, set `endpoint: "http://searxng:8080"`.
- If Flynn runs on the host (`pnpm dev` / `pnpm start`), set `endpoint: "http://localhost:18803"`. - If Flynn runs on the host (`pnpm dev` / `pnpm start`), set `endpoint: "http://127.0.0.1:18803"`.
- Set `fallback_endpoint` to your existing homelab SearXNG URL. - Set `fallback_endpoint` to your existing homelab SearXNG URL.
Example for host-run Flynn with local compose SearXNG and homelab backup: Example for host-run Flynn with local compose SearXNG and homelab backup:
@@ -557,7 +557,7 @@ Example for host-run Flynn with local compose SearXNG and homelab backup:
```yaml ```yaml
web_search: web_search:
provider: searxng provider: searxng
endpoint: "http://localhost:18803" endpoint: "http://127.0.0.1:18803"
fallback_endpoint: "https://searxng.homelab.local" fallback_endpoint: "https://searxng.homelab.local"
max_results: 5 max_results: 5
``` ```
@@ -795,6 +795,37 @@ Or via tools:
## Running as Service ## Running as Service
### Ollama (Recommended)
If running Ollama locally, use a user-level service to ensure it stays alive and reachable:
```bash
cat > ~/.config/systemd/user/ollama.service << 'EOF'
[Unit]
Description=Ollama Service (User)
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/ollama serve
Restart=always
RestartSec=3
Environment="HOME=%h"
Environment="OLLAMA_MODELS=%h/.ollama"
Environment="LD_LIBRARY_PATH=/usr/lib:/opt/cuda/lib64"
Environment="CUDA_VISIBLE_DEVICES=0"
Environment="OLLAMA_KEEP_ALIVE=-1"
[Install]
WantedBy=default.target
EOF
systemctl --user daemon-reload
systemctl --user enable --now ollama
```
### Flynn daemon
```bash ```bash
# Create systemd user service # Create systemd user service
mkdir -p ~/.config/systemd/user mkdir -p ~/.config/systemd/user
@@ -1020,7 +1051,7 @@ backup:
include_vectors: true include_vectors: true
minio: minio:
enabled: true enabled: true
endpoint: localhost:9000 endpoint: 127.0.0.1:9000
access_key: "${MINIO_ACCESS_KEY}" access_key: "${MINIO_ACCESS_KEY}"
secret_key: "${MINIO_SECRET_KEY}" secret_key: "${MINIO_SECRET_KEY}"
bucket: flynn-backups bucket: flynn-backups
@@ -1411,7 +1442,7 @@ memory:
|----------|--------| |----------|--------|
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) | | OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
| Gemini | `provider: gemini`, `api_key`, `model` | | Gemini | `provider: gemini`, `api_key`, `model` |
| Ollama | `provider: ollama`, `endpoint` (default: localhost:11434), `model` | | Ollama | `provider: ollama`, `endpoint` (default: 127.0.0.1:11434), `model` |
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` | | llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |
| Voyage AI | `provider: voyageai`, `api_key` or `VOYAGE_API_KEY`, `model` (default: `voyage-3-large`) | | Voyage AI | `provider: voyageai`, `api_key` or `VOYAGE_API_KEY`, `model` (default: `voyage-3-large`) |
+4 -4
View File
@@ -186,11 +186,11 @@ models:
ollama: ollama:
provider: ollama provider: ollama
model: glm-4.7-flash model: glm-4.7-flash
endpoint: http://localhost:11434 endpoint: http://127.0.0.1:11434
llamacpp: llamacpp:
provider: llamacpp provider: llamacpp
model: gpt-oss-20b model: gpt-oss-20b
endpoint: http://localhost:8080 endpoint: http://127.0.0.1:8080
# #
# Then reference them in fallback_chain: # Then reference them in fallback_chain:
# fallback_chain: [ollama, llamacpp, local] # fallback_chain: [ollama, llamacpp, local]
@@ -311,7 +311,7 @@ memory:
enabled: true enabled: true
provider: ollama provider: ollama
model: nomic-embed-text model: nomic-embed-text
endpoint: http://localhost:11434 endpoint: http://127.0.0.1:11434
chunk_size: 512 chunk_size: 512
chunk_overlap: 50 chunk_overlap: 50
top_k: 5 top_k: 5
@@ -565,7 +565,7 @@ automation:
# enabled: true # enabled: true
# provider: # provider:
# type: custom # openai, groq, ollama, llamacpp, custom # type: custom # openai, groq, ollama, llamacpp, custom
# endpoint: "http://localhost:18801/v1/audio/transcriptions" # endpoint: "http://127.0.0.1:18801/v1/audio/transcriptions"
# api_key: "${WHISPER_API_KEY}" # Optional Bearer token # api_key: "${WHISPER_API_KEY}" # Optional Bearer token
# model: "whisper-1" # Model name (default: whisper-1) # model: "whisper-1" # Model name (default: whisper-1)
# talk_mode: # talk_mode: