docs: improve local backend reliability and documentation
- Switch local endpoints from localhost to 127.0.0.1 to avoid IPv6 resolution issues\n- Add recommended Ollama user systemd service example with keep-alive and CUDA support
This commit is contained in:
@@ -448,7 +448,7 @@ audio:
|
|||||||
enabled: true
|
enabled: true
|
||||||
provider:
|
provider:
|
||||||
type: custom # openai, groq, ollama, llamacpp, custom
|
type: custom # openai, groq, ollama, llamacpp, custom
|
||||||
endpoint: "http://localhost:18801/v1/audio/transcriptions"
|
endpoint: "http://127.0.0.1:18801/v1/audio/transcriptions"
|
||||||
api_key: "${WHISPER_API_KEY}" # Optional Bearer token
|
api_key: "${WHISPER_API_KEY}" # Optional Bearer token
|
||||||
model: "whisper-1" # Model name (default: whisper-1)
|
model: "whisper-1" # Model name (default: whisper-1)
|
||||||
```
|
```
|
||||||
@@ -549,7 +549,7 @@ docker compose --profile search up -d searxng
|
|||||||
Local-first + homelab fallback pattern:
|
Local-first + homelab fallback pattern:
|
||||||
|
|
||||||
- If Flynn runs in Docker Compose, set `endpoint: "http://searxng:8080"`.
|
- If Flynn runs in Docker Compose, set `endpoint: "http://searxng:8080"`.
|
||||||
- If Flynn runs on the host (`pnpm dev` / `pnpm start`), set `endpoint: "http://localhost:18803"`.
|
- If Flynn runs on the host (`pnpm dev` / `pnpm start`), set `endpoint: "http://127.0.0.1:18803"`.
|
||||||
- Set `fallback_endpoint` to your existing homelab SearXNG URL.
|
- Set `fallback_endpoint` to your existing homelab SearXNG URL.
|
||||||
|
|
||||||
Example for host-run Flynn with local compose SearXNG and homelab backup:
|
Example for host-run Flynn with local compose SearXNG and homelab backup:
|
||||||
@@ -557,7 +557,7 @@ Example for host-run Flynn with local compose SearXNG and homelab backup:
|
|||||||
```yaml
|
```yaml
|
||||||
web_search:
|
web_search:
|
||||||
provider: searxng
|
provider: searxng
|
||||||
endpoint: "http://localhost:18803"
|
endpoint: "http://127.0.0.1:18803"
|
||||||
fallback_endpoint: "https://searxng.homelab.local"
|
fallback_endpoint: "https://searxng.homelab.local"
|
||||||
max_results: 5
|
max_results: 5
|
||||||
```
|
```
|
||||||
@@ -795,6 +795,37 @@ Or via tools:
|
|||||||
|
|
||||||
## Running as Service
|
## Running as Service
|
||||||
|
|
||||||
|
### Ollama (Recommended)
|
||||||
|
|
||||||
|
If running Ollama locally, use a user-level service to ensure it stays alive and reachable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat > ~/.config/systemd/user/ollama.service << 'EOF'
|
||||||
|
[Unit]
|
||||||
|
Description=Ollama Service (User)
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
ExecStart=/usr/bin/ollama serve
|
||||||
|
Restart=always
|
||||||
|
RestartSec=3
|
||||||
|
Environment="HOME=%h"
|
||||||
|
Environment="OLLAMA_MODELS=%h/.ollama"
|
||||||
|
Environment="LD_LIBRARY_PATH=/usr/lib:/opt/cuda/lib64"
|
||||||
|
Environment="CUDA_VISIBLE_DEVICES=0"
|
||||||
|
Environment="OLLAMA_KEEP_ALIVE=-1"
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
|
EOF
|
||||||
|
|
||||||
|
systemctl --user daemon-reload
|
||||||
|
systemctl --user enable --now ollama
|
||||||
|
```
|
||||||
|
|
||||||
|
### Flynn daemon
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Create systemd user service
|
# Create systemd user service
|
||||||
mkdir -p ~/.config/systemd/user
|
mkdir -p ~/.config/systemd/user
|
||||||
@@ -1020,7 +1051,7 @@ backup:
|
|||||||
include_vectors: true
|
include_vectors: true
|
||||||
minio:
|
minio:
|
||||||
enabled: true
|
enabled: true
|
||||||
endpoint: localhost:9000
|
endpoint: 127.0.0.1:9000
|
||||||
access_key: "${MINIO_ACCESS_KEY}"
|
access_key: "${MINIO_ACCESS_KEY}"
|
||||||
secret_key: "${MINIO_SECRET_KEY}"
|
secret_key: "${MINIO_SECRET_KEY}"
|
||||||
bucket: flynn-backups
|
bucket: flynn-backups
|
||||||
@@ -1411,7 +1442,7 @@ memory:
|
|||||||
|----------|--------|
|
|----------|--------|
|
||||||
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
|
| OpenAI | `provider: openai`, `api_key`, `model` (default: `text-embedding-3-small`) |
|
||||||
| Gemini | `provider: gemini`, `api_key`, `model` |
|
| Gemini | `provider: gemini`, `api_key`, `model` |
|
||||||
| Ollama | `provider: ollama`, `endpoint` (default: localhost:11434), `model` |
|
| Ollama | `provider: ollama`, `endpoint` (default: 127.0.0.1:11434), `model` |
|
||||||
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |
|
| llama.cpp | `provider: llamacpp`, `endpoint`, optional `model` |
|
||||||
| Voyage AI | `provider: voyageai`, `api_key` or `VOYAGE_API_KEY`, `model` (default: `voyage-3-large`) |
|
| Voyage AI | `provider: voyageai`, `api_key` or `VOYAGE_API_KEY`, `model` (default: `voyage-3-large`) |
|
||||||
|
|
||||||
|
|||||||
+4
-4
@@ -186,11 +186,11 @@ models:
|
|||||||
ollama:
|
ollama:
|
||||||
provider: ollama
|
provider: ollama
|
||||||
model: glm-4.7-flash
|
model: glm-4.7-flash
|
||||||
endpoint: http://localhost:11434
|
endpoint: http://127.0.0.1:11434
|
||||||
llamacpp:
|
llamacpp:
|
||||||
provider: llamacpp
|
provider: llamacpp
|
||||||
model: gpt-oss-20b
|
model: gpt-oss-20b
|
||||||
endpoint: http://localhost:8080
|
endpoint: http://127.0.0.1:8080
|
||||||
#
|
#
|
||||||
# Then reference them in fallback_chain:
|
# Then reference them in fallback_chain:
|
||||||
# fallback_chain: [ollama, llamacpp, local]
|
# fallback_chain: [ollama, llamacpp, local]
|
||||||
@@ -311,7 +311,7 @@ memory:
|
|||||||
enabled: true
|
enabled: true
|
||||||
provider: ollama
|
provider: ollama
|
||||||
model: nomic-embed-text
|
model: nomic-embed-text
|
||||||
endpoint: http://localhost:11434
|
endpoint: http://127.0.0.1:11434
|
||||||
chunk_size: 512
|
chunk_size: 512
|
||||||
chunk_overlap: 50
|
chunk_overlap: 50
|
||||||
top_k: 5
|
top_k: 5
|
||||||
@@ -565,7 +565,7 @@ automation:
|
|||||||
# enabled: true
|
# enabled: true
|
||||||
# provider:
|
# provider:
|
||||||
# type: custom # openai, groq, ollama, llamacpp, custom
|
# type: custom # openai, groq, ollama, llamacpp, custom
|
||||||
# endpoint: "http://localhost:18801/v1/audio/transcriptions"
|
# endpoint: "http://127.0.0.1:18801/v1/audio/transcriptions"
|
||||||
# api_key: "${WHISPER_API_KEY}" # Optional Bearer token
|
# api_key: "${WHISPER_API_KEY}" # Optional Bearer token
|
||||||
# model: "whisper-1" # Model name (default: whisper-1)
|
# model: "whisper-1" # Model name (default: whisper-1)
|
||||||
# talk_mode:
|
# talk_mode:
|
||||||
|
|||||||
Reference in New Issue
Block a user