William Valentin
82 lines
2.1 KiB
Markdown
82 lines
2.1 KiB
Markdown
# Backend Switch Command Design
|
|
|
|
**Date:** 2026-02-05
|
|
**Status:** Approved
|
|
|
|
## Overview
|
|
|
|
Add `/backend` slash command to switch between local LLM providers (ollama, llamacpp) at runtime with lazy instantiation.
|
|
|
|
## Config Schema Extension
|
|
|
|
Extend `models` with optional `local_providers` object:
|
|
|
|
```yaml
|
|
models:
|
|
local:
|
|
provider: ollama
|
|
model: qwen2.5:14b
|
|
endpoint: http://localhost:11434
|
|
|
|
local_providers:
|
|
llamacpp:
|
|
endpoint: http://localhost:8080
|
|
# auth_token: optional
|
|
```
|
|
|
|
- `local` - Active local provider (backwards compatible)
|
|
- `local_providers` - Additional provider configs keyed by name
|
|
|
|
## Command
|
|
|
|
```
|
|
/backend Show current local provider
|
|
/backend ollama Switch to ollama
|
|
/backend llamacpp Switch to llamacpp
|
|
```
|
|
|
|
### Command Type
|
|
|
|
```typescript
|
|
| { type: 'backend'; provider?: string }
|
|
```
|
|
|
|
## ModelRouter Changes
|
|
|
|
```typescript
|
|
// Swap local client at runtime
|
|
setLocalClient(client: ModelClient, providerName: string): void
|
|
|
|
// Get current local provider name for display
|
|
getLocalProviderName(): string | undefined
|
|
```
|
|
|
|
## Flow
|
|
|
|
1. User runs `/backend llamacpp`
|
|
2. Command parsed, handler receives `{ type: 'backend', provider: 'llamacpp' }`
|
|
3. Handler looks up `config.models.local_providers.llamacpp`
|
|
4. If not found → error "Backend 'llamacpp' not configured"
|
|
5. Instantiate `LlamaCppClient` with config
|
|
6. Call `router.setLocalClient(client, 'llamacpp')`
|
|
7. Report "Switched to llamacpp"
|
|
|
|
## Error Handling
|
|
|
|
| Scenario | Response |
|
|
|----------|----------|
|
|
| `/backend` (no arg) | "Local backend: ollama" |
|
|
| Unknown provider | "Unknown backend 'foo'. Available: ollama, llamacpp" |
|
|
| Not configured | "Backend 'llamacpp' not configured in local_providers" |
|
|
| Server down | Fail on first chat (existing error handling) |
|
|
|
|
## Files Changed
|
|
|
|
| File | Change |
|
|
|------|--------|
|
|
| `src/config/schema.ts` | Add `local_providers` to modelsSchema |
|
|
| `src/models/router.ts` | Add `setLocalClient()`, `getLocalProviderName()` |
|
|
| `src/frontends/tui/commands.ts` | Add backend command parsing |
|
|
| `src/frontends/tui/minimal.ts` | Handle backend command |
|
|
| `src/daemon/index.ts` | Store config, add client factory function |
|