Backend Switch Command Design

Date: 2026-02-05 Status: Approved

Overview

Add /backend slash command to switch between local LLM providers (ollama, llamacpp) at runtime with lazy instantiation.

Config Schema Extension

Extend models with optional local_providers object:

models:
  local:
    provider: ollama
    model: qwen2.5:14b
    endpoint: http://localhost:11434

  local_providers:
    llamacpp:
      endpoint: http://localhost:8080
      # auth_token: optional

local - Active local provider (backwards compatible)
local_providers - Additional provider configs keyed by name

Command

/backend              Show current local provider
/backend ollama       Switch to ollama
/backend llamacpp     Switch to llamacpp

Command Type

| { type: 'backend'; provider?: string }

ModelRouter Changes

// Swap local client at runtime
setLocalClient(client: ModelClient, providerName: string): void

// Get current local provider name for display
getLocalProviderName(): string | undefined

Flow

User runs /backend llamacpp
Command parsed, handler receives { type: 'backend', provider: 'llamacpp' }
Handler looks up config.models.local_providers.llamacpp
If not found → error "Backend 'llamacpp' not configured"
Instantiate LlamaCppClient with config
Call router.setLocalClient(client, 'llamacpp')
Report "Switched to llamacpp"

Error Handling

Scenario	Response
`/backend` (no arg)	"Local backend: ollama"
Unknown provider	"Unknown backend 'foo'. Available: ollama, llamacpp"
Not configured	"Backend 'llamacpp' not configured in local_providers"
Server down	Fail on first chat (existing error handling)

Files Changed

File	Change
`src/config/schema.ts`	Add `local_providers` to modelsSchema
`src/models/router.ts`	Add `setLocalClient()`, `getLocalProviderName()`
`src/frontends/tui/commands.ts`	Add backend command parsing
`src/frontends/tui/minimal.ts`	Handle backend command
`src/daemon/index.ts`	Store config, add client factory function

2.1 KiB Raw Blame History