docs: add backend switch implementation plan
This commit is contained in:
William Valentin
@@ -0,0 +1,418 @@
|
||||
# Backend Switch Command Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add `/backend` slash command to switch between local LLM providers (ollama, llamacpp) at runtime.
|
||||
|
||||
**Architecture:** Extend config schema with `local_providers` object, add `setLocalClient()` to ModelRouter, add command parsing and handling in TUI.
|
||||
|
||||
**Tech Stack:** TypeScript, Zod (config validation), Vitest (testing)
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Extend Config Schema
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/config/schema.ts`
|
||||
|
||||
### Step 1: Add local_providers to schema
|
||||
|
||||
Update `src/config/schema.ts` - add `local_providers` field to modelsSchema:
|
||||
|
||||
```typescript
|
||||
const modelsSchema = z.object({
|
||||
local: modelConfigSchema.optional(),
|
||||
fast: modelConfigSchema.optional(),
|
||||
default: modelConfigSchema,
|
||||
complex: modelConfigSchema.optional(),
|
||||
fallback_chain: z.array(z.string()).default(['anthropic']),
|
||||
local_providers: z.record(z.string(), modelConfigSchema).optional(),
|
||||
});
|
||||
```
|
||||
|
||||
### Step 2: Verify build
|
||||
|
||||
Run: `npx tsc --noEmit`
|
||||
Expected: No errors
|
||||
|
||||
### Step 3: Commit
|
||||
|
||||
```bash
|
||||
git add src/config/schema.ts
|
||||
git commit -m "feat: add local_providers to config schema"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Add ModelRouter Methods
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/models/router.ts`
|
||||
- Modify: `src/models/router.test.ts`
|
||||
|
||||
### Step 1: Write the failing test
|
||||
|
||||
Add to `src/models/router.test.ts`:
|
||||
|
||||
```typescript
|
||||
describe('ModelRouter local client switching', () => {
|
||||
it('allows setting a new local client', () => {
|
||||
const mockDefault = { chat: vi.fn() } as unknown as ModelClient;
|
||||
const mockLocal1 = { chat: vi.fn() } as unknown as ModelClient;
|
||||
const mockLocal2 = { chat: vi.fn() } as unknown as ModelClient;
|
||||
|
||||
const router = new ModelRouter({
|
||||
default: mockDefault,
|
||||
local: mockLocal1,
|
||||
fallbackChain: [],
|
||||
});
|
||||
|
||||
expect(router.getLocalProviderName()).toBe(undefined);
|
||||
|
||||
router.setLocalClient(mockLocal2, 'llamacpp');
|
||||
|
||||
expect(router.getLocalProviderName()).toBe('llamacpp');
|
||||
expect(router.getClient('local')).toBe(mockLocal2);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Step 2: Run test to verify it fails
|
||||
|
||||
Run: `npm test -- src/models/router.test.ts`
|
||||
Expected: FAIL with "setLocalClient is not a function" or "getLocalProviderName is not a function"
|
||||
|
||||
### Step 3: Write minimal implementation
|
||||
|
||||
Add to `src/models/router.ts`:
|
||||
|
||||
1. Add private field after line 17:
|
||||
```typescript
|
||||
private localProviderName?: string;
|
||||
```
|
||||
|
||||
2. Add methods after `getClient()` (after line 113):
|
||||
```typescript
|
||||
setLocalClient(client: ModelClient, providerName: string): void {
|
||||
this.clients.set('local', client);
|
||||
this.localProviderName = providerName;
|
||||
}
|
||||
|
||||
getLocalProviderName(): string | undefined {
|
||||
return this.localProviderName;
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Run test to verify it passes
|
||||
|
||||
Run: `npm test -- src/models/router.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add src/models/router.ts src/models/router.test.ts
|
||||
git commit -m "feat: add setLocalClient and getLocalProviderName to ModelRouter"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add Backend Command Parsing
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/frontends/tui/commands.ts`
|
||||
- Modify: `src/frontends/tui/commands.test.ts`
|
||||
|
||||
### Step 1: Write the failing tests
|
||||
|
||||
Add to `src/frontends/tui/commands.test.ts`:
|
||||
|
||||
```typescript
|
||||
it('parses /backend command without argument', () => {
|
||||
expect(parseCommand('/backend')).toEqual({ type: 'backend' });
|
||||
});
|
||||
|
||||
it('parses /backend command with argument', () => {
|
||||
expect(parseCommand('/backend llamacpp')).toEqual({ type: 'backend', provider: 'llamacpp' });
|
||||
expect(parseCommand('/backend ollama')).toEqual({ type: 'backend', provider: 'ollama' });
|
||||
});
|
||||
```
|
||||
|
||||
### Step 2: Run test to verify it fails
|
||||
|
||||
Run: `npm test -- src/frontends/tui/commands.test.ts`
|
||||
Expected: FAIL - returns message type instead of backend type
|
||||
|
||||
### Step 3: Implement command parsing
|
||||
|
||||
Update `src/frontends/tui/commands.ts`:
|
||||
|
||||
1. Add to Command type (after line 8):
|
||||
```typescript
|
||||
| { type: 'backend'; provider?: string }
|
||||
```
|
||||
|
||||
2. Add parsing after the model block (after line 47):
|
||||
```typescript
|
||||
// Backend (with optional argument)
|
||||
if (trimmed === '/backend') {
|
||||
return { type: 'backend' };
|
||||
}
|
||||
if (trimmed.startsWith('/backend ')) {
|
||||
const provider = trimmed.slice('/backend '.length).trim();
|
||||
return { type: 'backend', provider };
|
||||
}
|
||||
```
|
||||
|
||||
3. Add to getHelpText() (after line 63, before /reset):
|
||||
```typescript
|
||||
/backend [provider] Show or switch local backend (ollama, llamacpp)
|
||||
```
|
||||
|
||||
4. Add to SLASH_COMMANDS array (after '/model'):
|
||||
```typescript
|
||||
'/backend',
|
||||
```
|
||||
|
||||
5. Add to COMMAND_TOOLTIPS (after '/model'):
|
||||
```typescript
|
||||
'/backend': 'Show or switch local backend (ollama, llamacpp)',
|
||||
```
|
||||
|
||||
### Step 4: Run test to verify it passes
|
||||
|
||||
Run: `npm test -- src/frontends/tui/commands.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add src/frontends/tui/commands.ts src/frontends/tui/commands.test.ts
|
||||
git commit -m "feat: add /backend command parsing"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Add Backend Command Handler to MinimalTui
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/frontends/tui/minimal.ts`
|
||||
- Modify: `src/frontends/tui/minimal.test.ts`
|
||||
|
||||
### Step 1: Write the failing test
|
||||
|
||||
Add to `src/frontends/tui/minimal.test.ts` (import ModelConfig type first):
|
||||
|
||||
```typescript
|
||||
import type { ModelConfig } from '../../config/schema.js';
|
||||
|
||||
describe('MinimalTui backend command', () => {
|
||||
it('switches local backend when provider is configured', async () => {
|
||||
const mockSession = {
|
||||
id: 'test',
|
||||
getHistory: () => [],
|
||||
addMessage: vi.fn(),
|
||||
clear: vi.fn(),
|
||||
};
|
||||
|
||||
const mockRouter = {
|
||||
getTier: () => 'default' as const,
|
||||
getAvailableTiers: () => ['default', 'local'],
|
||||
setTier: vi.fn(() => true),
|
||||
getLocalProviderName: () => 'ollama',
|
||||
setLocalClient: vi.fn(),
|
||||
chat: vi.fn(),
|
||||
getClient: vi.fn(),
|
||||
};
|
||||
|
||||
const localProviders: Record<string, ModelConfig> = {
|
||||
llamacpp: {
|
||||
provider: 'llamacpp',
|
||||
model: '',
|
||||
endpoint: 'http://localhost:8080',
|
||||
},
|
||||
};
|
||||
|
||||
const tui = new MinimalTui({
|
||||
session: mockSession as any,
|
||||
modelClient: mockRouter as any,
|
||||
modelRouter: mockRouter as any,
|
||||
systemPrompt: 'test',
|
||||
localProviders,
|
||||
});
|
||||
|
||||
// Access private method for testing
|
||||
await (tui as any).handleBackendCommand('llamacpp');
|
||||
|
||||
expect(mockRouter.setLocalClient).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Step 2: Run test to verify it fails
|
||||
|
||||
Run: `npm test -- src/frontends/tui/minimal.test.ts`
|
||||
Expected: FAIL - handleBackendCommand doesn't exist or localProviders not accepted
|
||||
|
||||
### Step 3: Implement handler
|
||||
|
||||
Update `src/frontends/tui/minimal.ts`:
|
||||
|
||||
1. Add import at top:
|
||||
```typescript
|
||||
import type { ModelConfig } from '../../config/schema.js';
|
||||
import { OllamaClient, LlamaCppClient } from '../../models/index.js';
|
||||
```
|
||||
|
||||
2. Add to MinimalTuiConfig interface (after line 23):
|
||||
```typescript
|
||||
localProviders?: Record<string, ModelConfig>;
|
||||
currentLocalProvider?: string;
|
||||
```
|
||||
|
||||
3. Add case in handleCommand switch (after line 100, before 'message'):
|
||||
```typescript
|
||||
case 'backend':
|
||||
this.handleBackendCommand(command.provider);
|
||||
break;
|
||||
```
|
||||
|
||||
4. Add handler method (after handleModelCommand, around line 130):
|
||||
```typescript
|
||||
private handleBackendCommand(provider?: string): void {
|
||||
const router = this.config.modelRouter;
|
||||
if (!router) {
|
||||
console.log('Backend switching not available.\n');
|
||||
return;
|
||||
}
|
||||
|
||||
if (!provider) {
|
||||
const current = router.getLocalProviderName() ?? this.config.currentLocalProvider ?? 'unknown';
|
||||
const available = this.getAvailableBackends();
|
||||
console.log(`Current local backend: ${current}`);
|
||||
console.log(`Available: ${available.join(', ')}\n`);
|
||||
return;
|
||||
}
|
||||
|
||||
const providerConfig = this.config.localProviders?.[provider];
|
||||
if (!providerConfig) {
|
||||
const available = this.getAvailableBackends();
|
||||
console.log(`Backend '${provider}' not configured.`);
|
||||
console.log(`Available: ${available.join(', ')}\n`);
|
||||
return;
|
||||
}
|
||||
|
||||
const client = this.createLocalClient(providerConfig);
|
||||
if (!client) {
|
||||
console.log(`Failed to create client for '${provider}'.\n`);
|
||||
return;
|
||||
}
|
||||
|
||||
router.setLocalClient(client, provider);
|
||||
console.log(`Switched to backend: ${provider}\n`);
|
||||
}
|
||||
|
||||
private getAvailableBackends(): string[] {
|
||||
const backends: string[] = [];
|
||||
if (this.config.currentLocalProvider) {
|
||||
backends.push(this.config.currentLocalProvider);
|
||||
}
|
||||
if (this.config.localProviders) {
|
||||
backends.push(...Object.keys(this.config.localProviders));
|
||||
}
|
||||
return [...new Set(backends)];
|
||||
}
|
||||
|
||||
private createLocalClient(config: ModelConfig): ModelClient | null {
|
||||
if (config.provider === 'ollama') {
|
||||
return new OllamaClient({
|
||||
model: config.model,
|
||||
host: config.endpoint,
|
||||
});
|
||||
}
|
||||
if (config.provider === 'llamacpp') {
|
||||
return new LlamaCppClient({
|
||||
endpoint: config.endpoint ?? 'http://localhost:8080',
|
||||
authToken: config.auth_token,
|
||||
});
|
||||
}
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Run test to verify it passes
|
||||
|
||||
Run: `npm test -- src/frontends/tui/minimal.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add src/frontends/tui/minimal.ts src/frontends/tui/minimal.test.ts
|
||||
git commit -m "feat: add /backend command handler to MinimalTui"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Wire Up Config in TUI Entry Point
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/tui.ts`
|
||||
|
||||
### Step 1: Read current file and update
|
||||
|
||||
Read `src/tui.ts` to understand current structure, then update to pass `localProviders` and `currentLocalProvider` to MinimalTui.
|
||||
|
||||
The changes needed:
|
||||
1. Extract `local_providers` from config
|
||||
2. Extract current local provider name from `config.models.local.provider`
|
||||
3. Pass both to MinimalTui config
|
||||
|
||||
### Step 2: Run full test suite
|
||||
|
||||
Run: `npm test`
|
||||
Expected: All tests pass
|
||||
|
||||
### Step 3: Commit
|
||||
|
||||
```bash
|
||||
git add src/tui.ts
|
||||
git commit -m "feat: wire up localProviders config to TUI"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Verify and Final Test
|
||||
|
||||
**Files:** None (verification only)
|
||||
|
||||
### Step 1: Run full test suite
|
||||
|
||||
Run: `npm test`
|
||||
Expected: All tests pass
|
||||
|
||||
### Step 2: Type check
|
||||
|
||||
Run: `npx tsc --noEmit`
|
||||
Expected: No errors
|
||||
|
||||
### Step 3: Build
|
||||
|
||||
Run: `npm run build`
|
||||
Expected: Build succeeds
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Task | Description | Tests Added |
|
||||
|------|-------------|-------------|
|
||||
| 1 | Config schema extension | 0 |
|
||||
| 2 | ModelRouter methods | 1 |
|
||||
| 3 | Command parsing | 2 |
|
||||
| 4 | MinimalTui handler | 1 |
|
||||
| 5 | Wire up config | 0 |
|
||||
| 6 | Verification | 0 |
|
||||
|
||||
**Total new tests:** 4
|
||||
**Files modified:** 7
|
||||
Reference in New Issue
Block a user