feat: implement model persistence with per-session overrides
- Add session_config SQLite table for per-session settings - Update routing to support session override → agent config → global default resolution chain - Upgrade WebChat SessionBridge from NativeAgent to AgentOrchestrator - Add /model, /local, /cloud commands to Telegram adapter - Add /model command to WebChat gateway handlers - Clear session overrides on /reset command - Pass memoryStore and config through to SessionBridge - Add comprehensive tests for all new functionality Fixes model persistence bug where TUI model changes didn't affect WebChat/Telegram sessions. Now: - TUI /model sets global default (persists across restarts, affects all new sessions) - WebChat/Telegram /model sets session override (only that conversation, cleared on /reset) - WebChat sessions gain AgentOrchestrator features (delegation, compaction, memory)
This commit is contained in:
William Valentin
@@ -0,0 +1,870 @@
|
||||
# Plan: Model Persistence with Per-Session Overrides
|
||||
|
||||
## Summary
|
||||
|
||||
This plan fixes model tier persistence so it works correctly across TUI, WebChat, and Telegram channels, and adds per-session model overrides that survive daemon restarts. Currently the model tier is a single global preference stored in `preferences.json` and propagated via `ModelRouter.setTier()`. This creates three problems:
|
||||
|
||||
1. **Global coupling**: Switching models in TUI changes WebChat sessions too (via `SessionBridge.onTierChanged`)
|
||||
2. **No per-session persistence**: Channel adapter sessions (Telegram, Discord, etc.) can't have independent model overrides
|
||||
3. **WebChat uses raw NativeAgent**: The `SessionBridge` creates `NativeAgent` instances directly instead of `AgentOrchestrator`, losing delegation, compaction, memory, and proper usage tracking
|
||||
|
||||
The solution introduces a `session_config` SQLite table for per-session key-value settings, wires it through `SessionManager`, and changes the model resolution chain to: **session override → agent config → global default**.
|
||||
|
||||
## Context
|
||||
|
||||
- **Request**: Fix model persistence across all channels with per-session overrides
|
||||
- **Codebase**: Flynn multi-channel AI assistant daemon with TUI, WebChat (gateway), Telegram, Discord, Slack adapters
|
||||
- **Constraints**: Must be backwards-compatible; existing `preferences.json` global default must still work; no breaking changes to gateway protocol
|
||||
|
||||
## Current Architecture (Problems)
|
||||
|
||||
### Problem 1: Global Model Tier Coupling
|
||||
|
||||
In `src/daemon/index.ts` (lines 104-110):
|
||||
```ts
|
||||
modelRouter.setOnTierChange((tier) => savePreference(dataDir, 'modelTier', tier));
|
||||
```
|
||||
|
||||
In `src/gateway/session-bridge.ts` (lines 38-51):
|
||||
```ts
|
||||
// SessionBridge subscribes to tier changes and updates ALL WebChat agents
|
||||
router.addOnTierChange((tier: ModelTier) => this.onTierChanged(tier));
|
||||
// ...
|
||||
private onTierChanged(tier: ModelTier): void {
|
||||
this.currentTier = tier;
|
||||
for (const agent of this.agents.values()) {
|
||||
agent.setModelTier(tier);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This means `/model fast` in TUI immediately switches all WebChat sessions to `fast` too.
|
||||
|
||||
### Problem 2: No Per-Session Config Persistence
|
||||
|
||||
The `SessionStore` only stores messages. There's no mechanism to persist per-session settings like model tier overrides. When the daemon restarts, all sessions lose their model tier.
|
||||
|
||||
### Problem 3: WebChat Uses NativeAgent Instead of AgentOrchestrator
|
||||
|
||||
`SessionBridge.getOrCreateAgent()` (line 186) creates raw `NativeAgent` instances:
|
||||
```ts
|
||||
agent = new NativeAgent({
|
||||
modelClient: this.config.modelClient,
|
||||
systemPrompt: this.config.systemPrompt,
|
||||
session,
|
||||
toolRegistry: this.config.toolRegistry,
|
||||
toolExecutor: this.config.toolExecutor,
|
||||
});
|
||||
```
|
||||
|
||||
This means WebChat sessions lack: delegation to sub-agents, automatic compaction, memory injection, proper cost tracking, and tool policy context.
|
||||
|
||||
### Problem 4: Telegram Has No `/model` Command in Channel Adapter
|
||||
|
||||
The `TelegramAdapter` (channel-based) has `/reset` and `/start` commands but no `/model` command. The old `src/frontends/telegram/bot.ts` has `/local`, `/cloud`, `/model` commands but those operate on a shared `NativeAgent`, not per-session.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### Decision 1: SQLite `session_config` Table for Per-Session Settings
|
||||
- **Choice**: Add a `session_config` table with `(session_id, key, value)` columns
|
||||
- **Rationale**: Sessions already persist in SQLite; config is a natural extension. Key-value is flexible enough for future settings (not just model tier).
|
||||
- **Alternatives considered**: JSON file per session (too many files), adding columns to messages table (wrong abstraction), in-memory only (lost on restart)
|
||||
|
||||
### Decision 2: Three-Level Model Resolution Chain
|
||||
- **Choice**: Session override → Agent config → Global default
|
||||
- **Rationale**: Allows users to override per-session while falling back to agent routing config and then global preferences. This matches the existing pattern in `routing.ts` line 63.
|
||||
- **Alternatives considered**: Two-level (session → global) loses agent config routing; single global (current) is too coarse
|
||||
|
||||
### Decision 3: WebChat Upgrade to AgentOrchestrator
|
||||
- **Choice**: Replace `NativeAgent` with `AgentOrchestrator` in `SessionBridge`
|
||||
- **Rationale**: WebChat sessions should have feature parity with channel adapter sessions (delegation, compaction, memory). This is a prerequisite for per-session model persistence to work properly.
|
||||
- **Alternatives considered**: Keep NativeAgent but add a tier override field (doesn't fix delegation/compaction gap)
|
||||
|
||||
### Decision 4: Model Commands via Channel Message Metadata
|
||||
- **Choice**: Handle `/model` in Telegram adapter by sending metadata through the message handler, and in WebChat via the agent.send handler
|
||||
- **Rationale**: Consistent with existing `/reset` pattern — commands are dispatched as `InboundMessage` with `metadata.isCommand = true`
|
||||
- **Alternatives considered**: Separate RPC endpoint for model switching (adds protocol complexity)
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Database Schema — `session_config` Table
|
||||
|
||||
**Files**: `src/session/store.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Add `session_config` table creation in `init()` method
|
||||
- [ ] Add CRUD methods for session config
|
||||
|
||||
**Details**:
|
||||
|
||||
In `init()` (line 22), add to the `this.db.exec()` call:
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS session_config (
|
||||
session_id TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
PRIMARY KEY (session_id, key)
|
||||
);
|
||||
```
|
||||
|
||||
Add these new methods to `SessionStore`:
|
||||
|
||||
```ts
|
||||
/** Get a single config value for a session. */
|
||||
getSessionConfig(sessionId: string, key: string): string | undefined {
|
||||
const stmt = this.db.prepare(
|
||||
'SELECT value FROM session_config WHERE session_id = ? AND key = ?',
|
||||
);
|
||||
const row = stmt.get(sessionId, key) as { value: string } | undefined;
|
||||
return row?.value;
|
||||
}
|
||||
|
||||
/** Get all config values for a session. */
|
||||
getAllSessionConfig(sessionId: string): Record<string, string> {
|
||||
const stmt = this.db.prepare(
|
||||
'SELECT key, value FROM session_config WHERE session_id = ?',
|
||||
);
|
||||
const rows = stmt.all(sessionId) as Array<{ key: string; value: string }>;
|
||||
const result: Record<string, string> = {};
|
||||
for (const row of rows) {
|
||||
result[row.key] = row.value;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/** Set a config value for a session. Upserts (INSERT OR REPLACE). */
|
||||
setSessionConfig(sessionId: string, key: string, value: string): void {
|
||||
this.db.prepare(
|
||||
'INSERT OR REPLACE INTO session_config (session_id, key, value) VALUES (?, ?, ?)',
|
||||
).run(sessionId, key, value);
|
||||
}
|
||||
|
||||
/** Delete a config value for a session. */
|
||||
deleteSessionConfig(sessionId: string, key: string): void {
|
||||
this.db.prepare(
|
||||
'DELETE FROM session_config WHERE session_id = ? AND key = ?',
|
||||
).run(sessionId, key);
|
||||
}
|
||||
|
||||
/** Delete all config for a session (used when session is cleared/pruned). */
|
||||
clearSessionConfig(sessionId: string): void {
|
||||
this.db.prepare(
|
||||
'DELETE FROM session_config WHERE session_id = ?',
|
||||
).run(sessionId);
|
||||
}
|
||||
```
|
||||
|
||||
Also update `clearSession()` (line 80) to also clear config:
|
||||
|
||||
```ts
|
||||
clearSession(sessionId: string): void {
|
||||
const transaction = this.db.transaction(() => {
|
||||
this.db.prepare('DELETE FROM messages WHERE session_id = ?').run(sessionId);
|
||||
this.db.prepare('DELETE FROM session_config WHERE session_id = ?').run(sessionId);
|
||||
});
|
||||
transaction();
|
||||
}
|
||||
```
|
||||
|
||||
And update `pruneStale()` (line 92) to also clean up config for pruned sessions:
|
||||
|
||||
```ts
|
||||
pruneStale(beforeTimestamp: number): string[] {
|
||||
// ... existing stale detection ...
|
||||
const deleteMessages = this.db.prepare('DELETE FROM messages WHERE session_id = ?');
|
||||
const deleteConfig = this.db.prepare('DELETE FROM session_config WHERE session_id = ?');
|
||||
const transaction = this.db.transaction(() => {
|
||||
for (const { session_id } of stale) {
|
||||
deleteMessages.run(session_id);
|
||||
deleteConfig.run(session_id);
|
||||
}
|
||||
});
|
||||
transaction();
|
||||
return stale.map(r => r.session_id);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 2: SessionManager Config Methods
|
||||
|
||||
**Files**: `src/session/manager.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Add `getSessionConfig()`, `setSessionConfig()`, `deleteSessionConfig()` methods to `SessionManager`
|
||||
- [ ] Add config methods to `ManagedSession`
|
||||
- [ ] Update `Session` interface
|
||||
|
||||
**Details**:
|
||||
|
||||
Update the `Session` interface (line 5):
|
||||
|
||||
```ts
|
||||
export interface Session {
|
||||
id: string;
|
||||
addMessage(message: Message): void;
|
||||
getHistory(): Message[];
|
||||
clear(): void;
|
||||
replaceHistory(messages: Message[]): void;
|
||||
getConfig(key: string): string | undefined;
|
||||
setConfig(key: string, value: string): void;
|
||||
deleteConfig(key: string): void;
|
||||
}
|
||||
```
|
||||
|
||||
Add methods to `ManagedSession` (after line 66):
|
||||
|
||||
```ts
|
||||
getConfig(key: string): string | undefined {
|
||||
return this.store.getSessionConfig(this.id, key);
|
||||
}
|
||||
|
||||
setConfig(key: string, value: string): void {
|
||||
this.store.setSessionConfig(this.id, key, value);
|
||||
}
|
||||
|
||||
deleteConfig(key: string): void {
|
||||
this.store.deleteSessionConfig(this.id, key);
|
||||
}
|
||||
```
|
||||
|
||||
Add convenience methods to `SessionManager` (after line 131):
|
||||
|
||||
```ts
|
||||
/** Get a session config value. */
|
||||
getSessionConfig(frontend: string, userId: string, key: string): string | undefined {
|
||||
const session = this.getSession(frontend, userId);
|
||||
return session.getConfig(key);
|
||||
}
|
||||
|
||||
/** Set a session config value. */
|
||||
setSessionConfig(frontend: string, userId: string, key: string, value: string): void {
|
||||
const session = this.getSession(frontend, userId);
|
||||
session.setConfig(key, value);
|
||||
}
|
||||
|
||||
/** Delete a session config value. */
|
||||
deleteSessionConfig(frontend: string, userId: string, key: string): void {
|
||||
const session = this.getSession(frontend, userId);
|
||||
session.deleteConfig(key);
|
||||
}
|
||||
```
|
||||
|
||||
Update `src/session/index.ts` to export the new `Session` interface properly (already done, `Session` is re-exported).
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Model Resolution in `routing.ts`
|
||||
|
||||
**Files**: `src/daemon/routing.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Read per-session model tier override when creating/reusing an agent
|
||||
- [ ] Persist model tier to session config when it changes
|
||||
- [ ] Add `/model` command handling in the message handler
|
||||
|
||||
**Details**:
|
||||
|
||||
In `getOrCreateAgent()` (line 43), after getting the session, read the session config:
|
||||
|
||||
```ts
|
||||
const session = deps.sessionManager.getSession(channel, senderId);
|
||||
|
||||
// Read per-session model tier override (persisted in SQLite)
|
||||
const sessionTierOverride = session.getConfig('modelTier') as ModelTier | undefined;
|
||||
|
||||
// Resolution chain: metadata (cron) → session override → agent config → global default
|
||||
const effectiveTier = tierFromMetadata
|
||||
?? sessionTierOverride
|
||||
?? agentConfig?.modelTier
|
||||
?? deps.config.agents.primary_tier
|
||||
?? 'default';
|
||||
```
|
||||
|
||||
This replaces line 63 which currently doesn't check session config.
|
||||
|
||||
Add a `/model` command handler in the `handler` function (after the existing `/usage` handler, around line 212):
|
||||
|
||||
```ts
|
||||
if (msg.metadata.command === 'model') {
|
||||
const modelArg = msg.metadata.commandArgs as string | undefined;
|
||||
const { orchestrator: agent } = getOrCreateAgent(msg.channel, msg.senderId, msg.metadata);
|
||||
const session = deps.sessionManager.getSession(msg.channel, msg.senderId);
|
||||
|
||||
if (!modelArg) {
|
||||
// Show current model tier
|
||||
const currentTier = agent.getModelTier();
|
||||
const sessionOverride = session.getConfig('modelTier');
|
||||
const available = deps.modelRouter.getAvailableTiers();
|
||||
const labels = deps.modelRouter.getAllLabels();
|
||||
const lines = [`Active tier: ${currentTier}${sessionOverride ? ' (session override)' : ''}`];
|
||||
for (const tier of available) {
|
||||
const label = labels[tier] ?? 'unknown';
|
||||
const marker = tier === currentTier ? ' ←' : '';
|
||||
lines.push(` ${tier}: ${label}${marker}`);
|
||||
}
|
||||
await reply({ text: lines.join('\n'), replyTo: msg.id });
|
||||
return;
|
||||
}
|
||||
|
||||
// Resolve alias and switch tier
|
||||
const tier = resolveModelAlias(modelArg);
|
||||
if (!deps.modelRouter.getAvailableTiers().includes(tier)) {
|
||||
await reply({ text: `Model tier not available: ${modelArg}`, replyTo: msg.id });
|
||||
return;
|
||||
}
|
||||
|
||||
// Persist to session config
|
||||
session.setConfig('modelTier', tier);
|
||||
|
||||
// Update the orchestrator's agent tier
|
||||
agent.setModelTier(tier);
|
||||
|
||||
const label = deps.modelRouter.getLabel(tier);
|
||||
await reply({ text: `Switched to model: ${tier} (${label})`, replyTo: msg.id });
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
Also add the `resolveModelAlias` import at the top:
|
||||
|
||||
```ts
|
||||
import { resolveModelAlias } from '../frontends/tui/commands.js';
|
||||
```
|
||||
|
||||
And update the audio tier resolution section (lines 217-228) to also check session override:
|
||||
|
||||
```ts
|
||||
let effectiveTier: string = deps.config.agents.primary_tier ?? 'default';
|
||||
const session = deps.sessionManager.getSession(msg.channel, msg.senderId);
|
||||
const sessionTierOverride = session.getConfig('modelTier');
|
||||
|
||||
if (msg.metadata?.modelTier) {
|
||||
effectiveTier = msg.metadata.modelTier as string;
|
||||
} else if (sessionTierOverride) {
|
||||
effectiveTier = sessionTierOverride;
|
||||
} else if (deps.agentRouter && deps.agentConfigRegistry) {
|
||||
// ... existing agent config lookup ...
|
||||
}
|
||||
```
|
||||
|
||||
**Important**: The agent cache in `routing.ts` keys on session ID + agent config name. When the model tier changes per-session, we need to invalidate the cached agent or update it in place. Since `AgentOrchestrator.setModelTier()` already exists and updates the underlying `NativeAgent`, updating in place is sufficient — no cache invalidation needed.
|
||||
|
||||
However, note that `tierFromMetadata` (for cron jobs) currently affects the cache key (line 55). Per-session overrides should NOT affect the cache key because the agent is already created and we update it in place.
|
||||
|
||||
---
|
||||
|
||||
### Step 4: Reset Command — Clear Session Overrides
|
||||
|
||||
**Files**: `src/daemon/routing.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] When `/reset` is handled, also clear session config
|
||||
|
||||
**Details**:
|
||||
|
||||
In the reset handler (line 167):
|
||||
|
||||
```ts
|
||||
if (msg.metadata.command === 'reset') {
|
||||
agent.reset();
|
||||
// Clear per-session config overrides
|
||||
const session = deps.sessionManager.getSession(msg.channel, msg.senderId);
|
||||
session.deleteConfig('modelTier');
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 5: WebChat SessionBridge — Upgrade to AgentOrchestrator
|
||||
|
||||
**Files**: `src/gateway/session-bridge.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Replace `NativeAgent` with `AgentOrchestrator`
|
||||
- [ ] Accept additional config for orchestrator creation (delegation, compaction, memory, etc.)
|
||||
- [ ] Remove global tier sync (each session manages its own tier)
|
||||
- [ ] Load per-session tier from session config on creation
|
||||
|
||||
**Details**:
|
||||
|
||||
Update the interface and imports:
|
||||
|
||||
```ts
|
||||
import { AgentOrchestrator, type DelegationConfig } from '../backends/native/orchestrator.js';
|
||||
import type { MemoryStore } from '../memory/store.js';
|
||||
import type { Config } from '../config/index.js';
|
||||
// Remove NativeAgent import (no longer directly used)
|
||||
```
|
||||
|
||||
Update `SessionBridgeConfig`:
|
||||
|
||||
```ts
|
||||
export interface SessionBridgeConfig {
|
||||
sessionManager: SessionManager;
|
||||
modelClient: ModelClient | ModelRouter;
|
||||
systemPrompt: string;
|
||||
toolRegistry: ToolRegistry;
|
||||
toolExecutor: ToolExecutor;
|
||||
config?: Config;
|
||||
memoryStore?: MemoryStore;
|
||||
}
|
||||
```
|
||||
|
||||
Replace `NativeAgent` references with `AgentOrchestrator`:
|
||||
|
||||
```ts
|
||||
interface ClientEntry {
|
||||
connectionId: string;
|
||||
sessionId: string;
|
||||
agent: AgentOrchestrator;
|
||||
busy: boolean;
|
||||
}
|
||||
|
||||
export class SessionBridge {
|
||||
private clients: Map<string, ClientEntry> = new Map();
|
||||
private agents: Map<string, AgentOrchestrator> = new Map();
|
||||
// ...
|
||||
```
|
||||
|
||||
**Remove** the `onTierChanged` method and the `addOnTierChange` subscription in the constructor. WebChat sessions should NOT follow TUI tier changes — they manage their own tier via session config.
|
||||
|
||||
Update `getOrCreateAgent()`:
|
||||
|
||||
```ts
|
||||
private getOrCreateAgent(sessionId: string): AgentOrchestrator {
|
||||
let agent = this.agents.get(sessionId);
|
||||
if (!agent) {
|
||||
const session = this.config.sessionManager.getSession('ws', sessionId);
|
||||
const config = this.config.config;
|
||||
|
||||
// Read per-session tier override from session config
|
||||
const sessionTier = session.getConfig?.('modelTier') as ModelTier | undefined;
|
||||
const primaryTier = sessionTier ?? config?.agents.primary_tier ?? 'default';
|
||||
|
||||
const delegationConfig: DelegationConfig = {
|
||||
compaction: config?.agents.delegation.compaction ?? 'fast',
|
||||
memory_extraction: config?.agents.delegation.memory_extraction ?? 'fast',
|
||||
classification: config?.agents.delegation.classification ?? 'fast',
|
||||
tool_summarisation: config?.agents.delegation.tool_summarisation ?? 'fast',
|
||||
complex_reasoning: config?.agents.delegation.complex_reasoning ?? 'complex',
|
||||
};
|
||||
|
||||
agent = new AgentOrchestrator({
|
||||
modelRouter: this.config.modelClient as ModelRouter,
|
||||
systemPrompt: this.config.systemPrompt,
|
||||
session,
|
||||
toolRegistry: this.config.toolRegistry,
|
||||
toolExecutor: this.config.toolExecutor,
|
||||
primaryTier,
|
||||
delegation: delegationConfig,
|
||||
maxDelegationDepth: config?.agents.max_delegation_depth ?? 3,
|
||||
maxIterations: config?.agents.max_iterations,
|
||||
compaction: config?.compaction.enabled ? {
|
||||
thresholdPct: config.compaction.threshold_pct,
|
||||
keepTurns: config.compaction.keep_turns,
|
||||
summaryMaxTokens: config.compaction.summary_max_tokens,
|
||||
} : undefined,
|
||||
modelName: config?.models.default.model,
|
||||
contextWindow: config?.models.default.context_window,
|
||||
memoryStore: this.config.memoryStore,
|
||||
});
|
||||
|
||||
this.agents.set(sessionId, agent);
|
||||
}
|
||||
return agent;
|
||||
}
|
||||
```
|
||||
|
||||
Update the `getAgent()` method signature and callers — they now get `AgentOrchestrator` which has the same `process()`, `reset()`, `setModelTier()`, `setOnToolUse()` methods.
|
||||
|
||||
Update `getAllUsage()` to use `AgentOrchestrator.getUsage()` which returns the richer `UsageReport` format:
|
||||
|
||||
```ts
|
||||
getAllUsage(): Array<{
|
||||
sessionId: string;
|
||||
primary: { inputTokens: number; outputTokens: number; calls: number };
|
||||
delegation: Record<string, { inputTokens: number; outputTokens: number; calls: number }>;
|
||||
total: { inputTokens: number; outputTokens: number; calls: number; estimatedCost: number };
|
||||
}> {
|
||||
const results = [];
|
||||
const seen = new Set<string>();
|
||||
for (const client of this.clients.values()) {
|
||||
if (seen.has(client.sessionId)) continue;
|
||||
seen.add(client.sessionId);
|
||||
const usage = client.agent.getUsage();
|
||||
results.push({ sessionId: client.sessionId, ...usage });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 6: WebChat Agent Handler — Add `/model` Command
|
||||
|
||||
**Files**: `src/gateway/handlers/agent.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Add `/model` command handling in `agent.send`
|
||||
- [ ] Persist model tier change to session config
|
||||
|
||||
**Details**:
|
||||
|
||||
In the command handling section (around line 47), add model command support:
|
||||
|
||||
```ts
|
||||
if (params.metadata?.isCommand) {
|
||||
try {
|
||||
if (params.metadata.command === 'reset') {
|
||||
agent.reset();
|
||||
// Clear session config
|
||||
const sessionId = deps.sessionBridge.getSessionId(connectionId);
|
||||
if (sessionId) {
|
||||
const session = deps.sessionManager.getSession('ws', sessionId);
|
||||
session.deleteConfig('modelTier');
|
||||
}
|
||||
send(makeEvent(request.id, 'done', { content: 'Session reset.' }));
|
||||
return;
|
||||
}
|
||||
|
||||
if (params.metadata.command === 'model') {
|
||||
const modelArg = params.metadata.commandArgs as string | undefined;
|
||||
const sessionId = deps.sessionBridge.getSessionId(connectionId);
|
||||
|
||||
if (!modelArg) {
|
||||
// Show current tier info
|
||||
const currentTier = agent.getModelTier();
|
||||
send(makeEvent(request.id, 'done', {
|
||||
content: `Current model tier: ${currentTier}`,
|
||||
}));
|
||||
return;
|
||||
}
|
||||
|
||||
// Validate tier
|
||||
const validTiers = ['fast', 'default', 'complex', 'local'];
|
||||
const tier = modelArg as ModelTier;
|
||||
if (!validTiers.includes(tier)) {
|
||||
send(makeEvent(request.id, 'done', {
|
||||
content: `Invalid tier: ${modelArg}. Available: ${validTiers.join(', ')}`,
|
||||
}));
|
||||
return;
|
||||
}
|
||||
|
||||
// Update agent tier
|
||||
agent.setModelTier(tier);
|
||||
|
||||
// Persist to session config
|
||||
if (sessionId) {
|
||||
const session = deps.sessionManager.getSession('ws', sessionId);
|
||||
session.setConfig('modelTier', tier);
|
||||
}
|
||||
|
||||
send(makeEvent(request.id, 'done', {
|
||||
content: `Switched to model tier: ${tier}`,
|
||||
}));
|
||||
return;
|
||||
}
|
||||
} finally {
|
||||
deps.sessionBridge.setBusy(connectionId, false);
|
||||
deps.metrics?.endRequest(requestId);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This requires updating `AgentHandlerDeps` to include `sessionManager`:
|
||||
|
||||
```ts
|
||||
export interface AgentHandlerDeps {
|
||||
sessionBridge: SessionBridge;
|
||||
laneQueue: LaneQueue;
|
||||
metrics?: MetricsCollector;
|
||||
sessionManager?: SessionManager;
|
||||
}
|
||||
```
|
||||
|
||||
And updating the handler registration in `server.ts` (line 126):
|
||||
|
||||
```ts
|
||||
const agentHandlers = createAgentHandlers({
|
||||
sessionBridge: this.sessionBridge,
|
||||
laneQueue: this.laneQueue,
|
||||
metrics: this.metrics,
|
||||
sessionManager: this.config.sessionManager,
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 7: Telegram Adapter — Add `/model` Command
|
||||
|
||||
**Files**: `src/channels/telegram/adapter.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Add `/model` command handler that dispatches through the message handler
|
||||
|
||||
**Details**:
|
||||
|
||||
After the `/reset` command handler (line 150), add:
|
||||
|
||||
```ts
|
||||
this.bot.command('model', async (ctx) => {
|
||||
if (!this.messageHandler) return;
|
||||
|
||||
const args = ctx.message?.text?.replace(/^\/model\s*/, '').trim() ?? '';
|
||||
|
||||
this.messageHandler({
|
||||
id: String(ctx.message?.message_id ?? Date.now()),
|
||||
channel: 'telegram',
|
||||
senderId: String(ctx.chat.id),
|
||||
senderName: ctx.from?.first_name,
|
||||
text: `/model ${args}`.trim(),
|
||||
timestamp: Date.now(),
|
||||
metadata: {
|
||||
isCommand: true,
|
||||
command: 'model',
|
||||
commandArgs: args || undefined,
|
||||
},
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Also add `/local` and `/cloud` as convenience aliases:
|
||||
|
||||
```ts
|
||||
this.bot.command('local', async (ctx) => {
|
||||
if (!this.messageHandler) return;
|
||||
this.messageHandler({
|
||||
id: String(ctx.message?.message_id ?? Date.now()),
|
||||
channel: 'telegram',
|
||||
senderId: String(ctx.chat.id),
|
||||
senderName: ctx.from?.first_name,
|
||||
text: '/model local',
|
||||
timestamp: Date.now(),
|
||||
metadata: { isCommand: true, command: 'model', commandArgs: 'local' },
|
||||
});
|
||||
});
|
||||
|
||||
this.bot.command('cloud', async (ctx) => {
|
||||
if (!this.messageHandler) return;
|
||||
this.messageHandler({
|
||||
id: String(ctx.message?.message_id ?? Date.now()),
|
||||
channel: 'telegram',
|
||||
senderId: String(ctx.chat.id),
|
||||
senderName: ctx.from?.first_name,
|
||||
text: '/model default',
|
||||
timestamp: Date.now(),
|
||||
metadata: { isCommand: true, command: 'model', commandArgs: 'default' },
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Step 8: Update Daemon `index.ts` — Stop Global Tier Sync
|
||||
|
||||
**Files**: `src/daemon/index.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Keep `modelRouter.setOnTierChange` for persisting the global default, but document that this is the GLOBAL default only
|
||||
- [ ] Remove `modelRouter.setOnTierChange` from triggering session-level changes
|
||||
|
||||
**Details**:
|
||||
|
||||
The current code (lines 104-110) is fine for the GLOBAL default:
|
||||
|
||||
```ts
|
||||
// Restore persisted global model tier default
|
||||
if (prefs.modelTier) {
|
||||
modelRouter.setTier(prefs.modelTier as ModelTier);
|
||||
}
|
||||
// Persist global default when changed via TUI
|
||||
modelRouter.setOnTierChange((tier) => savePreference(dataDir, 'modelTier', tier));
|
||||
```
|
||||
|
||||
This is correct — TUI `/model` changes the global default. The SessionBridge no longer syncs to this (per Step 5), so WebChat sessions are unaffected. Channel adapter sessions use their own session config.
|
||||
|
||||
No changes needed here, but the SessionBridge change in Step 5 is what decouples WebChat from this global sync.
|
||||
|
||||
---
|
||||
|
||||
### Step 9: Pass Additional Config to SessionBridge
|
||||
|
||||
**Files**: `src/daemon/services.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Pass `config` and `memoryStore` to `GatewayServer` → `SessionBridge`
|
||||
|
||||
**Details**:
|
||||
|
||||
In `createGateway()` (line 126), the `GatewayServer` constructor already receives `config`. We need to also pass `memoryStore` if available.
|
||||
|
||||
Update `GatewayDeps` in `services.ts`:
|
||||
|
||||
```ts
|
||||
export interface GatewayDeps {
|
||||
// ... existing fields ...
|
||||
memoryStore?: MemoryStore;
|
||||
}
|
||||
```
|
||||
|
||||
Update `createGateway()` to pass through:
|
||||
|
||||
```ts
|
||||
const gateway = new GatewayServer({
|
||||
// ... existing config ...
|
||||
memoryStore: deps.memoryStore,
|
||||
});
|
||||
```
|
||||
|
||||
Update `GatewayServerConfig` in `server.ts`:
|
||||
|
||||
```ts
|
||||
export interface GatewayServerConfig {
|
||||
// ... existing fields ...
|
||||
memoryStore?: MemoryStore;
|
||||
}
|
||||
```
|
||||
|
||||
And pass it to `SessionBridge` in `GatewayServer` constructor:
|
||||
|
||||
```ts
|
||||
this.sessionBridge = new SessionBridge({
|
||||
sessionManager: config.sessionManager,
|
||||
modelClient: config.modelClient,
|
||||
systemPrompt: config.systemPrompt,
|
||||
toolRegistry: config.toolRegistry,
|
||||
toolExecutor: config.toolExecutor,
|
||||
config: config.config,
|
||||
memoryStore: config.memoryStore,
|
||||
});
|
||||
```
|
||||
|
||||
Update `startDaemon()` in `daemon/index.ts` to pass `memoryStore`:
|
||||
|
||||
```ts
|
||||
const gateway = createGateway({
|
||||
config, sessionManager, modelRouter, systemPrompt, toolRegistry, toolExecutor,
|
||||
channelRegistry, pairingManager, lifecycle,
|
||||
getChannelAgents: () => channelAgents,
|
||||
memoryStore, // ADD THIS
|
||||
});
|
||||
```
|
||||
|
||||
This requires extracting `memoryStore` from `initMemory()` result (it's already destructured on line 97).
|
||||
|
||||
---
|
||||
|
||||
### Step 10: Update `InboundMessage` metadata Type
|
||||
|
||||
**Files**: `src/channels/types.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Add `commandArgs` to the metadata type
|
||||
|
||||
**Details**:
|
||||
|
||||
Check if `metadata` is typed or `Record<string, unknown>`. If typed, add:
|
||||
|
||||
```ts
|
||||
metadata?: {
|
||||
isCommand?: boolean;
|
||||
command?: string;
|
||||
commandArgs?: string;
|
||||
// ... other existing fields
|
||||
[key: string]: unknown;
|
||||
};
|
||||
```
|
||||
|
||||
If it's already `Record<string, unknown>`, no change needed (the field is dynamically accessed).
|
||||
|
||||
---
|
||||
|
||||
### Step 11: Gateway `agent.send` Handler — Update for AgentOrchestrator
|
||||
|
||||
**Files**: `src/gateway/handlers/agent.ts`
|
||||
|
||||
**Changes**:
|
||||
- [ ] Update `agent.send` to work with `AgentOrchestrator` instead of `NativeAgent`
|
||||
- [ ] The API is the same: `process()`, `reset()`, `setOnToolUse()` all exist on both
|
||||
|
||||
**Details**:
|
||||
|
||||
Since `AgentOrchestrator` exposes the same public API as `NativeAgent` for the operations used in `agent.ts`, the handler code largely works unchanged. The `getAgent()` method on `SessionBridge` now returns `AgentOrchestrator` instead of `NativeAgent`.
|
||||
|
||||
Key method compatibility check:
|
||||
- `agent.process(message, attachments)` ✅ (exists on both)
|
||||
- `agent.reset()` ✅ (exists on both)
|
||||
- `agent.setOnToolUse(callback)` ✅ (exists on both)
|
||||
- `agent.getUsage()` — returns `UsageReport` on orchestrator vs `{inputTokens, outputTokens, calls}` on NativeAgent. The handler at line 96 just calls `agent.process()`, so no usage call here.
|
||||
- `agent.getModelTier()` ✅ (exists on both)
|
||||
- `agent.setModelTier(tier)` ✅ (exists on both)
|
||||
|
||||
The `ToolUseEvent` type is the same for both.
|
||||
|
||||
---
|
||||
|
||||
## File Change Summary
|
||||
|
||||
| File | Change Type | Description |
|
||||
|------|------------|-------------|
|
||||
| `src/session/store.ts` | Modified | Add `session_config` table + CRUD methods |
|
||||
| `src/session/manager.ts` | Modified | Add config methods to Session interface, ManagedSession, SessionManager |
|
||||
| `src/daemon/routing.ts` | Modified | Session config resolution, `/model` command handler |
|
||||
| `src/gateway/session-bridge.ts` | Modified | Replace NativeAgent with AgentOrchestrator, remove global tier sync |
|
||||
| `src/gateway/handlers/agent.ts` | Modified | Add `/model` + `/reset` config clearing |
|
||||
| `src/gateway/server.ts` | Modified | Pass sessionManager to agent handlers, memoryStore to SessionBridge |
|
||||
| `src/channels/telegram/adapter.ts` | Modified | Add `/model`, `/local`, `/cloud` commands |
|
||||
| `src/daemon/services.ts` | Modified | Pass memoryStore through to gateway |
|
||||
| `src/daemon/index.ts` | Modified | Pass memoryStore to createGateway |
|
||||
| `src/channels/types.ts` | Modified | Add `commandArgs` to metadata (if typed) |
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- [ ] **`src/session/store.test.ts`**: Test `session_config` CRUD — set, get, getAll, delete, clearAll
|
||||
- [ ] **`src/session/store.test.ts`**: Test `clearSession()` also clears config
|
||||
- [ ] **`src/session/store.test.ts`**: Test `pruneStale()` also cleans config
|
||||
- [ ] **`src/session/manager.test.ts`**: Test `ManagedSession.getConfig/setConfig/deleteConfig`
|
||||
- [ ] **`src/daemon/routing.test.ts`**: Test model resolution chain (session → agent → global)
|
||||
- [ ] **`src/daemon/routing.test.ts`**: Test `/model` command sets session config and updates agent tier
|
||||
- [ ] **`src/daemon/routing.test.ts`**: Test `/reset` command clears session config
|
||||
- [ ] **`src/gateway/session-bridge.test.ts`**: Test that tier changes in TUI do NOT affect WebChat sessions
|
||||
- [ ] **`src/gateway/session-bridge.test.ts`**: Test that new agents load tier from session config
|
||||
- [ ] **`src/gateway/handlers/handlers.test.ts`**: Test `agent.send` with model command
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- [ ] Start daemon, set model in TUI, verify WebChat session stays on its own model
|
||||
- [ ] Set model via Telegram `/model fast`, restart daemon, verify Telegram session restores `fast`
|
||||
- [ ] Set model via WebChat, reconnect WebSocket, verify session restores the tier
|
||||
- [ ] `/reset` in any channel clears the model override back to global default
|
||||
|
||||
### Manual Tests
|
||||
|
||||
- [ ] TUI: `/model fast` → check TUI uses fast, WebChat still uses default
|
||||
- [ ] Telegram: `/model local` → check Telegram uses local, TUI unaffected
|
||||
- [ ] WebChat: send `{ "method": "agent.send", "params": { "message": "/model fast", "metadata": { "isCommand": true, "command": "model", "commandArgs": "fast" } } }` → verify tier switch
|
||||
- [ ] Restart daemon → verify all channel sessions restore their per-session model tier
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| WebChat upgrade to AgentOrchestrator may change behavior | AgentOrchestrator wraps NativeAgent with identical public API; test thoroughly |
|
||||
| Cached agents in routing.ts may get stale tier | `setModelTier()` updates in place, no cache invalidation needed |
|
||||
| Session config table migration on existing DBs | `CREATE TABLE IF NOT EXISTS` is safe; no migration needed |
|
||||
| Breaking changes to gateway protocol | No protocol changes — uses existing `metadata.isCommand` pattern |
|
||||
| Memory/performance impact of per-session config | Minimal — single row per session per config key; read on agent creation only |
|
||||
| Existing `preferences.json` global default conflicts with session override | Clear precedence chain: session → agent config → global. Documented above |
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Should `/model` in TUI also persist as a session override for the TUI session?** Currently TUI `/model` changes the global `ModelRouter` tier. Should it additionally write to the TUI session config? Recommendation: No — TUI's model switch is intentionally global (affects `preferences.json`). Per-session overrides are for remote channels.
|
||||
|
||||
2. **Should the WebChat dashboard show the per-session model tier?** The dashboard currently shows token usage per session. Showing the active model tier per session would be a nice UX enhancement but is out of scope for this plan.
|
||||
|
||||
3. **Should we expose a `sessions.config.get` / `sessions.config.set` gateway RPC method?** This would allow the WebChat UI to query/set arbitrary session config. Recommendation: Defer to a follow-up — the `/model` command via `agent.send` is sufficient for now.
|
||||
Reference in New Issue
Block a user