William Valentin
84 lines
2.2 KiB
Markdown
84 lines
2.2 KiB
Markdown
# Phase 2 PR #2 Checklist: Confidence-Based Routing
|
|
|
|
Created: 2026-02-12
|
|
Owner: Flynn core
|
|
Status: ready to implement
|
|
|
|
## Goal
|
|
|
|
Introduce confidence-based routing policy to choose between fast-path target execution and full LLM orchestration.
|
|
|
|
## Scope
|
|
|
|
In scope:
|
|
- routing policy schema with thresholds
|
|
- confidence output from intent matcher
|
|
- policy decision engine (`fast` vs `llm`)
|
|
- routing integration and gateway debug handler
|
|
|
|
Out of scope:
|
|
- history/topic boost (PR #3)
|
|
- command registry changes (already in Phase 1)
|
|
|
|
## Files
|
|
|
|
New:
|
|
- `src/routing/policy.ts`
|
|
- `src/routing/index.ts`
|
|
- `src/routing/policy.test.ts`
|
|
- `src/gateway/handlers/routing.ts`
|
|
|
|
Modified:
|
|
- `src/config/schema.ts`
|
|
- `src/intents/registry.ts`
|
|
- `src/intents/registry.test.ts`
|
|
- `src/daemon/index.ts`
|
|
- `src/daemon/routing.ts`
|
|
- `src/gateway/handlers/index.ts`
|
|
|
|
## Implementation Steps
|
|
|
|
1. Add `routing_policy` config:
|
|
- `enabled`
|
|
- `fast_path_threshold`
|
|
- `llm_threshold`
|
|
- `default_path` (`fast|llm`)
|
|
2. Extend intent matcher to return confidence score.
|
|
3. Implement `RoutingPolicy.decide(...)` with deterministic threshold logic.
|
|
4. Apply policy in `daemon/routing.ts` before `agent.process`:
|
|
- high confidence -> fast path
|
|
- medium/low -> standard orchestration
|
|
5. Add `routing.decide` gateway handler for inspection/testing.
|
|
6. Add unit/integration tests for threshold boundaries and fallback.
|
|
|
|
## Validation Commands
|
|
|
|
```bash
|
|
pnpm typecheck
|
|
pnpm test:run src/routing/policy.test.ts
|
|
pnpm test:run src/intents/registry.test.ts
|
|
pnpm test:run src/daemon/routing.test.ts
|
|
pnpm test:run
|
|
pnpm lint
|
|
pnpm build
|
|
```
|
|
|
|
## Acceptance Criteria
|
|
|
|
- Policy decisions are deterministic at threshold edges.
|
|
- Fast-path is only used when confidence meets configured threshold.
|
|
- Unmatched/low-confidence requests always fall back safely.
|
|
- Existing command and routing flows remain compatible.
|
|
- Routing decision logging is visible for debugging.
|
|
|
|
## Risks
|
|
|
|
- Over-aggressive fast-path causes loss of context.
|
|
- Mitigation: high default threshold and explicit fallback path.
|
|
- Confidence scoring instability.
|
|
- Mitigation: test edge cases and keep scoring simple.
|
|
|
|
## Commit Message
|
|
|
|
`feat(routing): add confidence-based fast-path policy`
|