flynn/docs/plans/phase2-pr2-confidence-routing-checklist.md

Phase 2 PR #2 Checklist: Confidence-Based Routing

Created: 2026-02-12 Owner: Flynn core Status: ready to implement

Goal

Introduce confidence-based routing policy to choose between fast-path target execution and full LLM orchestration.

Scope

In scope:

• routing policy schema with thresholds
• confidence output from intent matcher
• policy decision engine (fast vs llm)
• routing integration and gateway debug handler

Out of scope:

• history/topic boost (PR #3)
• command registry changes (already in Phase 1)

Files

New:

• src/routing/policy.ts
• src/routing/index.ts
• src/routing/policy.test.ts
• src/gateway/handlers/routing.ts

Modified:

• src/config/schema.ts
• src/intents/registry.ts
• src/intents/registry.test.ts
• src/daemon/index.ts
• src/daemon/routing.ts
• src/gateway/handlers/index.ts

Implementation Steps

1. Add routing_policy config:
   • enabled
   • fast_path_threshold
   • llm_threshold
   • default_path (fast|llm)
2. Extend intent matcher to return confidence score.
3. Implement RoutingPolicy.decide(...) with deterministic threshold logic.
4. Apply policy in daemon/routing.ts before agent.process:
   • high confidence -> fast path
   • medium/low -> standard orchestration
5. Add routing.decide gateway handler for inspection/testing.
6. Add unit/integration tests for threshold boundaries and fallback.

Validation Commands

pnpm typecheck
pnpm test:run src/routing/policy.test.ts
pnpm test:run src/intents/registry.test.ts
pnpm test:run src/daemon/routing.test.ts
pnpm test:run
pnpm lint
pnpm build

Acceptance Criteria

• Policy decisions are deterministic at threshold edges.
• Fast-path is only used when confidence meets configured threshold.
• Unmatched/low-confidence requests always fall back safely.
• Existing command and routing flows remain compatible.
• Routing decision logging is visible for debugging.

Risks

• Over-aggressive fast-path causes loss of context.
  • Mitigation: high default threshold and explicit fallback path.
• Confidence scoring instability.
  • Mitigation: test edge cases and keep scoring simple.

Commit Message

feat(routing): add confidence-based fast-path policy