William Valentin
112 lines
3.4 KiB
Markdown
112 lines
3.4 KiB
Markdown
# Gateway Sessions and Queueing (Agent Execution Model)
|
|
|
|
This document explains how the gateway maps WebSocket clients onto durable sessions, and how work is serialised per session so agent execution stays coherent under concurrent requests.
|
|
|
|
If you only want the protocol surface, see `docs/api/PROTOCOL.md`.
|
|
|
|
## Key Ideas
|
|
|
|
- A WebSocket client gets a `connectionId`.
|
|
- Each connection is attached to a `sessionId`.
|
|
- Agent work is queued per `sessionId` (FIFO), not per connection.
|
|
- Sessions persist in SQLite via `SessionManager` even if clients disconnect.
|
|
- Once dequeued, message routing may execute the native orchestrator path or an optional external backend path (`claude_code`, `opencode`, `codex`, `gemini`, `pi_embedded`) depending on agent/backend config.
|
|
- Runtime backend mode can be overridden manually via `/backend` command fast-path (`status`, `activate pi`, `deactivate pi`, `use config`) and is persisted in preferences.
|
|
- Backend routing outcomes are auditable via `backend.route` / `backend.success` / `backend.fallback`, which enables offline canary evaluation without changing gateway protocol methods.
|
|
|
|
## Component Map
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph CFG[Config + Runtime Policy]
|
|
QP[server.queue policy\nmode/cap/overflow/overrides]
|
|
BM[backend runtime mode\nconfig_default|force_native|force_pi_embedded]
|
|
end
|
|
|
|
subgraph GW[Gateway Process]
|
|
WS[WebSocket connection\n(connectionId)]
|
|
GS[GatewayServer]
|
|
LQ[LaneQueue\nper-session FIFO]
|
|
SB[SessionBridge\nconnectionId -> sessionId -> AgentOrchestrator]
|
|
AQ[AuditLogger\nqueue.preempt events]
|
|
end
|
|
|
|
subgraph CORE[Flynn Core]
|
|
SM[SessionManager\nin-memory cache + SQLite]
|
|
SS[SessionStore\nSQLite tables]
|
|
AO[AgentOrchestrator / External Backends]
|
|
end
|
|
|
|
WS --> GS
|
|
QP --> GS
|
|
BM --> GS
|
|
GS --> LQ
|
|
GS --> SB
|
|
LQ --> AQ
|
|
|
|
SB --> AO
|
|
SB --> SM
|
|
SM --> SS
|
|
```
|
|
|
|
## Session IDs (What Actually Gets Stored)
|
|
|
|
The durable session ID stored by `SessionManager` is:
|
|
|
|
`<frontend>:<userId>`
|
|
|
|
For the gateway:
|
|
|
|
- `SessionBridge.connect()` assigns a `connectionId` (UUID).
|
|
- It defaults the connection's `sessionId` to `ws:<connectionId>`.
|
|
- It then calls `SessionManager.getSession('ws', sessionId)`.
|
|
|
|
That means gateway sessions are stored as:
|
|
|
|
- `ws:ws:<connectionId>`
|
|
|
|
This is expected: the gateway adds its own namespace, and the session manager namespaces again by frontend.
|
|
|
|
Key files:
|
|
|
|
- `src/gateway/session-bridge.ts`
|
|
- `src/session/manager.ts`
|
|
|
|
## Per-Session FIFO Queueing (LaneQueue)
|
|
|
|
`agent.send` uses a lane ID derived from the session:
|
|
|
|
- lane = `SessionBridge.getSessionId(connectionId)` (preferred)
|
|
- fallback lane = `connectionId` (only if session lookup fails)
|
|
|
|
Within a lane:
|
|
|
|
- Only one request executes at a time.
|
|
- Later requests queue (FIFO) and start after the active request finishes.
|
|
|
|
Across lanes:
|
|
|
|
- Independent sessions run in parallel.
|
|
|
|
Key files:
|
|
|
|
- `src/gateway/lane-queue.ts`
|
|
- `src/gateway/handlers/agent.ts`
|
|
|
|
## Cancellation Semantics
|
|
|
|
`agent.cancel` performs two separate actions:
|
|
|
|
1. Cancels any queued (not-yet-started) work in the lane (`LaneQueue.cancel(laneId)`).
|
|
2. Requests cancellation of the active agent operation (`AgentOrchestrator.cancel()` via `SessionBridge.cancel()`).
|
|
|
|
Important:
|
|
|
|
- Cancellation is best-effort for the currently running work: it stops at the next safe point in the agent loop.
|
|
- Queued work is deterministically rejected.
|
|
|
|
Key files:
|
|
|
|
- `src/gateway/handlers/agent.ts`
|
|
- `src/backends/native/orchestrator.ts`
|