diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 35e7a07..11bebb6 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -40,6 +40,17 @@ Three phases, each delivering one complete capability. Phase 1 decomposes the mo **Requirements:** CONF-01, CONF-02, CONF-03 +**Plans:** 2 plans in 2 waves + +Plans: +- [ ] 02-01-PLAN.md — Core overlay merge (deepMerge + overlay-aware loadConfig + FLYNN_ENV resolution + tests) +- [ ] 02-02-PLAN.md — Doctor overlay validation (checkOverlayExists check) + +| Plan | Wave | Objective | Tasks | +|------|------|-----------|-------| +| 02-01 | 1 | Core overlay merge logic + FLYNN_ENV resolution + tests | 2 | +| 02-02 | 2 | Doctor overlay validation check | 1 | + **Success Criteria:** 1. Setting `FLYNN_ENV=docker` and starting Flynn loads config/docker.yaml merged on top of the base config — operator sees docker-specific settings active 2. Starting Flynn without FLYNN_ENV works exactly as before — existing config files unchanged, zero breakage @@ -64,11 +75,11 @@ Three phases, each delivering one complete capability. Phase 1 decomposes the mo | Phase | Status | Requirements | |-------|--------|--------------| | 1 — Daemon Decomposition | **complete** | DECO-01..08 (8) — 3 plans, 2 waves | -| 2 — Config Overlays | not_started | CONF-01..03 (3) | +| 2 — Config Overlays | **planned** | CONF-01..03 (3) — 2 plans, 2 waves | | 3 — Live Ops Dashboard | not_started | DASH-01..05 (5) | **Coverage:** 16/16 v1 requirements mapped ✓ --- *Roadmap created: 2026-02-09* -*Last updated: 2026-02-10* +*Last updated: 2026-02-09* diff --git a/.planning/phases/02-config-overlays/02-01-PLAN.md b/.planning/phases/02-config-overlays/02-01-PLAN.md new file mode 100644 index 0000000..52dac4c --- /dev/null +++ b/.planning/phases/02-config-overlays/02-01-PLAN.md @@ -0,0 +1,234 @@ +--- +phase: 02-config-overlays +plan: 01 +type: tdd +wave: 1 +depends_on: [] +files_modified: + - src/config/loader.ts + - src/config/loader.test.ts + - src/config/index.ts + - src/cli/shared.ts +autonomous: true + +must_haves: + truths: + - "Setting FLYNN_ENV=docker loads docker.yaml overlay merged on top of base config" + - "Starting Flynn without FLYNN_ENV works exactly as before — zero breakage" + - "Overlay values override base values at any nesting depth" + - "Overlay files do not need to repeat required base fields — merge happens before Zod validation" + - "Arrays in overlays replace base arrays (not concatenated)" + artifacts: + - path: "src/config/loader.ts" + provides: "deepMerge utility + overlay-aware loadConfig" + exports: ["loadConfig", "deepMerge"] + contains: "deepMerge" + - path: "src/config/loader.test.ts" + provides: "Tests for deep merge, overlay loading, env-only loading, missing overlay" + min_lines: 100 + - path: "src/cli/shared.ts" + provides: "resolveOverlayPath utility + overlay-aware loadConfigSafe" + exports: ["resolveOverlayPath"] + contains: "FLYNN_ENV" + - path: "src/config/index.ts" + provides: "Re-exports deepMerge" + contains: "deepMerge" + key_links: + - from: "src/cli/shared.ts" + to: "src/config/loader.ts" + via: "loadConfigSafe calls loadConfig with overlayPath" + pattern: "loadConfig\\(.*overlay" + - from: "src/config/loader.ts" + to: "src/config/loader.ts" + via: "loadConfig calls deepMerge before configSchema.parse" + pattern: "deepMerge.*parse|parse.*deepMerge" +--- + + +Implement config overlay merge system: deep merge utility, overlay-aware config loader, and FLYNN_ENV resolution. + +Purpose: Enable operators to manage environment-specific configs (dev/docker/production) with a base file plus lightweight overlay files, selected by FLYNN_ENV. + +Output: Working overlay merge in loader.ts, FLYNN_ENV resolution in shared.ts, comprehensive tests proving merge behavior and backward compatibility. + + + +@/home/will/.config/opencode/get-shit-done/workflows/execute-plan.md +@/home/will/.config/opencode/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md + +@src/config/loader.ts +@src/config/loader.test.ts +@src/config/schema.ts +@src/config/index.ts +@src/cli/shared.ts +@src/cli/start.ts + + + + + + Task 1: Implement deepMerge and overlay-aware loadConfig with tests + src/config/loader.ts, src/config/loader.test.ts, src/config/index.ts + +**RED phase — write failing tests first in `src/config/loader.test.ts`:** + +Add a new `describe('deepMerge')` block testing the exported `deepMerge` function: +1. "merges nested objects" — `deepMerge({a: {b: 1, c: 2}}, {a: {c: 3}})` → `{a: {b: 1, c: 3}}` +2. "overlay arrays replace base arrays" — `deepMerge({a: [1,2]}, {a: [3]})` → `{a: [3]}` +3. "adds new keys from overlay" — `deepMerge({a: 1}, {b: 2})` → `{a: 1, b: 2}` +4. "returns base when overlay is empty" — `deepMerge({a: 1}, {})` → `{a: 1}` +5. "handles deeply nested objects" — 3+ levels deep + +Add a new `describe('loadConfig with overlay')` block: +1. "merges overlay on top of base config" — create base config.yaml with full valid config (telegram + models), create overlay.yaml with only `server: { port: 9999 }`, call `loadConfig(basePath, overlayPath)`, assert `config.server.port === 9999` AND `config.telegram.bot_token` still has the base value. +2. "loads base-only when no overlay provided" — call `loadConfig(basePath)` (no second arg), assert it works identically to current behavior. This is the backward-compatibility test. +3. "expands env vars in both base and overlay" — set an env var, reference it in both base and overlay, verify both expand. +4. "throws when overlay path provided but file doesn't exist" — call `loadConfig(basePath, '/nonexistent/overlay.yaml')`, expect it to throw with a message containing the path. + +Run tests — they should FAIL (deepMerge and overlay parameter don't exist yet). + +**GREEN phase — implement in `src/config/loader.ts`:** + +Add `deepMerge` function (export it): +```typescript +export function deepMerge(base: Record, overlay: Record): Record { + const result = { ...base }; + for (const [key, value] of Object.entries(overlay)) { + if ( + value !== null && + typeof value === 'object' && + !Array.isArray(value) && + typeof result[key] === 'object' && + result[key] !== null && + !Array.isArray(result[key]) + ) { + result[key] = deepMerge(result[key] as Record, value as Record); + } else { + result[key] = value; + } + } + return result; +} +``` + +Key design decisions: +- Arrays REPLACE (not concat) — overlay `allowed_chat_ids: [999]` replaces base `[123]` +- `null` in overlay sets key to null (explicit nullification) +- Only plain objects recurse; arrays, primitives, and null are overwritten + +Modify `loadConfig` signature to accept optional overlay: +```typescript +export function loadConfig(configPath: string, overlayPath?: string): Config { + const rawContent = readFileSync(configPath, 'utf-8'); + let rawConfig = parse(rawContent); + + if (overlayPath) { + const overlayContent = readFileSync(overlayPath, 'utf-8'); + const overlayConfig = parse(overlayContent); + if (overlayConfig && typeof overlayConfig === 'object') { + rawConfig = deepMerge(rawConfig as Record, overlayConfig as Record); + } + } + + const expandedConfig = expandEnvVarsInObject(rawConfig); + return configSchema.parse(expandedConfig); +} +``` + +IMPORTANT: The merge happens BEFORE env var expansion and BEFORE Zod validation. This means: +- Overlay files don't need required fields (telegram, models) — those come from base +- Env vars in overlay get expanded too +- The final merged+expanded object is what Zod validates + +Update `src/config/index.ts` to re-export `deepMerge`: +```typescript +export { loadConfig, deepMerge } from './loader.js'; +``` + +Run `pnpm test:run src/config/loader.test.ts` — all tests should PASS. + + Run `pnpm test:run src/config/loader.test.ts` — all existing tests still pass, all new deepMerge and overlay tests pass. + deepMerge is exported and tested with 5+ cases. loadConfig accepts optional overlayPath, merges before validation. Backward compatibility confirmed (no overlay = same behavior). Existing 3 tests still pass. + + + + Task 2: Wire FLYNN_ENV resolution into shared.ts and update loadConfigSafe + src/cli/shared.ts + +Add `resolveOverlayPath` function to `src/cli/shared.ts`: + +```typescript +import { dirname, join } from 'path'; +import { existsSync } from 'fs'; + +/** + * Resolve overlay config path from FLYNN_ENV. + * If FLYNN_ENV is set, looks for {configDir}/{FLYNN_ENV}.yaml relative to the base config file. + * Returns undefined if FLYNN_ENV is not set. + * Returns the overlay path if FLYNN_ENV is set (even if file doesn't exist — caller decides error handling). + */ +export function resolveOverlayPath(basePath: string): string | undefined { + const env = process.env.FLYNN_ENV; + if (!env) return undefined; + const configDir = dirname(basePath); + return join(configDir, `${env}.yaml`); +} +``` + +Note: This function does NOT check if the file exists. Different callers handle missing overlays differently: +- `loadConfigSafe` / `loadConfig` → throw (file was expected) +- `doctor` → report as check failure (diagnostic, not crash) + +Update `loadConfigSafe` to pass overlay path to `loadConfig`: + +```typescript +export function loadConfigSafe(configPath?: string): { config?: Config; error?: string } { + const path = configPath ?? getConfigPath(); + try { + const overlayPath = resolveOverlayPath(path); + const config = loadConfig(path, overlayPath); + return { config }; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + return { error: `Failed to load config from ${path}: ${message}` }; + } +} +``` + +This means ALL CLI commands that use `loadConfigSafe()` (start, doctor, etc.) automatically get overlay support. The daemon itself (`startDaemon(config)`) receives the already-merged Config — no changes needed downstream. + +Update the import at the top of shared.ts to include `dirname` and `join` from `'path'` (resolve is already imported — add dirname and join). Note: `existsSync` is NOT needed in shared.ts since resolveOverlayPath doesn't check existence. + +Run the full test suite: `pnpm test:run` to verify nothing is broken. + + Run `pnpm test:run` — full suite passes (1077+ tests). No regressions. + resolveOverlayPath exported from shared.ts. loadConfigSafe passes overlay path to loadConfig. All CLI commands using loadConfigSafe() automatically get overlay support. Full test suite passes. + + + + + +1. `pnpm test:run src/config/loader.test.ts` — all deepMerge and overlay tests pass +2. `pnpm test:run` — full suite passes (1077+ tests, no regressions) +3. `pnpm typecheck` — no type errors +4. Manual verification: the merge happens BEFORE Zod validation (overlay files don't need required fields) + + + +- deepMerge function exported from config/loader.ts with 5+ test cases +- loadConfig accepts optional overlayPath, merges before Zod parse +- resolveOverlayPath in shared.ts resolves FLYNN_ENV to overlay path +- loadConfigSafe passes overlay path through to loadConfig +- No FLYNN_ENV set = exact same behavior as before (backward compatible) +- Full test suite passes with zero regressions + + + +After completion, create `.planning/phases/02-config-overlays/02-01-SUMMARY.md` + diff --git a/.planning/phases/02-config-overlays/02-02-PLAN.md b/.planning/phases/02-config-overlays/02-02-PLAN.md new file mode 100644 index 0000000..1aa7f8c --- /dev/null +++ b/.planning/phases/02-config-overlays/02-02-PLAN.md @@ -0,0 +1,121 @@ +--- +phase: 02-config-overlays +plan: 02 +type: execute +wave: 2 +depends_on: ["02-01"] +files_modified: + - src/cli/doctor.ts +autonomous: true + +must_haves: + truths: + - "Running flynn doctor with FLYNN_ENV=staging (no staging.yaml) reports a clear error identifying the missing overlay file" + - "Running flynn doctor without FLYNN_ENV shows no overlay-related check (or passes silently)" + - "Running flynn doctor with FLYNN_ENV=docker and docker.yaml present shows overlay check passing" + artifacts: + - path: "src/cli/doctor.ts" + provides: "checkOverlayExists check for FLYNN_ENV overlay validation" + contains: "checkOverlay" + key_links: + - from: "src/cli/doctor.ts" + to: "src/cli/shared.ts" + via: "imports resolveOverlayPath to determine overlay path" + pattern: "resolveOverlayPath" +--- + + +Add overlay file validation to `flynn doctor` — when FLYNN_ENV is set, verify the corresponding overlay YAML file exists. + +Purpose: Give operators clear diagnostic feedback when their environment overlay is misconfigured, rather than a cryptic file-not-found error at startup. + +Output: New doctor check in the existing checks array. + + + +@/home/will/.config/opencode/get-shit-done/workflows/execute-plan.md +@/home/will/.config/opencode/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/02-config-overlays/02-01-SUMMARY.md + +@src/cli/doctor.ts +@src/cli/shared.ts + + + + + + Task 1: Add checkOverlayExists to doctor.ts + src/cli/doctor.ts + +Import `resolveOverlayPath` from `./shared.js` (add to existing import line). + +Create a new check function following the existing pattern: + +```typescript +const checkOverlayExists: Check = async (ctx) => { + const overlayPath = resolveOverlayPath(ctx.configPath); + if (!overlayPath) { + return { status: 'skip', label: 'Config overlay', detail: '(FLYNN_ENV not set)' }; + } + const env = process.env.FLYNN_ENV; + if (existsSync(overlayPath)) { + return { status: 'pass', label: 'Config overlay', detail: `(${env}.yaml found)` }; + } + return { status: 'fail', label: 'Config overlay', detail: `FLYNN_ENV=${env} but ${overlayPath} not found` }; +}; +``` + +Insert `checkOverlayExists` into the `allChecks` array — place it right after `checkConfigExists` and before `checkConfigParses`. This is the natural position: first check the base config exists, then check the overlay exists, then parse/validate the merged result. + +The updated array should be: +```typescript +const allChecks: Check[] = [ + checkConfigExists, + checkOverlayExists, // <-- NEW + checkConfigParses, + checkConfigValidates, + checkEnvVars, + checkDataDir, + checkSessionDb, + checkModelConnectivity, + checkTelegram, + checkMcpServers, + checkSkills, + checkTailscale, +]; +``` + +Note: `existsSync` is already imported at the top of doctor.ts. `resolveOverlayPath` needs to be added to the import from `./shared.js`. + +Run `pnpm test:run` to verify no regressions. Run `pnpm typecheck` for type safety. + + Run `pnpm typecheck` — no type errors. Run `pnpm test:run` — full suite passes (1077+ tests). + checkOverlayExists added to doctor.ts checks array. When FLYNN_ENV is set, doctor reports pass/fail for overlay file existence with the overlay path in the detail. When FLYNN_ENV is not set, the check is skipped. Full test suite passes. + + + + + +1. `pnpm typecheck` — no type errors +2. `pnpm test:run` — full suite passes (1077+ tests) +3. Manual: `FLYNN_ENV=nonexistent pnpm start -- doctor` should show FAIL for Config overlay +4. Manual: `pnpm start -- doctor` (no FLYNN_ENV) should show SKIP for Config overlay + + + +- checkOverlayExists is in the doctor checks array after checkConfigExists +- FLYNN_ENV=staging with no staging.yaml → doctor reports FAIL with "FLYNN_ENV=staging but /path/to/staging.yaml not found" +- No FLYNN_ENV → doctor skips the overlay check +- FLYNN_ENV=docker with docker.yaml present → doctor reports PASS +- Full test suite passes with zero regressions + + + +After completion, create `.planning/phases/02-config-overlays/02-02-SUMMARY.md` +