flynn/docs/plans/phase1-pr3-memory-structure-checklist.md

# Phase 1 PR #3 Checklist: Memory Category Structure

Created: 2026-02-12
Owner: Flynn core
Status: ready to implement

## Goal

Introduce structured memory categories while preserving current memory behavior.

Categories:
- `facts`
- `preferences`
- `decisions`
- `projects`

This PR is infrastructure-only: no new user-facing commands yet.

## PR Boundary

In scope:
- Add category types/constants/utilities
- Add category-aware `MemoryStore` APIs
- Keep existing `read/write/search/getContextForPrompt` behavior intact
- Add category-aware retrieval helpers for follow-on PRs
- Unit tests for compatibility and correctness

Out of scope:
- Auto-categorization/extraction heuristics
- New slash commands (`/memory add --category ...`)
- Migration CLI that rewrites existing files
- UI/dashboard for category browsing

## Compatibility Model

Existing flat namespaces remain valid:
- `user`
- `global`
- `sessions/<id>`

Category namespaces are additive and path-based:
- `user/facts`
- `user/preferences`
- `global/decisions`
- `sessions/<id>/projects`

No destructive migration in this PR.

## File-by-File Diff Plan

1) `src/memory/categories.ts` (new)

- Add category constants and helpers.

```ts
export const MEMORY_CATEGORIES = ['facts', 'preferences', 'decisions', 'projects'] as const;
export type MemoryCategory = (typeof MEMORY_CATEGORIES)[number];

export function isMemoryCategory(value: string): value is MemoryCategory {
  return (MEMORY_CATEGORIES as readonly string[]).includes(value);
}

export function categoryNamespace(baseNamespace: string, category: MemoryCategory): string {
  return `${baseNamespace}/${category}`;
}
```

2) `src/memory/store.ts` (modify)

- Add category-aware methods without changing existing API behavior.

New methods:
- `readCategory(baseNamespace: string, category: MemoryCategory): string`
- `writeCategory(baseNamespace: string, category: MemoryCategory, content: string, mode: 'append' | 'replace'): void`
- `listCategories(baseNamespace: string): MemoryCategory[]`
- `readAllCategories(baseNamespace: string): Partial<Record<MemoryCategory, string>>`
- `search(query: string, opts?: { categories?: MemoryCategory[]; baseNamespacePrefix?: string }): SearchResult[]`

Notes:
- Keep old `search(query: string)` call pattern working via optional `opts`.
- Ensure dirty namespace tracking marks category namespaces too.
- Keep `getContextForPrompt()` backward compatible: include legacy sections first, then category sections if present and token budget allows.

3) `src/memory/index.ts` (modify)

- Export category types/utilities.

```ts
export * from './categories.js';
```

4) `src/memory/categories.test.ts` (new)

- Validate category constants/helpers:
  - `isMemoryCategory` true/false paths
  - namespace composition correctness

5) `src/memory/store.test.ts` (modify/add)

- Add category coverage:
  - category read/write/append/replace
  - list/readAll categories
  - filtered search by category
  - backward compatibility for legacy namespaces
  - `getContextForPrompt()` includes legacy + category content under token budget

6) `src/tools/builtin/memory.ts` (modify docs-only or no-op code)

- If tool schemas/docs mention namespaces, document category namespace pattern so next PR can add command-level category args cleanly.

## Implementation Steps

1. Add `categories.ts` with constants/types/helpers.
2. Extend `MemoryStore` with category methods (additive only).
3. Extend `search` to support optional category filters.
4. Update prompt context composition to include categories safely.
5. Export new memory APIs from memory index barrel.
6. Add unit tests for category + compatibility behavior.
7. Run full validation.

## Validation Commands

```bash
pnpm typecheck
pnpm test:run src/memory/categories.test.ts
pnpm test:run src/memory/store.test.ts
pnpm test:run
pnpm lint
pnpm build
```

## Acceptance Criteria

- Existing memory paths (`user`, `global`, `sessions/<id>`) still work unchanged.
- Category APIs function for all four categories.
- Search works both with and without category filters.
- No data loss or rewrite required for existing memory files.
- `getContextForPrompt()` remains stable and token-bounded.
- Dirty namespace/indexing behavior remains correct for new category paths.
- All existing tests pass; new memory tests added.

## Quality Gates

- Backward compatibility:
  - Legacy reads/writes/searches unchanged.
  - Old memory files remain readable with zero migration.
- Retrieval correctness:
  - Category-filtered searches return only matching namespaces.
  - Unfiltered searches still span all namespaces.
- Performance:
  - No meaningful regression for unfiltered search/list operations.
  - Category filtering avoids unnecessary extra scans where possible.
- Reliability:
  - Missing category files return empty strings (not errors).
  - Invalid categories are rejected at type-level and guarded in runtime helper paths.

## Risks and Mitigations

- Risk: subtle prompt-context ordering changes.
  - Mitigation: preserve legacy section order, append category sections after legacy content.
- Risk: token budget overrun from extra sections.
  - Mitigation: enforce existing truncation logic after composing all sections.
- Risk: search API break due to signature change.
  - Mitigation: keep `opts` optional and maintain old call contract.

## Suggested Commit Message

`feat(memory): add structured category namespaces with backward-compatible APIs`

## Follow-up PRs

1. Add category-aware memory tool arguments and slash command UX.
2. Add auto-extraction into categories during compaction/memory pipeline.
3. Add migration/normalization utility for legacy memory into categories.