will/flynn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
flynn/docs/councils/ai-council-production-scaffold.md
T

8.8 KiB
Raw Permalink Blame History

AI Council Scaffold (Production-Ready)

Generated: 2026-02-21T18:05:48.200896

This package defines:

  • Two councils (Deterministic, Probabilistic) with 4 roles each (Arbiter, FreeThinker, Grounder, Writer)
  • A controlled “Bridge Packet” exchange between Arbiters only
  • A Meta-Merge stage for final selection
  • Standard message schemas and state objects suitable for orchestration (LangGraph, Temporal, custom event loop, etc.)

Design principle: protocol beats prompts. Most failures are coordination failures, not “model intelligence” failures.

1) Entities

Councils

  • Council D (Deterministic): consistency, clarity, feasibility, low drift
  • Council P (Probabilistic): novelty, exploration, reframing, controlled speculation

Roles (per council)

  • Arbiter (Lead): requests work, scores, selects, stops
  • Free Thinker: divergent generator
  • Grounder: convergent concretizer (not a skeptic)
  • Writer: ideation graph + versioned briefs

Meta Roles

  • Meta-Arbiter: merges v2 briefs and selects winners
  • Meta-Writer (optional): final pack, decision log, next experiments

2) Core Artifacts

2.1 Input Package (shared)

A single bundle sent to both councils at the start.

Required fields

  • Problem statement (1–3 paragraphs)
  • Success definition (what “good” looks like)
  • Constraints (budget/time/tech/legal/ethics)
  • Context (what’s been tried, known facts, prior art)
  • Timebox (round count and max output size)

Optional fields

  • Stakeholders & priorities
  • Non-goals
  • Risk tolerance (low/med/high)
  • Domain glossary

2.2 Idea Card (atomic unit)

All agents output in this format to keep merging clean.

Idea Card

  • id: D-03 or P-11
  • title: 3–8 words
  • one_liner: 1 sentence
  • mechanism: 2–6 bullets (why it might work)
  • assumptions: 2–6 bullets
  • constraints: 2–6 bullets (dependencies, limits)
  • mve: minimal viable experiment
    • steps: 3–7 bullets
    • success_metric: 1–2 sentences
    • time_cost: (e.g., “2 hours”, “1 week”)
    • resources: bullets
  • risks: 2–6 bullets
  • upside: what “winning” looks like
  • tags: list of strings (domain, novelty, cost, etc.)

2.3 Bridge Packet (Arbiter ↔ Arbiter only)

Single exchange. No debate. Bandwidth-capped.

  • top_ideas: 3 Idea Cards (full)
  • fragile_assumptions: 3 bullets
  • key_questions: 3 bullets
  • themes: 1–2 bullets
  • do_not_do: 1–2 bullets (things that hurt independence, premature convergence)

2.4 Brief (versioned)

Produced by each council Writer after each phase.

Brief vX

  • Summary (5–10 bullets)
  • Shortlist (3–5 ideas) with scores
  • Ideation graph delta (what changed since last version)
  • Open questions (prioritized)
  • Recommended next actions (MVEs)

2.5 Final Idea Brief Pack

Produced after Meta-Merge.

  • Primary bets (1–3)
  • Secondary bets (2–4)
  • MVEs for each
  • Assumptions to validate (ranked)
  • Decision log (why these won)
  • “Kill criteria” (what would make us drop an idea)
  • Next data to collect

3) Scoring Rubric (Arbiters & Meta-Arbiter)

Score 0–5 on:

  • novelty
  • feasibility
  • leverage (impact if true)
  • clarity (can we explain/execute it)
  • evidence_alignment (fits known facts)

Decision label:

  • advance
  • hold
  • drop

Convergence guidance:

  • If top-2 ideas are stable across rounds and marginal novelty decreases, signal convergence.
  • If D and P independently converge on similar cores, treat that as strong signal.

4) Workflow (Islands + Thin Bridge)

Phase 0 — Setup

  1. User provides Input Package.
  2. Writers create Context Snapshot v0 and empty ideation graph.

Phase 1 — Independent Ideation (no cross-council contact)

Per council:

  1. Arbiter requests Idea Cards from Free Thinker (target 10–20).
  2. Arbiter selects top 5 and requests grounding.
  3. Grounder expands top 5 with X/Y/Z + MVEs.
  4. Writer compiles Brief v1 + graph.

Phase 2 — Bridge (Arbiter ↔ Arbiter only)

  1. D-Arbiter sends Bridge Packet to P-Arbiter.
  2. P-Arbiter sends Bridge Packet to D-Arbiter. (No further discussion.)

Phase 3 — Second Pass (still mostly independent)

Per council:

  1. Arbiter asks Free Thinker for 3 variants inspired by received Bridge Packet.
  2. Arbiter asks Grounder to instrument the best 3 into clearer MVEs.
  3. Writer compiles Brief v2.

Phase 4 — Meta Merge

  1. Meta-Arbiter ingests both briefs.
  2. Selects final bets + experiments.
  3. Meta-Writer outputs Final Idea Brief Pack.

5) Interaction Protocol (message types)

Allowed message routes

  • Within a council: Arbiter ↔ (FreeThinker, Grounder, Writer)
  • Bridge: D-Arbiter ↔ P-Arbiter only
  • Meta: Meta-Arbiter reads both briefs; optional Meta-Writer

Hard rules

  • No cross-council messages except Bridge Packets.
  • No role switching inside an agent.
  • Writers do not “invent” new ideas; they only capture, structure, and diff.
  • Grounders do not veto; they convert into testable plans and surface constraints.

6) Prompts (Production Templates)

These are role instructions you pin as system prompts or role cards. Fill {INPUT_PACKAGE} at runtime.

6.1 D-Arbiter (Deterministic)

Role: You are the Arbiter for the Deterministic Council. Style: crisp, bounded, structured. Minimal speculation. Primary duties:

  • Request outputs in Idea Card format.
  • Score ideas with the rubric.
  • Select top candidates and request grounding.
  • Stop when diminishing returns or convergence.

Operating rules:

  • Do not generate more than 2 original ideas yourself.
  • Push for executable MVEs.
  • Enforce constraints strictly.

Start instructions:

  1. Ask D-FreeThinker for 10–15 Idea Cards based on {INPUT_PACKAGE}.
  2. After receiving, shortlist 5 and ask D-Grounder to expand them with X/Y/Z + MVE.
  3. Ask D-Writer to produce Brief v1.

6.2 P-Arbiter (Probabilistic)

Role: You are the Arbiter for the Probabilistic Council. Style: theme-hunting, novelty-protecting, but still structured. Primary duties:

  • Maximize novelty density early.
  • Require speculative claims to be labeled.
  • Prevent premature convergence in Phase 1.

Operating rules:

  • Do not terminate Phase 1 early unless redundancy is severe.
  • Enforce Idea Card format.
  • After Phase 1, move to grounding and testability.

6.3 D-Free Thinker

Role: Divergent generator under constraints. Output: 10–15 Idea Cards. Rules:

  • Keep ideas feasible within constraints.
  • Avoid vague visions; include mechanisms.
  • Keep each card short.

6.4 P-Free Thinker

Role: Divergent generator optimized for novelty. Output: 15–25 Idea Cards, then expand top 5 with richer mechanism. Rules:

  • Cross-domain analogies encouraged.
  • Mark uncertainty:
    • [SPECULATIVE] for leaps
    • [GROUNDED] for common/known patterns
  • Avoid pure fantasy: each idea must include a plausible mechanism.

6.5 D-Grounder

Role: Convergent concretizer. Input: 3–5 selected Idea Cards. Output: Updated cards with:

  • Preconditions (X/Y/Z)
  • Dependencies
  • MVE with success metric Rules:
  • Do not kill ideas.
  • If infeasible, propose a “cheap proxy” test.

6.6 P-Grounder

Role: Preserve weirdness; make it testable. Input: 3–5 selected Idea Cards. Output: MVEs that validate the “kernel” cheaply. Rules:

  • Keep the original spirit.
  • Convert big leaps into small experiments.

6.7 Writer (both councils)

Role: Scribe + graph curator. Live duties:

  • Track Idea Cards, scores, shortlist, assumptions, constraints.
  • Maintain ideation graph (nodes and edges).
  • Produce versioned briefs with deltas.

Rules:

  • No new ideas.
  • If conflicting facts appear, record them as “tension edges”.
  • Keep briefs short and merge-friendly.

6.8 Meta-Arbiter

Role: Final selector. Input: D Brief v2 + P Brief v2. Output: Final selection + rationale. Rules:

  • Apply rubric consistently.
  • Prefer portfolios: 1–3 primary bets + 2–4 secondary.
  • Specify MVEs and kill criteria.

7) Ideation Graph Model (minimal)

Nodes:

  • idea
  • assumption
  • constraint
  • experiment
  • evidence

Edges:

  • supports
  • depends_on
  • risks
  • tests
  • contradicts

Graph delta per version:

  • nodes_added, edges_added, nodes_updated, edges_updated

8) Quickstart Checklist

  1. Write Input Package.
  2. Run Phase 1 for Council D and Council P independently.
  3. Exchange Bridge Packets (Arbiter↔Arbiter only).
  4. Run Phase 3 short second pass.
  5. Meta-merge into Final Idea Brief Pack.

9) Optional Extensions (keep it lean)

  • Add “Historian” (cross-session memory) only if you run repeated projects.
  • Add “User Proxy” only if stakeholder conflicts are chronic.
  • Add “Devil’s Engineer” only after primary bets are chosen.
Reference in New Issue View Git Blame Copy Permalink