will/flynn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(eval): enforce min-sample gates in canary artifacts and decision log

Browse Source
This commit is contained in:
William Valentin
2026-02-23 22:34:54 -08:00
parent 34d1562ce8
commit 2d42f65b9f
7 changed files with 76 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/artifacts/pi_embedded_eval_window_a_2026-02-24.md
+5 -2
View File
@@ -32,8 +32,8 @@
| Category | Count | Percent |
| --- | ---: | ---: |
| loaded pi module does not expose a supported session factory (expected one of: c | 1 | 50.00% |
| pi agent runtime produced no assistant text | 1 | 50.00% |
| empty_assistant_text | 1 | 50.00% |
| pi_module_interface | 1 | 50.00% |
## Top Fallback Reasons
@@ -43,6 +43,9 @@
## Gate Evaluation
- Gate result: HOLD
- [x] Minimum target routes: actual=8, threshold=>= 8
- [x] Minimum baseline routes: actual=2, threshold=>= 2
- [x] Minimum target external attempts: actual=8, threshold=>= 8
- [x] Completion rate delta (target - baseline): actual=0.00pp, threshold=>= -2.00pp
- [ ] P50 latency delta (target - baseline): actual=259ms, threshold=<= 250ms
- [ ] P95 latency delta (target - baseline): actual=5695ms, threshold=<= 700ms