Add usage tracking design (fc-005)
Design for /usage command to track model usage: - Parse existing history files (no new logging infrastructure) - Configurable log levels: minimal, standard, detailed - Commands: /usage [today|week|month|all] --by [agent|skill|model] - Token estimation and delegation tracking Also added future considerations: - fc-036: API token billing support - fc-037: Automatic usage summary - fc-038: Usage dashboard file 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
195
plans/2026-01-01-usage-tracking-design.md
Normal file
195
plans/2026-01-01-usage-tracking-design.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Model Usage & Cost Tracking Design
|
||||
|
||||
**Date:** 2026-01-01
|
||||
**Status:** Approved
|
||||
**FC Reference:** fc-005
|
||||
|
||||
## Overview
|
||||
|
||||
Build a `/usage` command to track and report model usage across sessions. Since we're on Claude Pro subscription ($20/month fixed), focus is on **usage optimization** rather than dollar tracking.
|
||||
|
||||
## Goals
|
||||
|
||||
1. **Cost visibility** — Understand usage patterns
|
||||
2. **Optimization insights** — Identify where cheaper models could be used
|
||||
3. **Usage patterns** — See which agents/skills are used most
|
||||
4. **Rate limit awareness** — Track against subscription limits
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
/usage command
|
||||
│
|
||||
▼
|
||||
Parse existing history files
|
||||
│
|
||||
├─→ history/index.json (session metadata)
|
||||
└─→ history/*.jsonl (session content)
|
||||
│
|
||||
▼
|
||||
Extract metrics:
|
||||
- Session times and duration
|
||||
- Commands/skills used
|
||||
- Agent mentions
|
||||
- Delegation calls
|
||||
- Token estimates
|
||||
│
|
||||
▼
|
||||
Aggregate and display report
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `~/.claude/commands/usage.md` | Slash command definition |
|
||||
| `~/.claude/skills/usage/SKILL.md` | Query logic and formatting |
|
||||
| `~/.claude/state/usage/config.json` | Log level + preferences |
|
||||
|
||||
## Log Levels
|
||||
|
||||
Configurable via `/usage --set-level`:
|
||||
|
||||
| Level | Data Captured | Use Case |
|
||||
|-------|---------------|----------|
|
||||
| `minimal` | Session ID, time, model | Low overhead |
|
||||
| `standard` | + agents, skills, tokens | Balanced (default) |
|
||||
| `detailed` | + commands, delegations, errors | Full debugging |
|
||||
|
||||
## Session Data Format
|
||||
|
||||
Parsed from history, structured as:
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "s-20260101-143022-a1b2c3",
|
||||
"start_time": "2026-01-01T14:30:22-08:00",
|
||||
"end_time": "2026-01-01T15:45:10-08:00",
|
||||
"duration_mins": 75,
|
||||
"model": "opus",
|
||||
"log_level": "standard",
|
||||
|
||||
"agents": ["personal-assistant", "linux-sysadmin"],
|
||||
"skills": ["gmail", "gcal"],
|
||||
"commands": ["/gmail", "/gcal today", "/usage"],
|
||||
|
||||
"delegations": {
|
||||
"sonnet": 3,
|
||||
"haiku": 5
|
||||
},
|
||||
|
||||
"token_estimate": {
|
||||
"input": 45000,
|
||||
"output": 12000
|
||||
},
|
||||
|
||||
"errors": 0
|
||||
}
|
||||
```
|
||||
|
||||
### Field Availability by Log Level
|
||||
|
||||
| Field | Minimal | Standard | Detailed |
|
||||
|-------|---------|----------|----------|
|
||||
| session_id, times, model | Yes | Yes | Yes |
|
||||
| agents, skills | | Yes | Yes |
|
||||
| token_estimate | | Yes | Yes |
|
||||
| commands | | | Yes |
|
||||
| delegations | | | Yes |
|
||||
| errors | | | Yes |
|
||||
|
||||
## Command Interface
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```
|
||||
/usage # Summary (last 7 days)
|
||||
/usage today # Today's sessions
|
||||
/usage week # Last 7 days (default)
|
||||
/usage month # Last 30 days
|
||||
/usage all # All time
|
||||
```
|
||||
|
||||
### Grouping
|
||||
|
||||
```
|
||||
/usage --by agent # Group by agent
|
||||
/usage --by skill # Group by skill
|
||||
/usage --by model # Group by model tier
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
```
|
||||
/usage --set-level minimal|standard|detailed
|
||||
/usage --show-config
|
||||
```
|
||||
|
||||
## Example Output
|
||||
|
||||
```
|
||||
📊 Usage Summary — Last 7 Days
|
||||
|
||||
Sessions: 12
|
||||
Total time: 8h 32m
|
||||
Model: opus (primary)
|
||||
|
||||
┌─────────────┬──────────┬────────┐
|
||||
│ Agent │ Sessions │ Time │
|
||||
├─────────────┼──────────┼────────┤
|
||||
│ PA │ 12 │ 8h 32m │
|
||||
│ sysadmin │ 4 │ 1h 15m │
|
||||
│ k8s-orch │ 2 │ 45m │
|
||||
└─────────────┴──────────┴────────┘
|
||||
|
||||
Delegations: 23 total
|
||||
→ Sonnet: 15 (65%)
|
||||
→ Haiku: 8 (35%)
|
||||
|
||||
Skills: gmail (8), gcal (5), k8s-status (3)
|
||||
|
||||
Tokens (est): ~125K input, ~42K output
|
||||
```
|
||||
|
||||
## Detection Patterns
|
||||
|
||||
| Pattern | Detected As |
|
||||
|---------|-------------|
|
||||
| `/gmail`, `/gcal`, `/k8s:*` | Skill usage |
|
||||
| `linux-sysadmin`, `k8s-orchestrator` | Agent usage |
|
||||
| `gmail_delegate.py`, `gcal_delegate.py` | Delegations |
|
||||
| `--model sonnet`, `--model haiku` | Model tier |
|
||||
|
||||
## Token Estimation
|
||||
|
||||
- Rough estimate: `character_count / 4 ≈ tokens`
|
||||
- More accurate if Claude CLI provides usage metadata
|
||||
|
||||
## Config File
|
||||
|
||||
`~/.claude/state/usage/config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"log_level": "standard",
|
||||
"default_range": "week",
|
||||
"track_tokens": true
|
||||
}
|
||||
```
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
| FC | Title | Status |
|
||||
|----|-------|--------|
|
||||
| fc-036 | API token billing support | Deferred |
|
||||
| fc-037 | Automatic usage summary | Identified |
|
||||
| fc-038 | Usage dashboard file | Identified |
|
||||
|
||||
## Implementation Checklist
|
||||
|
||||
- [ ] Create `~/.claude/state/usage/config.json`
|
||||
- [ ] Create `~/.claude/skills/usage/SKILL.md`
|
||||
- [ ] Create `~/.claude/commands/usage.md`
|
||||
- [ ] Implement history parsing logic
|
||||
- [ ] Test with existing session data
|
||||
- [ ] Update fc-005 status to resolved
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user