0e1e47d577
- Updated design document status to Implemented - Marked all checklist items complete - Updated fc-005 in future-considerations.json 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
196 lines
4.7 KiB
Markdown
196 lines
4.7 KiB
Markdown
# Model Usage & Cost Tracking Design
|
|
|
|
**Date:** 2026-01-01
|
|
**Status:** Implemented
|
|
**FC Reference:** fc-005 (resolved)
|
|
|
|
## Overview
|
|
|
|
Build a `/usage` command to track and report model usage across sessions. Since we're on Claude Pro subscription ($20/month fixed), focus is on **usage optimization** rather than dollar tracking.
|
|
|
|
## Goals
|
|
|
|
1. **Cost visibility** — Understand usage patterns
|
|
2. **Optimization insights** — Identify where cheaper models could be used
|
|
3. **Usage patterns** — See which agents/skills are used most
|
|
4. **Rate limit awareness** — Track against subscription limits
|
|
|
|
## Architecture
|
|
|
|
```
|
|
/usage command
|
|
│
|
|
▼
|
|
Parse existing history files
|
|
│
|
|
├─→ history/index.json (session metadata)
|
|
└─→ history/*.jsonl (session content)
|
|
│
|
|
▼
|
|
Extract metrics:
|
|
- Session times and duration
|
|
- Commands/skills used
|
|
- Agent mentions
|
|
- Delegation calls
|
|
- Token estimates
|
|
│
|
|
▼
|
|
Aggregate and display report
|
|
```
|
|
|
|
## Components
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `~/.claude/commands/usage.md` | Slash command definition |
|
|
| `~/.claude/skills/usage/SKILL.md` | Query logic and formatting |
|
|
| `~/.claude/state/usage/config.json` | Log level + preferences |
|
|
|
|
## Log Levels
|
|
|
|
Configurable via `/usage --set-level`:
|
|
|
|
| Level | Data Captured | Use Case |
|
|
|-------|---------------|----------|
|
|
| `minimal` | Session ID, time, model | Low overhead |
|
|
| `standard` | + agents, skills, tokens | Balanced (default) |
|
|
| `detailed` | + commands, delegations, errors | Full debugging |
|
|
|
|
## Session Data Format
|
|
|
|
Parsed from history, structured as:
|
|
|
|
```json
|
|
{
|
|
"session_id": "s-20260101-143022-a1b2c3",
|
|
"start_time": "2026-01-01T14:30:22-08:00",
|
|
"end_time": "2026-01-01T15:45:10-08:00",
|
|
"duration_mins": 75,
|
|
"model": "opus",
|
|
"log_level": "standard",
|
|
|
|
"agents": ["personal-assistant", "linux-sysadmin"],
|
|
"skills": ["gmail", "gcal"],
|
|
"commands": ["/gmail", "/gcal today", "/usage"],
|
|
|
|
"delegations": {
|
|
"sonnet": 3,
|
|
"haiku": 5
|
|
},
|
|
|
|
"token_estimate": {
|
|
"input": 45000,
|
|
"output": 12000
|
|
},
|
|
|
|
"errors": 0
|
|
}
|
|
```
|
|
|
|
### Field Availability by Log Level
|
|
|
|
| Field | Minimal | Standard | Detailed |
|
|
|-------|---------|----------|----------|
|
|
| session_id, times, model | Yes | Yes | Yes |
|
|
| agents, skills | | Yes | Yes |
|
|
| token_estimate | | Yes | Yes |
|
|
| commands | | | Yes |
|
|
| delegations | | | Yes |
|
|
| errors | | | Yes |
|
|
|
|
## Command Interface
|
|
|
|
### Basic Usage
|
|
|
|
```
|
|
/usage # Summary (last 7 days)
|
|
/usage today # Today's sessions
|
|
/usage week # Last 7 days (default)
|
|
/usage month # Last 30 days
|
|
/usage all # All time
|
|
```
|
|
|
|
### Grouping
|
|
|
|
```
|
|
/usage --by agent # Group by agent
|
|
/usage --by skill # Group by skill
|
|
/usage --by model # Group by model tier
|
|
```
|
|
|
|
### Configuration
|
|
|
|
```
|
|
/usage --set-level minimal|standard|detailed
|
|
/usage --show-config
|
|
```
|
|
|
|
## Example Output
|
|
|
|
```
|
|
📊 Usage Summary — Last 7 Days
|
|
|
|
Sessions: 12
|
|
Total time: 8h 32m
|
|
Model: opus (primary)
|
|
|
|
┌─────────────┬──────────┬────────┐
|
|
│ Agent │ Sessions │ Time │
|
|
├─────────────┼──────────┼────────┤
|
|
│ PA │ 12 │ 8h 32m │
|
|
│ sysadmin │ 4 │ 1h 15m │
|
|
│ k8s-orch │ 2 │ 45m │
|
|
└─────────────┴──────────┴────────┘
|
|
|
|
Delegations: 23 total
|
|
→ Sonnet: 15 (65%)
|
|
→ Haiku: 8 (35%)
|
|
|
|
Skills: gmail (8), gcal (5), k8s-status (3)
|
|
|
|
Tokens (est): ~125K input, ~42K output
|
|
```
|
|
|
|
## Detection Patterns
|
|
|
|
| Pattern | Detected As |
|
|
|---------|-------------|
|
|
| `/gmail`, `/gcal`, `/k8s:*` | Skill usage |
|
|
| `linux-sysadmin`, `k8s-orchestrator` | Agent usage |
|
|
| `gmail_delegate.py`, `gcal_delegate.py` | Delegations |
|
|
| `--model sonnet`, `--model haiku` | Model tier |
|
|
|
|
## Token Estimation
|
|
|
|
- Rough estimate: `character_count / 4 ≈ tokens`
|
|
- More accurate if Claude CLI provides usage metadata
|
|
|
|
## Config File
|
|
|
|
`~/.claude/state/usage/config.json`:
|
|
|
|
```json
|
|
{
|
|
"log_level": "standard",
|
|
"default_range": "week",
|
|
"track_tokens": true
|
|
}
|
|
```
|
|
|
|
## Future Enhancements
|
|
|
|
| FC | Title | Status |
|
|
|----|-------|--------|
|
|
| fc-036 | API token billing support | Deferred |
|
|
| fc-037 | Automatic usage summary | Identified |
|
|
| fc-038 | Usage dashboard file | Identified |
|
|
|
|
## Implementation Checklist
|
|
|
|
- [x] Create `~/.claude/state/usage/config.json`
|
|
- [x] Create `~/.claude/skills/usage/SKILL.md`
|
|
- [x] Create `~/.claude/commands/usage.md`
|
|
- [x] Implement history parsing logic
|
|
- [x] Test with existing session data
|
|
- [x] Update fc-005 status to resolved
|