claude-code/docs/plans/2026-01-01-claude-realtime-monitoring-implementation.md

# Claude Real-Time Monitoring (SSE) Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Add real-time-ish monitoring of Claude Code agent activity to the existing Go dashboard, with a backlog (last 200 history events) plus SSE updates, shown in a new “Live” UI feed with prettified rows and expandable raw JSON.

**Architecture:** Add an in-memory `EventHub` (pub/sub + ring buffer) that receives events from a `HistoryTailer` (tails `~/.claude/history.jsonl`) and a debounced file watcher (for `stats-cache.json` and `state/component-registry.json`). Expose a REST backlog endpoint (`/api/claude/live/backlog`) returning normalized `Event` objects (newest→oldest) and an SSE stream endpoint (`/api/claude/stream`) that pushes new events to the browser. Frontend uses `EventSource` plus batching (render every 1–2s).

**Tech Stack:** Go 1.21+, `chi`, vanilla HTML/CSS/JS, optional `fsnotify`.

---

## Decisions (locked)

- Transport: SSE first (WebSockets later).
- Acceptable latency: 2–5 seconds.
- UI: prettified table with expandable raw JSON.
- Backlog: enabled, default `limit=200`.
- Backlog ordering: **newest → oldest**.
- Parsing: **generic-first**, best-effort extraction of a few fields; always preserve raw JSON.
- Backlog response format: **normalized `events`** (not just raw lines).

---

## Data Contract

### Event JSON

All events share:

```json
{
  "id": 123,
  "ts": "2026-01-01T12:00:00Z",
  "type": "history.append",
  "data": {
    "summary": {
      "sessionId": "...",
      "project": "...",
      "display": "/model"
    },
    "rawLine": "{...}",
    "json": { "...": "..." },
    "parseError": "..."
  }
}
```

Event types:
- `history.append`
- `file.changed`
- `server.notice`
- `server.error`

---

## Task 1: Add `Event` types

**Files:**
- Create: `~/.claude/dashboard/internal/claude/events.go`
- Test: `~/.claude/dashboard/internal/claude/events_test.go`

**Step 1: Write the failing test**

```go
package claude

import "testing"

func TestEventTypesCompile(t *testing.T) {
	_ = Event{}
	_ = EventTypeHistoryAppend
	_ = EventTypeFileChanged
	_ = EventTypeServerNotice
	_ = EventTypeServerError
}
```

**Step 2: Run test to verify it fails**

Run: `go test ./...`
Expected: FAIL with `undefined: Event`

**Step 3: Write minimal implementation**

```go
package claude

import "time"

type EventType string

const (
	EventTypeHistoryAppend EventType = "history.append"
	EventTypeFileChanged   EventType = "file.changed"
	EventTypeServerNotice  EventType = "server.notice"
	EventTypeServerError   EventType = "server.error"
)

type Event struct {
	ID   int64     `json:"id"`
	TS   time.Time `json:"ts"`
	Type EventType `json:"type"`
	Data any       `json:"data"`
}
```

**Step 4: Run test to verify it passes**

Run: `go test ./...`
Expected: PASS

**Step 5: Commit**

```bash
git add internal/claude/events.go internal/claude/events_test.go
git commit -m "feat: add real-time event types"
```

---

## Task 2: Implement `EventHub` (pub/sub + ring buffer)

**Files:**
- Create: `~/.claude/dashboard/internal/claude/eventhub.go`
- Test: `~/.claude/dashboard/internal/claude/eventhub_test.go`

**Step 1: Write failing tests**

```go
package claude

import (
	"testing"
	"time"
)

func TestEventHub_PublishSubscribe(t *testing.T) {
	hub := NewEventHub(10)
	ch, cancel := hub.Subscribe()
	defer cancel()

	hub.Publish(Event{TS: time.Unix(1, 0), Type: EventTypeServerNotice, Data: map[string]any{"msg": "hi"}})

	select {
	case ev := <-ch:
		if ev.Type != EventTypeServerNotice {
			t.Fatalf("type=%s", ev.Type)
		}
		if ev.ID == 0 {
			t.Fatalf("expected id to be assigned")
		}
	default:
		t.Fatalf("expected event")
	}
}

func TestEventHub_ReplaySince(t *testing.T) {
	hub := NewEventHub(3)
	hub.Publish(Event{TS: time.Unix(1, 0), Type: EventTypeServerNotice}) // id 1
	hub.Publish(Event{TS: time.Unix(2, 0), Type: EventTypeServerNotice}) // id 2
	hub.Publish(Event{TS: time.Unix(3, 0), Type: EventTypeServerNotice}) // id 3

	got := hub.ReplaySince(1)
	if len(got) != 2 {
		t.Fatalf("len=%d", len(got))
	}
	if got[0].ID != 2 || got[1].ID != 3 {
		t.Fatalf("ids=%d,%d", got[0].ID, got[1].ID)
	}
}
```

**Step 2: Run tests to verify RED**

Run: `go test ./...`
Expected: FAIL with `undefined: NewEventHub`

**Step 3: Write minimal implementation**

Implement:
- `type EventHub struct { ... }`
- `NewEventHub(bufferSize int) *EventHub`
- `Publish(ev Event) Event`:
  - assign `ID` if zero using an internal counter
  - set `TS = time.Now()` if zero
  - append to ring buffer
  - broadcast to subscriber channels (non-blocking send)
- `Subscribe() (chan Event, func())` returns a buffered channel and a cancel func
- `ReplaySince(lastID int64) []Event`

**Step 4: Run tests to verify GREEN**

Run: `go test ./...`
Expected: PASS

**Step 5: Commit**

```bash
git add internal/claude/eventhub.go internal/claude/eventhub_test.go
git commit -m "feat: add event hub with replay buffer"
```

---

## Task 3: Tail last N lines helper (newest → oldest)

**Files:**
- Create: `~/.claude/dashboard/internal/claude/tail.go`
- Test: `~/.claude/dashboard/internal/claude/tail_test.go`

**Step 1: Write the failing test**

```go
package claude

import (
	"os"
	"path/filepath"
	"strings"
	"testing"
)

func TestTailLastNLines_NewestFirst(t *testing.T) {
	dir := t.TempDir()
	p := filepath.Join(dir, "history.jsonl")

	var b strings.Builder
	for i := 1; i <= 5; i++ {
		b.WriteString("line")
		b.WriteString([]string{"1","2","3","4","5"}[i-1])
		b.WriteString("\n")
	}
	if err := os.WriteFile(p, []byte(b.String()), 0o600); err != nil {
		t.Fatalf("write: %v", err)
	}

	lines, err := TailLastNLines(p, 2)
	if err != nil {
		t.Fatalf("TailLastNLines: %v", err)
	}
	if len(lines) != 2 {
		t.Fatalf("len=%d", len(lines))
	}
	if lines[0] != "line5" || lines[1] != "line4" {
		t.Fatalf("got=%v", lines)
	}
}
```

**Step 2: Run test to verify it fails**

Run: `go test ./...`
Expected: FAIL with `undefined: TailLastNLines`

**Step 3: Minimal implementation**

Create `~/.claude/dashboard/internal/claude/tail.go`:

- `TailLastNLines(path string, n int) ([]string, error)`
- First implementation can be simple (read whole file + split) with a TODO noting potential optimization.
- Return text lines without trailing newline; newest→oldest ordering.

**Step 4: Run tests to verify it passes**

Run: `go test ./...`
Expected: PASS

**Step 5: Commit**

```bash
git add internal/claude/tail.go internal/claude/tail_test.go
git commit -m "feat: add tail last N lines helper"
```

---

## Task 4: Add backlog endpoint returning normalized events

**Files:**
- Create: `~/.claude/dashboard/internal/api/claude_live_handlers.go`
- Modify: `~/.claude/dashboard/internal/api/claude_handlers.go` (only to share helper if needed)
- Modify: `~/.claude/dashboard/cmd/server/main.go`
- Test: `~/.claude/dashboard/internal/api/claude_live_handlers_test.go`

**Step 1: Write failing test**

```go
package api

import (
	"net/http"
	"net/http/httptest"
	"os"
	"path/filepath"
	"testing"

	"github.com/go-chi/chi/v5"
	"github.com/will/k8s-agent-dashboard/internal/claude"
)

type fakeClaudeDirLoader struct{ dir string }

func (f fakeClaudeDirLoader) ClaudeDir() string                                { return f.dir }
func (f fakeClaudeDirLoader) LoadStatsCache() (*claude.StatsCache, error)      { return &claude.StatsCache{}, nil }
func (f fakeClaudeDirLoader) ListDir(name string) ([]claude.DirEntry, error)   { return nil, nil }
func (f fakeClaudeDirLoader) FileMeta(relPath string) (claude.FileMeta, error) { return claude.FileMeta{}, nil }
func (f fakeClaudeDirLoader) PathExists(relPath string) bool                   { return true }

func TestClaudeLiveBacklog_DefaultLimit(t *testing.T) {
	dir := t.TempDir()
	p := filepath.Join(dir, "history.jsonl")
	if err := os.WriteFile(p, []byte("{\"display\":\"/model\"}\n"), 0o600); err != nil {
		t.Fatalf("write: %v", err)
	}

	loader := fakeClaudeDirLoader{dir: dir}

	r := chi.NewRouter()
	r.Get("/api/claude/live/backlog", GetClaudeLiveBacklog(loader))

	req := httptest.NewRequest(http.MethodGet, "/api/claude/live/backlog", nil)
	w := httptest.NewRecorder()
	r.ServeHTTP(w, req)

	if w.Code != http.StatusOK {
		t.Fatalf("status=%d body=%s", w.Code, w.Body.String())
	}
	// Assert response includes an "events" array with at least 1 event.
	if !jsonContainsKey(t, w.Body.Bytes(), "events") {
		t.Fatalf("expected events in response: %s", w.Body.String())
	}
}
```

**Step 2: Run test to verify RED**

Run: `go test ./...`
Expected: FAIL with `undefined: GetClaudeLiveBacklog`

**Step 3: Minimal implementation**

Create `~/.claude/dashboard/internal/api/claude_live_handlers.go`:

- `GetClaudeLiveBacklog(loader ClaudeLoader) http.HandlerFunc`
- Query param: `limit` (default 200; clamp 1..1000)
- Use `claude.TailLastNLines(filepath.Join(loader.ClaudeDir(), "history.jsonl"), limit)`
- For each line, create a `claude.Event` with:
  - `Type: claude.EventTypeHistoryAppend`
  - `TS: time.Now()` (or parse timestamp if present in JSON)
  - `Data` contains: `rawLine`, optionally `json`, optionally `parseError`, and `summary` (best effort)
- JSON parsing should be schema-agnostic: unmarshal into `map[string]any`.
- Summary extraction should look for keys: `sessionId`, `project`, `display` (strings).

Return payload:

```json
{ "limit": 200, "events": [ ... ] }
```

**Step 4: Run tests to verify GREEN**

Run: `go test ./...`
Expected: PASS

**Step 5: Wire route**

Modify `~/.claude/dashboard/cmd/server/main.go` to register:
- `GET /api/claude/live/backlog`

**Step 6: Run tests again**

Run: `go test ./...`
Expected: PASS

**Step 7: Commit**

```bash
git add cmd/server/main.go internal/api/claude_live_handlers.go internal/api/claude_live_handlers_test.go internal/claude/tail.go internal/claude/tail_test.go
git commit -m "feat: add claude live backlog endpoint"
```

---

## Task 5: Add SSE stream endpoint

**Files:**
- Create: `~/.claude/dashboard/internal/api/claude_stream_handlers.go`
- Modify: `~/.claude/dashboard/cmd/server/main.go`
- Test: `~/.claude/dashboard/internal/api/claude_stream_handlers_test.go`

**Step 1: Write failing test**

```go
package api

import (
	"net/http"
	"net/http/httptest"
	"strings"
	"testing"
	"time"

	"github.com/go-chi/chi/v5"
	"github.com/will/k8s-agent-dashboard/internal/claude"
)

func TestClaudeStream_SendsEvent(t *testing.T) {
	hub := claude.NewEventHub(10)

	r := chi.NewRouter()
	r.Get("/api/claude/stream", GetClaudeStream(hub))

	req := httptest.NewRequest(http.MethodGet, "/api/claude/stream", nil)
	w := httptest.NewRecorder()

	// Publish after handler starts.
	go func() {
		time.Sleep(10 * time.Millisecond)
		hub.Publish(claude.Event{Type: claude.EventTypeServerNotice, Data: map[string]any{"msg": "hi"}})
	}()

	r.ServeHTTP(w, req)

	if ct := w.Header().Get("Content-Type"); !strings.Contains(ct, "text/event-stream") {
		t.Fatalf("content-type=%q", ct)
	}
	if !strings.Contains(w.Body.String(), "event:") || !strings.Contains(w.Body.String(), "data:") {
		t.Fatalf("body=%s", w.Body.String())
	}
}
```

**Step 2: Run test to verify RED**

Run: `go test ./...`
Expected: FAIL with `undefined: GetClaudeStream`

**Step 3: Implement minimal SSE handler**

Create `~/.claude/dashboard/internal/api/claude_stream_handlers.go`:

- `GetClaudeStream(hub *claude.EventHub) http.HandlerFunc`
- Set headers:
  - `Content-Type: text/event-stream`
  - `Cache-Control: no-cache`
- Subscribe to hub; write events in SSE format:

```
event: <type>
id: <id>
data: <json>


```

- Flush after each event.
- Keep it minimal; add keepalive pings later.

**Step 4: Run tests to verify GREEN**

Run: `go test ./...`
Expected: PASS

**Step 5: Wire route**

Modify `~/.claude/dashboard/cmd/server/main.go` to register:
- `GET /api/claude/stream`

**Step 6: Run tests again**

Run: `go test ./...`
Expected: PASS

**Step 7: Commit**

```bash
git add cmd/server/main.go internal/api/claude_stream_handlers.go internal/api/claude_stream_handlers_test.go internal/claude/eventhub.go internal/claude/eventhub_test.go
git commit -m "feat: add claude sse stream endpoint"
```

---

## Task 6: Implement HistoryTailer to publish hub events

**Files:**
- Create: `~/.claude/dashboard/internal/claude/history_tailer.go`
- Test: `~/.claude/dashboard/internal/claude/history_tailer_test.go`
- Modify: `~/.claude/dashboard/cmd/server/main.go`

**Step 1: Write failing test**

```go
package claude

import (
	"os"
	"path/filepath"
	"testing"
	"time"
)

func TestHistoryTailer_EmitsOnAppend(t *testing.T) {
	dir := t.TempDir()
	p := filepath.Join(dir, "history.jsonl")
	if err := os.WriteFile(p, []byte(""), 0o600); err != nil {
		t.Fatalf("write: %v", err)
	}

	hub := NewEventHub(10)
	ch, cancel := hub.Subscribe()
	defer cancel()

	stop := make(chan struct{})
	go TailHistoryFile(stop, hub, p)

	// Append a line
	if err := os.WriteFile(p, []byte("{\"display\":\"/status\"}\n"), 0o600); err != nil {
		t.Fatalf("append: %v", err)
	}

	select {
	case ev := <-ch:
		if ev.Type != EventTypeHistoryAppend {
			t.Fatalf("type=%s", ev.Type)
		}
	case <-time.After(200 * time.Millisecond):
		t.Fatalf("timed out waiting for event")
	}

	close(stop)
}
```

**Step 2: Run tests to verify RED**

Run: `go test ./...`
Expected: FAIL with `undefined: TailHistoryFile`

**Step 3: Minimal implementation**

Create `~/.claude/dashboard/internal/claude/history_tailer.go` implementing:

- `TailHistoryFile(stop <-chan struct{}, hub *EventHub, path string)`
- Simple polling loop (since target latency is 2–5s):
  - Every 500ms–1s, stat file size
  - If size grew, read new bytes from offset, split on `\n`, publish `history.append` events
  - If size shrank, reset offset to 0 and publish `server.notice`

Also implement an internal helper to parse a history line into event `Data` with `summary` extraction (same logic as backlog).

**Step 4: Run tests to verify GREEN**

Run: `go test ./...`
Expected: PASS

**Step 5: Wire tailer in server**

Modify `~/.claude/dashboard/cmd/server/main.go`:
- Create hub at startup: `hub := claude.NewEventHub(1000)`
- Start goroutine tailing `filepath.Join(*claudeDir, "history.jsonl")`

**Step 6: Run tests again**

Run: `go test ./...`
Expected: PASS

**Step 7: Commit**

```bash
git add cmd/server/main.go internal/claude/history_tailer.go internal/claude/history_tailer_test.go
git commit -m "feat: stream history.jsonl appends via event hub"
```

---

## Task 7: Frontend Live view (EventSource + batching)

**Files:**
- Modify: `~/.claude/dashboard/cmd/server/web/index.html`
- Modify: `~/.claude/dashboard/cmd/server/web/static/js/app.js`
- Modify: `~/.claude/dashboard/cmd/server/web/static/css/style.css`

**Step 1: Add Live tab + markup**

- Add nav button: `data-view="live"`
- Add section:
  - `id="live-view"`
  - table `id="claude-live-table"` and a connection indicator `id="claude-live-conn"`

**Step 2: Add JS backlog fetch + EventSource**

Modify `~/.claude/dashboard/cmd/server/web/static/js/app.js`:
- On DOMContentLoaded, create `EventSource('/api/claude/stream')`
- Maintain:
  - `let pendingLiveEvents = []`
  - `let liveEvents = []` (cap at 500)
- Every 1000ms:
  - move pending → live
  - render table rows
- Fetch backlog once:
  - `GET /api/claude/live/backlog?limit=200`
  - prepend/append into `liveEvents` (newest→oldest returned; UI should render newest first at top)

**Step 3: CSS**

- Add a small connection badge style (green/yellow/red)
- Ensure table remains readable

**Step 4: Manual verification**

Run:
- `go test ./...`
- `go run ./cmd/server --port 8080 --data /tmp/k8s --claude ~/.claude`

Expected:
- Live tab loads backlog rows
- New history events appear on subsequent CLI activity

**Step 5: Commit**

```bash
git add cmd/server/web/index.html cmd/server/web/static/js/app.js cmd/server/web/static/css/style.css
git commit -m "feat: add live feed UI with SSE batching"
```

---

## Task 8: End-to-end verification

**Files:**
- None (unless a bug requires fixes)

**Step 1: Run full test suite**

Run: `go test ./...`
Expected: PASS (0 failures)

**Step 2: Manual smoke check**

Run:
- `go run ./cmd/server --port 8080 --data /tmp/k8s --claude ~/.claude`

Check:
- `curl -N http://localhost:8080/api/claude/stream` prints SSE lines
- `curl http://localhost:8080/api/claude/live/backlog?limit=5` returns `events` array
- Browser Live tab updates

---

## Execution handoff

Plan complete and saved to `~/.claude/docs/plans/2026-01-01-claude-realtime-monitoring-implementation.md`.

Two execution options:

1. Subagent-Driven (this session) — I dispatch fresh subagent per task, review between tasks, fast iteration
2. Parallel Session (separate) — Open new session with `superpowers:executing-plans`, batch execution with checkpoints

Which approach?