Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f3cb082c36 | |||
| db0d9f97b2 | |||
| 94603b19a5 | |||
| 45b7e4bcf7 | |||
| 7ca8caeecb | |||
| c21b152de8 |
@@ -45,3 +45,7 @@ tmp_unused
|
|||||||
# Todos (managed by Claude Code)
|
# Todos (managed by Claude Code)
|
||||||
todos/
|
todos/
|
||||||
repos/homelab
|
repos/homelab
|
||||||
|
|
||||||
|
# RAG search data (generated vector stores and caches)
|
||||||
|
data/
|
||||||
|
skills/rag-search/venv/
|
||||||
|
|||||||
@@ -0,0 +1,388 @@
|
|||||||
|
# Agentic RAG Design
|
||||||
|
|
||||||
|
**Date:** 2025-01-21
|
||||||
|
**Status:** Ready for implementation
|
||||||
|
**Category:** Agent memory / Knowledge retrieval
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Add semantic search to the existing Claude agent system, enabling multi-source reasoning that combines personal context (state files, memory, decisions) with external documentation.
|
||||||
|
|
||||||
|
### Goals
|
||||||
|
|
||||||
|
- Retrieve relevant past decisions and preferences when answering questions
|
||||||
|
- Search external docs (k0s, ArgoCD, Prometheus, etc.) for technical reference
|
||||||
|
- Cross-reference personal context with official documentation
|
||||||
|
- Support iterative query refinement (agentic behavior)
|
||||||
|
|
||||||
|
### Non-Goals (Future Considerations)
|
||||||
|
|
||||||
|
Deferred to `future-considerations.json`:
|
||||||
|
|
||||||
|
- **fc-043**: Auto-sync on tool version change
|
||||||
|
- **fc-044**: Broad doc indexing (hundreds of sources)
|
||||||
|
- **fc-045**: K8s deployment
|
||||||
|
- **fc-046**: Query caching
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
User question
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Personal Assistant (existing)
|
||||||
|
│
|
||||||
|
├── Decides if RAG would help
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
rag-search skill (new)
|
||||||
|
│
|
||||||
|
├── Query embedding
|
||||||
|
├── Vector similarity search
|
||||||
|
├── Return ranked chunks with metadata
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Claude reasons over results
|
||||||
|
│
|
||||||
|
├── Good enough? → Answer
|
||||||
|
└── Need more? → Reformulate, search again
|
||||||
|
```
|
||||||
|
|
||||||
|
### Two Indexes
|
||||||
|
|
||||||
|
| Index | Contents | Update Frequency |
|
||||||
|
|-------|----------|------------------|
|
||||||
|
| **personal** | `~/.claude/state/` files, memory, decisions, preferences | Daily |
|
||||||
|
| **docs** | External documentation (k0s, ArgoCD, etc.) | Daily |
|
||||||
|
|
||||||
|
### Why Two Indexes
|
||||||
|
|
||||||
|
- Different update frequencies
|
||||||
|
- Different retrieval strategies (personal may weight recency)
|
||||||
|
- Can query one or both depending on the question
|
||||||
|
|
||||||
|
## Components
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ rag-search skill │
|
||||||
|
│ (Claude invokes this) │
|
||||||
|
└─────────────────────┬───────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
┌─────────────┴─────────────┐
|
||||||
|
▼ ▼
|
||||||
|
┌───────────────────┐ ┌───────────────────┐
|
||||||
|
│ Personal Index │ │ Docs Index │
|
||||||
|
│ │ │ │
|
||||||
|
│ ~/.claude/state/* │ │ External docs │
|
||||||
|
│ memory/*.json │ │ (k0s, ArgoCD...) │
|
||||||
|
│ kb.json │ │ │
|
||||||
|
└────────┬──────────┘ └────────┬──────────┘
|
||||||
|
│ │
|
||||||
|
└──────────┬──────────────┘
|
||||||
|
▼
|
||||||
|
┌───────────────────┐
|
||||||
|
│ Vector Store │
|
||||||
|
│ (ChromaDB) │
|
||||||
|
│ │
|
||||||
|
│ Collections: │
|
||||||
|
│ - personal │
|
||||||
|
│ - docs │
|
||||||
|
└────────┬──────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌───────────────────┐
|
||||||
|
│ Embedding Model │
|
||||||
|
│ (sentence- │
|
||||||
|
│ transformers) │
|
||||||
|
└───────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Stack
|
||||||
|
|
||||||
|
| Component | Choice | Notes |
|
||||||
|
|-----------|--------|-------|
|
||||||
|
| Vector store | ChromaDB | Pure Python, no external deps |
|
||||||
|
| Embeddings | sentence-transformers (all-MiniLM-L6-v2) | Runs on arm64, ~90MB |
|
||||||
|
| Storage | `~/.claude/data/rag-search/` | Local to workstation |
|
||||||
|
|
||||||
|
## Skill Structure
|
||||||
|
|
||||||
|
**Location:** `~/.claude/skills/rag-search/`
|
||||||
|
|
||||||
|
```
|
||||||
|
rag-search/
|
||||||
|
├── SKILL.md # Instructions for Claude
|
||||||
|
├── scripts/
|
||||||
|
│ ├── search.py # Main search entry point
|
||||||
|
│ ├── index_personal.py # Index state files
|
||||||
|
│ ├── index_docs.py # Index external docs
|
||||||
|
│ └── add_doc_source.py # Add new doc source
|
||||||
|
└── references/
|
||||||
|
└── sources.json # Configured doc sources
|
||||||
|
```
|
||||||
|
|
||||||
|
## Skill Interface
|
||||||
|
|
||||||
|
### Invocation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Basic search (both indexes)
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py "how did I configure ArgoCD sync?"
|
||||||
|
|
||||||
|
# Search specific index
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py --index personal "past decisions about caching"
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py --index docs "k0s node maintenance"
|
||||||
|
|
||||||
|
# Control result count
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py --top-k 10 "prometheus alerting rules"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Output Format
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"query": "how did I configure ArgoCD sync?",
|
||||||
|
"results": [
|
||||||
|
{
|
||||||
|
"rank": 1,
|
||||||
|
"score": 0.847,
|
||||||
|
"source": "personal",
|
||||||
|
"file": "memory/decisions.json",
|
||||||
|
"chunk": "Decided to use ArgoCD auto-sync with self-heal disabled...",
|
||||||
|
"metadata": {"date": "2025-01-15", "context": "k8s setup"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"rank": 2,
|
||||||
|
"score": 0.823,
|
||||||
|
"source": "docs",
|
||||||
|
"file": "argocd/sync-options.md",
|
||||||
|
"chunk": "Auto-sync can be configured with selfHeal and prune options...",
|
||||||
|
"metadata": {"doc_version": "2.9", "url": "https://..."}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"searched_collections": ["personal", "docs"],
|
||||||
|
"total_chunks_searched": 1847
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### SKILL.md Guidance
|
||||||
|
|
||||||
|
- Start with broad query, refine if results aren't relevant
|
||||||
|
- Cross-reference personal decisions with docs when both appear
|
||||||
|
- Cite sources in answers (file + date for personal, URL for docs)
|
||||||
|
|
||||||
|
## External Docs Management
|
||||||
|
|
||||||
|
### Source Registry
|
||||||
|
|
||||||
|
**Location:** `~/.claude/skills/rag-search/references/sources.json`
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"id": "k0s",
|
||||||
|
"name": "k0s Documentation",
|
||||||
|
"type": "git",
|
||||||
|
"url": "https://github.com/k0sproject/k0s.git",
|
||||||
|
"path": "docs/",
|
||||||
|
"glob": "**/*.md",
|
||||||
|
"version": "v1.30.0",
|
||||||
|
"last_indexed": "2025-01-20T10:00:00Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "argocd",
|
||||||
|
"name": "ArgoCD Documentation",
|
||||||
|
"type": "web",
|
||||||
|
"base_url": "https://argo-cd.readthedocs.io/en/stable/",
|
||||||
|
"pages": ["user-guide/sync-options/", "operator-manual/"],
|
||||||
|
"last_indexed": "2025-01-18T14:30:00Z"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Adding Sources
|
||||||
|
|
||||||
|
```bash
|
||||||
|
~/.claude/skills/rag-search/scripts/add_doc_source.py \
|
||||||
|
--id "cilium" \
|
||||||
|
--name "Cilium Docs" \
|
||||||
|
--type git \
|
||||||
|
--url "https://github.com/cilium/cilium.git" \
|
||||||
|
--path "Documentation/" \
|
||||||
|
--glob "**/*.md"
|
||||||
|
|
||||||
|
# Then index it
|
||||||
|
~/.claude/skills/rag-search/scripts/index_docs.py --source cilium
|
||||||
|
```
|
||||||
|
|
||||||
|
### Update Strategies
|
||||||
|
|
||||||
|
| Strategy | Command | When |
|
||||||
|
|----------|---------|------|
|
||||||
|
| Manual | `index_docs.py --source <id>` | After version upgrade |
|
||||||
|
| All sources | `index_docs.py --all` | Periodic refresh |
|
||||||
|
|
||||||
|
## Periodic Refresh
|
||||||
|
|
||||||
|
Daily systemd timer on workstation.
|
||||||
|
|
||||||
|
### Service
|
||||||
|
|
||||||
|
**Location:** `~/.config/systemd/user/rag-index.service`
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=Refresh RAG search indexes
|
||||||
|
After=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=%h/.claude/skills/rag-search/scripts/index_docs.py --all --quiet
|
||||||
|
ExecStartPost=%h/.claude/skills/rag-search/scripts/index_personal.py --quiet
|
||||||
|
Environment=PATH=%h/.claude/skills/rag-search/venv/bin:/usr/bin
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
|
```
|
||||||
|
|
||||||
|
### Timer
|
||||||
|
|
||||||
|
**Location:** `~/.config/systemd/user/rag-index.timer`
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=Daily RAG index refresh
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnCalendar=daily
|
||||||
|
Persistent=true
|
||||||
|
RandomizedDelaySec=3600
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
```
|
||||||
|
|
||||||
|
### Enable
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user daemon-reload
|
||||||
|
systemctl --user enable --now rag-index.timer
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual Trigger
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user start rag-index.service
|
||||||
|
journalctl --user -u rag-index.service # View logs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Resource Requirements
|
||||||
|
|
||||||
|
**Target:** Workstation or Pi5 8GB
|
||||||
|
|
||||||
|
| Component | RAM | Disk | Notes |
|
||||||
|
|-----------|-----|------|-------|
|
||||||
|
| Embedding model (all-MiniLM-L6-v2) | ~256MB | ~90MB | Loaded on-demand |
|
||||||
|
| ChromaDB | ~100-500MB | Varies | Scales with index size |
|
||||||
|
| Index: personal (~50 files) | — | ~5MB | Small, fast to query |
|
||||||
|
| Index: docs (10-20 sources) | — | ~100-500MB | Depends on doc volume |
|
||||||
|
| Indexing process (peak) | ~1GB | — | During embedding generation |
|
||||||
|
|
||||||
|
**Pi3 1GB:** Not suitable for this workload.
|
||||||
|
|
||||||
|
## Chunking Strategy
|
||||||
|
|
||||||
|
| Index | Strategy |
|
||||||
|
|-------|----------|
|
||||||
|
| Personal | Per JSON key or logical section (decisions, preferences, facts as separate chunks) |
|
||||||
|
| Docs | ~500 tokens per chunk with overlap, preserve headers as metadata |
|
||||||
|
|
||||||
|
## Implementation Notes
|
||||||
|
|
||||||
|
### Recommended: Ralph Loop
|
||||||
|
|
||||||
|
This design is suitable for Ralph loop implementation:
|
||||||
|
- Clear success criteria (tests, functional checks)
|
||||||
|
- Iterative refinement expected (tuning chunking, embeddings)
|
||||||
|
- Automatic verification possible
|
||||||
|
|
||||||
|
### Model Delegation
|
||||||
|
|
||||||
|
Use appropriate models for each phase:
|
||||||
|
|
||||||
|
| Phase | Task | Model |
|
||||||
|
|-------|------|-------|
|
||||||
|
| 1 | Set up ChromaDB + embedding model | Haiku |
|
||||||
|
| 2 | Write `index_personal.py` | Sonnet |
|
||||||
|
| 3 | Write `index_docs.py` | Sonnet |
|
||||||
|
| 4 | Write `search.py` | Sonnet |
|
||||||
|
| 5 | Write SKILL.md | Haiku |
|
||||||
|
| 6 | Integration tests | Sonnet |
|
||||||
|
| 7 | End-to-end validation | Sonnet |
|
||||||
|
|
||||||
|
### Ralph Invocation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
/ralph-loop "Implement rag-search skill per docs/plans/2025-01-21-agentic-rag-design.md.
|
||||||
|
|
||||||
|
Delegate to appropriate models:
|
||||||
|
- Haiku: setup, docs, simple scripts
|
||||||
|
- Sonnet: implementation, tests, debugging
|
||||||
|
- Opus: only if stuck on complex reasoning
|
||||||
|
|
||||||
|
Success criteria:
|
||||||
|
1. ChromaDB + embeddings working
|
||||||
|
2. Personal index populated from ~/.claude/state
|
||||||
|
3. At least one external doc source indexed
|
||||||
|
4. search.py returns relevant results
|
||||||
|
5. All tests pass
|
||||||
|
|
||||||
|
Output <promise>COMPLETE</promise> when done." --max-iterations 30 --completion-promise "COMPLETE"
|
||||||
|
```
|
||||||
|
|
||||||
|
### When NOT to use Ralph
|
||||||
|
|
||||||
|
- Design decisions still needed (use brainstorming first)
|
||||||
|
- Requires human judgment mid-implementation
|
||||||
|
- One-shot simple tasks
|
||||||
|
|
||||||
|
## Workflow Integration
|
||||||
|
|
||||||
|
```
|
||||||
|
/superpowers:brainstorm
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Design doc created
|
||||||
|
(docs/plans/YYYY-MM-DD-*-design.md)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
"Ready to implement?"
|
||||||
|
│
|
||||||
|
┌────┴────┐
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
Simple Complex/Iterative
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
Manual /ralph-loop
|
||||||
|
or TDD with design doc
|
||||||
|
as spec
|
||||||
|
```
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
| Aspect | Decision |
|
||||||
|
|--------|----------|
|
||||||
|
| **Architecture** | Extend existing Claude skill system with semantic search |
|
||||||
|
| **Indexes** | Two: personal (state files) + docs (external) |
|
||||||
|
| **Vector store** | ChromaDB (local, no deps) |
|
||||||
|
| **Embeddings** | sentence-transformers (all-MiniLM-L6-v2) |
|
||||||
|
| **Skill interface** | `rag-search` skill with `search.py` CLI |
|
||||||
|
| **Doc management** | `sources.json` registry, git/web fetching |
|
||||||
|
| **Refresh** | systemd user timer, daily |
|
||||||
|
| **Storage** | `~/.claude/data/rag-search/` |
|
||||||
|
| **Hardware** | Runs on workstation (Pi5 8GB capable if needed) |
|
||||||
|
| **Implementation** | Ralph loop with Haiku/Sonnet subagent delegation |
|
||||||
@@ -4,10 +4,10 @@
|
|||||||
"frontend-design@claude-plugins-official": [
|
"frontend-design@claude-plugins-official": [
|
||||||
{
|
{
|
||||||
"scope": "user",
|
"scope": "user",
|
||||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/frontend-design/6d3752c000e2",
|
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/frontend-design/15b07b46dab3",
|
||||||
"version": "6d3752c000e2",
|
"version": "15b07b46dab3",
|
||||||
"installedAt": "2025-12-24T19:08:12.422Z",
|
"installedAt": "2025-12-24T19:08:12.422Z",
|
||||||
"lastUpdated": "2025-12-24T19:08:12.422Z",
|
"lastUpdated": "2026-01-05T07:21:36.978Z",
|
||||||
"gitCommitSha": "6d3752c000e2b3d0e6137bd7adb04895d6f40f14",
|
"gitCommitSha": "6d3752c000e2b3d0e6137bd7adb04895d6f40f14",
|
||||||
"isLocal": true
|
"isLocal": true
|
||||||
}
|
}
|
||||||
@@ -26,10 +26,10 @@
|
|||||||
"commit-commands@claude-plugins-official": [
|
"commit-commands@claude-plugins-official": [
|
||||||
{
|
{
|
||||||
"scope": "user",
|
"scope": "user",
|
||||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/commit-commands/6d3752c000e2",
|
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/commit-commands/15b07b46dab3",
|
||||||
"version": "6d3752c000e2",
|
"version": "15b07b46dab3",
|
||||||
"installedAt": "2025-12-24T19:10:05.451Z",
|
"installedAt": "2025-12-24T19:10:05.451Z",
|
||||||
"lastUpdated": "2025-12-24T19:10:36.843Z",
|
"lastUpdated": "2026-01-05T07:21:36.984Z",
|
||||||
"isLocal": true
|
"isLocal": true
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
@@ -69,10 +69,10 @@
|
|||||||
"ralph-wiggum@claude-plugins-official": [
|
"ralph-wiggum@claude-plugins-official": [
|
||||||
{
|
{
|
||||||
"scope": "user",
|
"scope": "user",
|
||||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/ralph-wiggum/6d3752c000e2",
|
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/ralph-wiggum/15b07b46dab3",
|
||||||
"version": "6d3752c000e2",
|
"version": "15b07b46dab3",
|
||||||
"installedAt": "2026-01-02T19:47:02.395Z",
|
"installedAt": "2026-01-02T19:47:02.395Z",
|
||||||
"lastUpdated": "2026-01-02T19:47:11.472Z",
|
"lastUpdated": "2026-01-05T07:21:36.997Z",
|
||||||
"gitCommitSha": "de89f3066c68d7a2f2d4190173fa46c26e2f30fd",
|
"gitCommitSha": "de89f3066c68d7a2f2d4190173fa46c26e2f30fd",
|
||||||
"isLocal": true
|
"isLocal": true
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,7 +5,7 @@
|
|||||||
"repo": "anthropics/claude-plugins-official"
|
"repo": "anthropics/claude-plugins-official"
|
||||||
},
|
},
|
||||||
"installLocation": "/home/will/.claude/plugins/marketplaces/claude-plugins-official",
|
"installLocation": "/home/will/.claude/plugins/marketplaces/claude-plugins-official",
|
||||||
"lastUpdated": "2026-01-04T20:00:29.464Z"
|
"lastUpdated": "2026-01-05T07:22:46.460Z"
|
||||||
},
|
},
|
||||||
"superpowers-marketplace": {
|
"superpowers-marketplace": {
|
||||||
"source": {
|
"source": {
|
||||||
|
|||||||
@@ -1,16 +1,15 @@
|
|||||||
# Morning Report - Sun Jan 04, 2026
|
# Morning Report - Sun Jan 04, 2026
|
||||||
|
|
||||||
## 🌤 Weather
|
## 🌤 Weather
|
||||||
Weather unavailable: <urlopen error timed out>
|
Seattle: 51°F, Partly cloudy | High 52° Low 43°
|
||||||
|
|
||||||
## 📧 Email
|
## 📧 Email
|
||||||
10 unread, 0 urgent
|
15 unread
|
||||||
|
• Capital One | Quicks - Your requested balance summary
|
||||||
- Chase - Your Chase Freedom Unlimited Visa balance is...
|
• Uber Receipts - [Personal] Your Saturday evening trip wi
|
||||||
- Chase - Your rewards balance has reached 0 POINTS
|
• Experian - William, it's time to check your utiliza
|
||||||
- USPS - PO Box Price Changes Coming
|
• Experteer Search Age - William, we have 2 new opportunities for
|
||||||
- Uber Receipts - Your Saturday evening trip with Uber
|
• Chase - You can start your mortgage preapproval
|
||||||
- Chase - You made an online, phone, or mail transaction...
|
|
||||||
|
|
||||||
## 📅 Today
|
## 📅 Today
|
||||||
• 2:00 PM - Seattle Saturday (SAM + QED + Lecosho) (5h)
|
• 2:00 PM - Seattle Saturday (SAM + QED + Lecosho) (5h)
|
||||||
@@ -18,17 +17,14 @@ Weather unavailable: <urlopen error timed out>
|
|||||||
## 📈 Stocks
|
## 📈 Stocks
|
||||||
CRWV $79.32 +10.8% ▲ NVDA $188.85 +1.3% ▲ MSFT $472.94 -2.2% ▼
|
CRWV $79.32 +10.8% ▲ NVDA $188.85 +1.3% ▲ MSFT $472.94 -2.2% ▼
|
||||||
|
|
||||||
## ✅ Tasks
|
|
||||||
⚠️ Could not fetch tasks: ('invalid_scope: Bad Request', {'error': 'invalid_scope', 'error_description': 'Bad Request'})
|
|
||||||
|
|
||||||
## 🖥 Infrastructure
|
## 🖥 Infrastructure
|
||||||
K8s: 🟢 | Workstation: 🟢
|
K8s: 🟢 | Workstation: 🟢
|
||||||
|
|
||||||
## 📰 Tech News
|
## 📰 Tech News
|
||||||
• Anti-Aging Injection Regrows Knee Cartilage and Prevents Art... (Hacker News)
|
• C-Sentinel: System prober that captures "system fingerprints... (Hacker News)
|
||||||
• How I archived 10 years of memories using Spotify (Hacker News)
|
• Show HN: An LLM-Powered PCB Schematic Checker (Major Update) (Hacker News)
|
||||||
• Can I finally start using Wayland in 2026? (Lobsters)
|
• Can I finally start using Wayland in 2026? (Lobsters)
|
||||||
• Saying goodbye to the servers at our physical datacenter - S... (Lobsters)
|
• Saying goodbye to the servers at our physical datacenter (Lobsters)
|
||||||
|
|
||||||
---
|
---
|
||||||
*Generated: 2026-01-04 08:00:33 PT*
|
*Generated: 2026-01-04 14:40:48 PT*
|
||||||
+11
-15
@@ -1,16 +1,15 @@
|
|||||||
# Morning Report - Sun Jan 04, 2026
|
# Morning Report - Sun Jan 04, 2026
|
||||||
|
|
||||||
## 🌤 Weather
|
## 🌤 Weather
|
||||||
Weather unavailable: <urlopen error timed out>
|
Seattle: 51°F, Partly cloudy | High 52° Low 43°
|
||||||
|
|
||||||
## 📧 Email
|
## 📧 Email
|
||||||
10 unread, 0 urgent
|
15 unread
|
||||||
|
• Capital One | Quicks - Your requested balance summary
|
||||||
- Chase - Your Chase Freedom Unlimited Visa balance is...
|
• Uber Receipts - [Personal] Your Saturday evening trip wi
|
||||||
- Chase - Your rewards balance has reached 0 POINTS
|
• Experian - William, it's time to check your utiliza
|
||||||
- USPS - PO Box Price Changes Coming
|
• Experteer Search Age - William, we have 2 new opportunities for
|
||||||
- Uber Receipts - Your Saturday evening trip with Uber
|
• Chase - You can start your mortgage preapproval
|
||||||
- Chase - You made an online, phone, or mail transaction...
|
|
||||||
|
|
||||||
## 📅 Today
|
## 📅 Today
|
||||||
• 2:00 PM - Seattle Saturday (SAM + QED + Lecosho) (5h)
|
• 2:00 PM - Seattle Saturday (SAM + QED + Lecosho) (5h)
|
||||||
@@ -18,17 +17,14 @@ Weather unavailable: <urlopen error timed out>
|
|||||||
## 📈 Stocks
|
## 📈 Stocks
|
||||||
CRWV $79.32 +10.8% ▲ NVDA $188.85 +1.3% ▲ MSFT $472.94 -2.2% ▼
|
CRWV $79.32 +10.8% ▲ NVDA $188.85 +1.3% ▲ MSFT $472.94 -2.2% ▼
|
||||||
|
|
||||||
## ✅ Tasks
|
|
||||||
⚠️ Could not fetch tasks: ('invalid_scope: Bad Request', {'error': 'invalid_scope', 'error_description': 'Bad Request'})
|
|
||||||
|
|
||||||
## 🖥 Infrastructure
|
## 🖥 Infrastructure
|
||||||
K8s: 🟢 | Workstation: 🟢
|
K8s: 🟢 | Workstation: 🟢
|
||||||
|
|
||||||
## 📰 Tech News
|
## 📰 Tech News
|
||||||
• Anti-Aging Injection Regrows Knee Cartilage and Prevents Art... (Hacker News)
|
• C-Sentinel: System prober that captures "system fingerprints... (Hacker News)
|
||||||
• How I archived 10 years of memories using Spotify (Hacker News)
|
• Show HN: An LLM-Powered PCB Schematic Checker (Major Update) (Hacker News)
|
||||||
• Can I finally start using Wayland in 2026? (Lobsters)
|
• Can I finally start using Wayland in 2026? (Lobsters)
|
||||||
• Saying goodbye to the servers at our physical datacenter - S... (Lobsters)
|
• Saying goodbye to the servers at our physical datacenter (Lobsters)
|
||||||
|
|
||||||
---
|
---
|
||||||
*Generated: 2026-01-04 08:00:33 PT*
|
*Generated: 2026-01-04 14:40:48 PT*
|
||||||
@@ -12,6 +12,7 @@ Agent skills that extend Claude's capabilities. Model-invoked (Claude decides wh
|
|||||||
| `sysadmin-health` | Arch Linux health check | `health-check.sh` |
|
| `sysadmin-health` | Arch Linux health check | `health-check.sh` |
|
||||||
| `usage` | Session usage tracking | `usage_report.py` |
|
| `usage` | Session usage tracking | `usage_report.py` |
|
||||||
| `programmer-add-project` | Register projects | (workflow only) |
|
| `programmer-add-project` | Register projects | (workflow only) |
|
||||||
|
| `rag-search` | Semantic search (state + docs) | `search.py`, `index_personal.py`, `index_docs.py` |
|
||||||
|
|
||||||
## Skill Structure
|
## Skill Structure
|
||||||
|
|
||||||
|
|||||||
@@ -25,6 +25,7 @@
|
|||||||
"show_tomorrow": true
|
"show_tomorrow": true
|
||||||
},
|
},
|
||||||
"tasks": {
|
"tasks": {
|
||||||
|
"enabled": false,
|
||||||
"max_display": 5,
|
"max_display": 5,
|
||||||
"show_due_dates": true
|
"show_due_dates": true
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -9,11 +9,13 @@ from pathlib import Path
|
|||||||
|
|
||||||
def fetch_events(mode: str = "today") -> list:
|
def fetch_events(mode: str = "today") -> list:
|
||||||
"""Fetch calendar events directly using gmail_mcp library."""
|
"""Fetch calendar events directly using gmail_mcp library."""
|
||||||
os.environ.setdefault('GMAIL_CREDENTIALS_PATH', os.path.expanduser('~/.gmail-mcp/credentials.json'))
|
os.environ.setdefault(
|
||||||
|
"GMAIL_CREDENTIALS_PATH", os.path.expanduser("~/.gmail-mcp/credentials.json")
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Add gmail venv to path
|
# Add gmail venv to path
|
||||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.13/site-packages"
|
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.14/site-packages"
|
||||||
if str(venv_site) not in sys.path:
|
if str(venv_site) not in sys.path:
|
||||||
sys.path.insert(0, str(venv_site))
|
sys.path.insert(0, str(venv_site))
|
||||||
|
|
||||||
@@ -22,26 +24,32 @@ def fetch_events(mode: str = "today") -> list:
|
|||||||
service = get_calendar_service()
|
service = get_calendar_service()
|
||||||
now = datetime.utcnow()
|
now = datetime.utcnow()
|
||||||
|
|
||||||
if mode == 'today':
|
if mode == "today":
|
||||||
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||||
end = start + timedelta(days=1)
|
end = start + timedelta(days=1)
|
||||||
elif mode == 'tomorrow':
|
elif mode == "tomorrow":
|
||||||
start = (now + timedelta(days=1)).replace(hour=0, minute=0, second=0, microsecond=0)
|
start = (now + timedelta(days=1)).replace(
|
||||||
|
hour=0, minute=0, second=0, microsecond=0
|
||||||
|
)
|
||||||
end = start + timedelta(days=1)
|
end = start + timedelta(days=1)
|
||||||
else:
|
else:
|
||||||
start = now
|
start = now
|
||||||
end = now + timedelta(days=7)
|
end = now + timedelta(days=7)
|
||||||
|
|
||||||
events_result = service.events().list(
|
events_result = (
|
||||||
calendarId='primary',
|
service.events()
|
||||||
timeMin=start.isoformat() + 'Z',
|
.list(
|
||||||
timeMax=end.isoformat() + 'Z',
|
calendarId="primary",
|
||||||
singleEvents=True,
|
timeMin=start.isoformat() + "Z",
|
||||||
orderBy='startTime',
|
timeMax=end.isoformat() + "Z",
|
||||||
maxResults=20
|
singleEvents=True,
|
||||||
).execute()
|
orderBy="startTime",
|
||||||
|
maxResults=20,
|
||||||
|
)
|
||||||
|
.execute()
|
||||||
|
)
|
||||||
|
|
||||||
return events_result.get('items', [])
|
return events_result.get("items", [])
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
return [{"error": str(e)}]
|
return [{"error": str(e)}]
|
||||||
@@ -62,7 +70,9 @@ def format_events(today_events: list, tomorrow_events: list = None) -> str:
|
|||||||
|
|
||||||
if "dateTime" in start:
|
if "dateTime" in start:
|
||||||
# Timed event
|
# Timed event
|
||||||
dt = datetime.fromisoformat(start["dateTime"].replace("Z", "+00:00"))
|
dt = datetime.fromisoformat(
|
||||||
|
start["dateTime"].replace("Z", "+00:00")
|
||||||
|
)
|
||||||
time_str = dt.strftime("%I:%M %p").lstrip("0")
|
time_str = dt.strftime("%I:%M %p").lstrip("0")
|
||||||
elif "date" in start:
|
elif "date" in start:
|
||||||
time_str = "All day"
|
time_str = "All day"
|
||||||
@@ -73,13 +83,19 @@ def format_events(today_events: list, tomorrow_events: list = None) -> str:
|
|||||||
# Calculate duration if end time available
|
# Calculate duration if end time available
|
||||||
end = event.get("end", {})
|
end = event.get("end", {})
|
||||||
if "dateTime" in start and "dateTime" in end:
|
if "dateTime" in start and "dateTime" in end:
|
||||||
start_dt = datetime.fromisoformat(start["dateTime"].replace("Z", "+00:00"))
|
start_dt = datetime.fromisoformat(
|
||||||
end_dt = datetime.fromisoformat(end["dateTime"].replace("Z", "+00:00"))
|
start["dateTime"].replace("Z", "+00:00")
|
||||||
|
)
|
||||||
|
end_dt = datetime.fromisoformat(
|
||||||
|
end["dateTime"].replace("Z", "+00:00")
|
||||||
|
)
|
||||||
mins = int((end_dt - start_dt).total_seconds() / 60)
|
mins = int((end_dt - start_dt).total_seconds() / 60)
|
||||||
if mins >= 60:
|
if mins >= 60:
|
||||||
hours = mins // 60
|
hours = mins // 60
|
||||||
remaining = mins % 60
|
remaining = mins % 60
|
||||||
duration = f" ({hours}h{remaining}m)" if remaining else f" ({hours}h)"
|
duration = (
|
||||||
|
f" ({hours}h{remaining}m)" if remaining else f" ({hours}h)"
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
duration = f" ({mins}m)"
|
duration = f" ({mins}m)"
|
||||||
|
|
||||||
@@ -92,17 +108,23 @@ def format_events(today_events: list, tomorrow_events: list = None) -> str:
|
|||||||
|
|
||||||
# Tomorrow preview
|
# Tomorrow preview
|
||||||
if tomorrow_events is not None:
|
if tomorrow_events is not None:
|
||||||
if tomorrow_events and (len(tomorrow_events) == 0 or "error" not in tomorrow_events[0]):
|
if tomorrow_events and (
|
||||||
|
len(tomorrow_events) == 0 or "error" not in tomorrow_events[0]
|
||||||
|
):
|
||||||
count = len(tomorrow_events)
|
count = len(tomorrow_events)
|
||||||
if count > 0:
|
if count > 0:
|
||||||
first = tomorrow_events[0]
|
first = tomorrow_events[0]
|
||||||
start = first.get("start", {})
|
start = first.get("start", {})
|
||||||
if "dateTime" in start:
|
if "dateTime" in start:
|
||||||
dt = datetime.fromisoformat(start["dateTime"].replace("Z", "+00:00"))
|
dt = datetime.fromisoformat(
|
||||||
|
start["dateTime"].replace("Z", "+00:00")
|
||||||
|
)
|
||||||
first_time = dt.strftime("%I:%M %p").lstrip("0")
|
first_time = dt.strftime("%I:%M %p").lstrip("0")
|
||||||
else:
|
else:
|
||||||
first_time = "All day"
|
first_time = "All day"
|
||||||
lines.append(f"Tomorrow: {count} event{'s' if count > 1 else ''}, first at {first_time}")
|
lines.append(
|
||||||
|
f"Tomorrow: {count} event{'s' if count > 1 else ''}, first at {first_time}"
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
lines.append("Tomorrow: No events")
|
lines.append("Tomorrow: No events")
|
||||||
|
|
||||||
@@ -126,7 +148,7 @@ def collect(config: dict) -> dict:
|
|||||||
"icon": "📅",
|
"icon": "📅",
|
||||||
"content": formatted,
|
"content": formatted,
|
||||||
"raw": {"today": today_events, "tomorrow": tomorrow_events},
|
"raw": {"today": today_events, "tomorrow": tomorrow_events},
|
||||||
"error": today_events[0].get("error") if has_error else None
|
"error": today_events[0].get("error") if has_error else None,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -11,37 +11,49 @@ from pathlib import Path
|
|||||||
def fetch_unread_emails(days: int = 7, max_results: int = 15) -> list:
|
def fetch_unread_emails(days: int = 7, max_results: int = 15) -> list:
|
||||||
"""Fetch unread emails directly using gmail_mcp library."""
|
"""Fetch unread emails directly using gmail_mcp library."""
|
||||||
# Set credentials path
|
# Set credentials path
|
||||||
os.environ.setdefault('GMAIL_CREDENTIALS_PATH', os.path.expanduser('~/.gmail-mcp/credentials.json'))
|
os.environ.setdefault(
|
||||||
|
"GMAIL_CREDENTIALS_PATH", os.path.expanduser("~/.gmail-mcp/credentials.json")
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Add gmail venv to path
|
# Add gmail venv to path
|
||||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.13/site-packages"
|
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.14/site-packages"
|
||||||
if str(venv_site) not in sys.path:
|
if str(venv_site) not in sys.path:
|
||||||
sys.path.insert(0, str(venv_site))
|
sys.path.insert(0, str(venv_site))
|
||||||
|
|
||||||
from gmail_mcp.utils.GCP.gmail_auth import get_gmail_service
|
from gmail_mcp.utils.GCP.gmail_auth import get_gmail_service
|
||||||
|
|
||||||
service = get_gmail_service()
|
service = get_gmail_service()
|
||||||
results = service.users().messages().list(
|
results = (
|
||||||
userId='me',
|
service.users()
|
||||||
q=f'is:unread newer_than:{days}d',
|
.messages()
|
||||||
maxResults=max_results
|
.list(
|
||||||
).execute()
|
userId="me", q=f"is:unread newer_than:{days}d", maxResults=max_results
|
||||||
|
)
|
||||||
|
.execute()
|
||||||
|
)
|
||||||
|
|
||||||
emails = []
|
emails = []
|
||||||
for msg in results.get('messages', []):
|
for msg in results.get("messages", []):
|
||||||
detail = service.users().messages().get(
|
detail = (
|
||||||
userId='me',
|
service.users()
|
||||||
id=msg['id'],
|
.messages()
|
||||||
format='metadata',
|
.get(
|
||||||
metadataHeaders=['From', 'Subject']
|
userId="me",
|
||||||
).execute()
|
id=msg["id"],
|
||||||
headers = {h['name']: h['value'] for h in detail['payload']['headers']}
|
format="metadata",
|
||||||
emails.append({
|
metadataHeaders=["From", "Subject"],
|
||||||
'from': headers.get('From', 'Unknown'),
|
)
|
||||||
'subject': headers.get('Subject', '(no subject)'),
|
.execute()
|
||||||
'id': msg['id']
|
)
|
||||||
})
|
headers = {h["name"]: h["value"] for h in detail["payload"]["headers"]}
|
||||||
|
emails.append(
|
||||||
|
{
|
||||||
|
"from": headers.get("From", "Unknown"),
|
||||||
|
"subject": headers.get("Subject", "(no subject)"),
|
||||||
|
"id": msg["id"],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
return emails
|
return emails
|
||||||
|
|
||||||
@@ -79,10 +91,17 @@ Output the formatted email section, nothing else."""
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
["/home/will/.local/bin/claude", "--print", "--model", "sonnet", "-p", prompt],
|
[
|
||||||
|
"/home/will/.local/bin/claude",
|
||||||
|
"--print",
|
||||||
|
"--model",
|
||||||
|
"sonnet",
|
||||||
|
"-p",
|
||||||
|
prompt,
|
||||||
|
],
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
text=True,
|
text=True,
|
||||||
timeout=60
|
timeout=60,
|
||||||
)
|
)
|
||||||
|
|
||||||
if result.returncode == 0 and result.stdout.strip():
|
if result.returncode == 0 and result.stdout.strip():
|
||||||
@@ -131,7 +150,7 @@ def collect(config: dict) -> dict:
|
|||||||
"content": formatted,
|
"content": formatted,
|
||||||
"raw": emails if not has_error else None,
|
"raw": emails if not has_error else None,
|
||||||
"count": len(emails) if not has_error else 0,
|
"count": len(emails) if not has_error else 0,
|
||||||
"error": emails[0].get("error") if has_error else None
|
"error": emails[0].get("error") if has_error else None,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ from datetime import datetime
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
# Add gmail venv to path for Google API libraries
|
# Add gmail venv to path for Google API libraries
|
||||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.13/site-packages"
|
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.14/site-packages"
|
||||||
if str(venv_site) not in sys.path:
|
if str(venv_site) not in sys.path:
|
||||||
sys.path.insert(0, str(venv_site))
|
sys.path.insert(0, str(venv_site))
|
||||||
|
|
||||||
@@ -18,6 +18,7 @@ try:
|
|||||||
from google_auth_oauthlib.flow import InstalledAppFlow
|
from google_auth_oauthlib.flow import InstalledAppFlow
|
||||||
from google.auth.transport.requests import Request
|
from google.auth.transport.requests import Request
|
||||||
from googleapiclient.discovery import build
|
from googleapiclient.discovery import build
|
||||||
|
|
||||||
GOOGLE_API_AVAILABLE = True
|
GOOGLE_API_AVAILABLE = True
|
||||||
except ImportError:
|
except ImportError:
|
||||||
GOOGLE_API_AVAILABLE = False
|
GOOGLE_API_AVAILABLE = False
|
||||||
@@ -57,7 +58,11 @@ def fetch_tasks(max_results: int = 10) -> list:
|
|||||||
try:
|
try:
|
||||||
creds = get_credentials()
|
creds = get_credentials()
|
||||||
if not creds:
|
if not creds:
|
||||||
return [{"error": "Tasks API not authenticated - run: ~/.claude/mcp/gmail/venv/bin/python ~/.claude/skills/morning-report/scripts/collectors/gtasks.py --auth"}]
|
return [
|
||||||
|
{
|
||||||
|
"error": "Tasks API not authenticated - run: ~/.claude/mcp/gmail/venv/bin/python ~/.claude/skills/morning-report/scripts/collectors/gtasks.py --auth"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
service = build("tasks", "v1", credentials=creds)
|
service = build("tasks", "v1", credentials=creds)
|
||||||
|
|
||||||
@@ -69,12 +74,16 @@ def fetch_tasks(max_results: int = 10) -> list:
|
|||||||
tasklist_id = tasklists["items"][0]["id"]
|
tasklist_id = tasklists["items"][0]["id"]
|
||||||
|
|
||||||
# Get tasks
|
# Get tasks
|
||||||
results = service.tasks().list(
|
results = (
|
||||||
tasklist=tasklist_id,
|
service.tasks()
|
||||||
maxResults=max_results,
|
.list(
|
||||||
showCompleted=False,
|
tasklist=tasklist_id,
|
||||||
showHidden=False
|
maxResults=max_results,
|
||||||
).execute()
|
showCompleted=False,
|
||||||
|
showHidden=False,
|
||||||
|
)
|
||||||
|
.execute()
|
||||||
|
)
|
||||||
|
|
||||||
tasks = results.get("items", [])
|
tasks = results.get("items", [])
|
||||||
return tasks
|
return tasks
|
||||||
@@ -150,7 +159,7 @@ def collect(config: dict) -> dict:
|
|||||||
"content": formatted,
|
"content": formatted,
|
||||||
"raw": tasks if not has_error else None,
|
"raw": tasks if not has_error else None,
|
||||||
"count": len(tasks) if not has_error else 0,
|
"count": len(tasks) if not has_error else 0,
|
||||||
"error": tasks[0].get("error") if has_error else None
|
"error": tasks[0].get("error") if has_error else None,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ from collectors import weather, stocks, infra, news
|
|||||||
# These may fail if gmail venv not activated
|
# These may fail if gmail venv not activated
|
||||||
try:
|
try:
|
||||||
from collectors import gmail, gcal, gtasks
|
from collectors import gmail, gcal, gtasks
|
||||||
|
|
||||||
GOOGLE_COLLECTORS = True
|
GOOGLE_COLLECTORS = True
|
||||||
except ImportError:
|
except ImportError:
|
||||||
GOOGLE_COLLECTORS = False
|
GOOGLE_COLLECTORS = False
|
||||||
@@ -29,10 +30,7 @@ LOG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
|||||||
logging.basicConfig(
|
logging.basicConfig(
|
||||||
level=logging.INFO,
|
level=logging.INFO,
|
||||||
format="%(asctime)s - %(levelname)s - %(message)s",
|
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||||
handlers=[
|
handlers=[logging.FileHandler(LOG_PATH), logging.StreamHandler()],
|
||||||
logging.FileHandler(LOG_PATH),
|
|
||||||
logging.StreamHandler()
|
|
||||||
]
|
|
||||||
)
|
)
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -58,10 +56,17 @@ def collect_section(name: str, collector_func, config: dict) -> dict:
|
|||||||
"section": name,
|
"section": name,
|
||||||
"icon": "❓",
|
"icon": "❓",
|
||||||
"content": f"⚠️ {name} unavailable: {e}",
|
"content": f"⚠️ {name} unavailable: {e}",
|
||||||
"error": str(e)
|
"error": str(e),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def is_section_enabled(name: str, config: dict) -> bool:
|
||||||
|
"""Check if a section is enabled in config."""
|
||||||
|
section_key = name.lower()
|
||||||
|
section_config = config.get(section_key, {})
|
||||||
|
return section_config.get("enabled", True)
|
||||||
|
|
||||||
|
|
||||||
def collect_all(config: dict) -> list:
|
def collect_all(config: dict) -> list:
|
||||||
"""Collect all sections in parallel."""
|
"""Collect all sections in parallel."""
|
||||||
collectors = [
|
collectors = [
|
||||||
@@ -72,11 +77,12 @@ def collect_all(config: dict) -> list:
|
|||||||
]
|
]
|
||||||
|
|
||||||
if GOOGLE_COLLECTORS:
|
if GOOGLE_COLLECTORS:
|
||||||
collectors.extend([
|
if is_section_enabled("email", config):
|
||||||
("Email", gmail.collect),
|
collectors.append(("Email", gmail.collect))
|
||||||
("Calendar", gcal.collect),
|
if is_section_enabled("calendar", config):
|
||||||
("Tasks", gtasks.collect),
|
collectors.append(("Calendar", gcal.collect))
|
||||||
])
|
if is_section_enabled("tasks", config):
|
||||||
|
collectors.append(("Tasks", gtasks.collect))
|
||||||
else:
|
else:
|
||||||
logger.warning("Google collectors not available - run with gmail venv")
|
logger.warning("Google collectors not available - run with gmail venv")
|
||||||
|
|
||||||
@@ -95,12 +101,14 @@ def collect_all(config: dict) -> list:
|
|||||||
results.append(result)
|
results.append(result)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Future {name} exception: {e}")
|
logger.error(f"Future {name} exception: {e}")
|
||||||
results.append({
|
results.append(
|
||||||
"section": name,
|
{
|
||||||
"icon": "❓",
|
"section": name,
|
||||||
"content": f"⚠️ {name} failed: {e}",
|
"icon": "❓",
|
||||||
"error": str(e)
|
"content": f"⚠️ {name} failed: {e}",
|
||||||
})
|
"error": str(e),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
return results
|
return results
|
||||||
|
|
||||||
@@ -111,13 +119,21 @@ def render_report(sections: list, config: dict) -> str:
|
|||||||
date_str = now.strftime("%a %b %d, %Y")
|
date_str = now.strftime("%a %b %d, %Y")
|
||||||
time_str = now.strftime("%I:%M %p %Z").strip()
|
time_str = now.strftime("%I:%M %p %Z").strip()
|
||||||
|
|
||||||
lines = [
|
lines = [f"# Morning Report - {date_str}", ""]
|
||||||
f"# Morning Report - {date_str}",
|
|
||||||
""
|
|
||||||
]
|
|
||||||
|
|
||||||
# Order sections
|
# Order sections
|
||||||
order = ["Weather", "Email", "Calendar", "Today", "Stocks", "Tasks", "Infra", "Infrastructure", "News", "Tech News"]
|
order = [
|
||||||
|
"Weather",
|
||||||
|
"Email",
|
||||||
|
"Calendar",
|
||||||
|
"Today",
|
||||||
|
"Stocks",
|
||||||
|
"Tasks",
|
||||||
|
"Infra",
|
||||||
|
"Infrastructure",
|
||||||
|
"News",
|
||||||
|
"Tech News",
|
||||||
|
]
|
||||||
|
|
||||||
# Sort by order
|
# Sort by order
|
||||||
section_map = {s.get("section", ""): s for s in sections}
|
section_map = {s.get("section", ""): s for s in sections}
|
||||||
@@ -137,10 +153,7 @@ def render_report(sections: list, config: dict) -> str:
|
|||||||
lines.append("")
|
lines.append("")
|
||||||
|
|
||||||
# Footer
|
# Footer
|
||||||
lines.extend([
|
lines.extend(["---", f"*Generated: {now.strftime('%Y-%m-%d %H:%M:%S')} PT*"])
|
||||||
"---",
|
|
||||||
f"*Generated: {now.strftime('%Y-%m-%d %H:%M:%S')} PT*"
|
|
||||||
])
|
|
||||||
|
|
||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|
||||||
@@ -148,7 +161,9 @@ def render_report(sections: list, config: dict) -> str:
|
|||||||
def save_report(content: str, config: dict) -> Path:
|
def save_report(content: str, config: dict) -> Path:
|
||||||
"""Save report to file and archive."""
|
"""Save report to file and archive."""
|
||||||
output_config = config.get("output", {})
|
output_config = config.get("output", {})
|
||||||
output_path = Path(output_config.get("path", "~/.claude/reports/morning.md")).expanduser()
|
output_path = Path(
|
||||||
|
output_config.get("path", "~/.claude/reports/morning.md")
|
||||||
|
).expanduser()
|
||||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
# Write main report
|
# Write main report
|
||||||
|
|||||||
@@ -0,0 +1,123 @@
|
|||||||
|
---
|
||||||
|
name: rag-search
|
||||||
|
description: Semantic search across personal state files and external documentation
|
||||||
|
triggers: [search, find, lookup, what did, how did, when did, past decisions, previous, documentation, docs]
|
||||||
|
---
|
||||||
|
|
||||||
|
# RAG Search Skill
|
||||||
|
|
||||||
|
Semantic search across two indexes:
|
||||||
|
- **personal**: Your state files, memory, decisions, preferences
|
||||||
|
- **docs**: External documentation (k0s, ArgoCD, etc.)
|
||||||
|
|
||||||
|
## When to Use
|
||||||
|
|
||||||
|
- "What decisions did I make about X?"
|
||||||
|
- "How did I configure Y?"
|
||||||
|
- "What does the k0s documentation say about Z?"
|
||||||
|
- "Find my past notes on..."
|
||||||
|
- Cross-referencing personal context with official docs
|
||||||
|
|
||||||
|
## Scripts
|
||||||
|
|
||||||
|
All scripts use the venv at `~/.claude/skills/rag-search/venv/`.
|
||||||
|
|
||||||
|
### Search (Primary Interface)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Search both indexes
|
||||||
|
~/.claude/skills/rag-search/venv/bin/python \
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py "query"
|
||||||
|
|
||||||
|
# Search specific index
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py --index personal "query"
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py --index docs "query"
|
||||||
|
|
||||||
|
# Control result count
|
||||||
|
~/.claude/skills/rag-search/scripts/search.py --top-k 10 "query"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Index Management
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Reindex personal state files
|
||||||
|
~/.claude/skills/rag-search/venv/bin/python \
|
||||||
|
~/.claude/skills/rag-search/scripts/index_personal.py
|
||||||
|
|
||||||
|
# Index all doc sources
|
||||||
|
~/.claude/skills/rag-search/venv/bin/python \
|
||||||
|
~/.claude/skills/rag-search/scripts/index_docs.py --all
|
||||||
|
|
||||||
|
# Index specific doc source
|
||||||
|
~/.claude/skills/rag-search/scripts/index_docs.py --source k0s
|
||||||
|
```
|
||||||
|
|
||||||
|
### Adding Doc Sources
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add a git-based doc source
|
||||||
|
~/.claude/skills/rag-search/venv/bin/python \
|
||||||
|
~/.claude/skills/rag-search/scripts/add_doc_source.py \
|
||||||
|
--id "argocd" \
|
||||||
|
--name "ArgoCD Documentation" \
|
||||||
|
--type git \
|
||||||
|
--url "https://github.com/argoproj/argo-cd.git" \
|
||||||
|
--path "docs/" \
|
||||||
|
--glob "**/*.md"
|
||||||
|
|
||||||
|
# List configured sources
|
||||||
|
~/.claude/skills/rag-search/scripts/add_doc_source.py --list
|
||||||
|
```
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
|
||||||
|
Search returns JSON:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"query": "your search query",
|
||||||
|
"results": [
|
||||||
|
{
|
||||||
|
"rank": 1,
|
||||||
|
"score": 0.847,
|
||||||
|
"source": "personal",
|
||||||
|
"file": "memory/decisions.json",
|
||||||
|
"chunk": "Relevant text content...",
|
||||||
|
"metadata": {"date": "2025-01-15"}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"searched_collections": ["personal", "docs"],
|
||||||
|
"total_chunks_searched": 1847
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Search Strategy
|
||||||
|
|
||||||
|
1. **Start broad** - Use general terms first
|
||||||
|
2. **Refine if needed** - Add specific keywords if results aren't relevant
|
||||||
|
3. **Cross-reference** - When both personal and docs results appear, synthesize them
|
||||||
|
4. **Cite sources** - Include file paths and dates in your answers
|
||||||
|
|
||||||
|
## Example Workflow
|
||||||
|
|
||||||
|
User asks: "How should I configure ArgoCD sync?"
|
||||||
|
|
||||||
|
1. Search both indexes:
|
||||||
|
```bash
|
||||||
|
search.py "ArgoCD sync configuration"
|
||||||
|
```
|
||||||
|
|
||||||
|
2. If personal results exist, prioritize those (user's past decisions)
|
||||||
|
|
||||||
|
3. Supplement with docs results for official guidance
|
||||||
|
|
||||||
|
4. Synthesize answer:
|
||||||
|
> Based on your previous decision (decisions.json, 2025-01-15), you configured ArgoCD with auto-sync enabled but self-heal disabled. The ArgoCD docs recommend this for production environments where you want automatic deployment but manual intervention for drift correction.
|
||||||
|
|
||||||
|
## Maintenance
|
||||||
|
|
||||||
|
Indexes should be refreshed periodically:
|
||||||
|
- Personal: After significant state changes
|
||||||
|
- Docs: After tool version upgrades
|
||||||
|
|
||||||
|
A systemd timer can automate this (see design doc for setup).
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"id": "k0s",
|
||||||
|
"name": "k0s Documentation",
|
||||||
|
"type": "git",
|
||||||
|
"url": "https://github.com/k0sproject/k0s.git",
|
||||||
|
"path": "docs/",
|
||||||
|
"glob": "**/*.md",
|
||||||
|
"version": "main",
|
||||||
|
"last_indexed": "2026-01-04T23:27:40.175671"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
Executable
+205
@@ -0,0 +1,205 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
RAG Search - Add Documentation Source
|
||||||
|
|
||||||
|
Adds a new documentation source to the registry.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Constants
|
||||||
|
SKILL_DIR = Path(__file__).parent.parent
|
||||||
|
SOURCES_FILE = SKILL_DIR / "references" / "sources.json"
|
||||||
|
|
||||||
|
|
||||||
|
def load_sources() -> list[dict]:
|
||||||
|
"""Load configured documentation sources."""
|
||||||
|
if not SOURCES_FILE.exists():
|
||||||
|
return []
|
||||||
|
with open(SOURCES_FILE) as f:
|
||||||
|
data = json.load(f)
|
||||||
|
return data.get("sources", [])
|
||||||
|
|
||||||
|
|
||||||
|
def save_sources(sources: list[dict]) -> None:
|
||||||
|
"""Save documentation sources."""
|
||||||
|
SOURCES_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(SOURCES_FILE, "w") as f:
|
||||||
|
json.dump({"sources": sources}, f, indent=2)
|
||||||
|
|
||||||
|
|
||||||
|
def add_source(
|
||||||
|
source_id: str,
|
||||||
|
name: str,
|
||||||
|
source_type: str,
|
||||||
|
url: str = None,
|
||||||
|
path: str = None,
|
||||||
|
glob: str = "**/*.md",
|
||||||
|
version: str = None,
|
||||||
|
base_url: str = None,
|
||||||
|
) -> dict:
|
||||||
|
"""
|
||||||
|
Add a new documentation source.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
source_id: Unique identifier for the source
|
||||||
|
name: Human-readable name
|
||||||
|
source_type: "git" or "local"
|
||||||
|
url: Git repository URL (for git type)
|
||||||
|
path: Path within repo or local path
|
||||||
|
glob: File pattern to match
|
||||||
|
version: Git tag/branch (for git type)
|
||||||
|
base_url: Base URL for documentation links
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The created source configuration
|
||||||
|
"""
|
||||||
|
sources = load_sources()
|
||||||
|
|
||||||
|
# Check for existing source
|
||||||
|
existing = [s for s in sources if s["id"] == source_id]
|
||||||
|
if existing:
|
||||||
|
raise ValueError(f"Source already exists: {source_id}")
|
||||||
|
|
||||||
|
# Build source config
|
||||||
|
source = {
|
||||||
|
"id": source_id,
|
||||||
|
"name": name,
|
||||||
|
"type": source_type,
|
||||||
|
}
|
||||||
|
|
||||||
|
if source_type == "git":
|
||||||
|
if not url:
|
||||||
|
raise ValueError("Git sources require --url")
|
||||||
|
source["url"] = url
|
||||||
|
if version:
|
||||||
|
source["version"] = version
|
||||||
|
elif source_type == "local":
|
||||||
|
if not path:
|
||||||
|
raise ValueError("Local sources require --path")
|
||||||
|
source["path"] = str(Path(path).expanduser())
|
||||||
|
else:
|
||||||
|
raise ValueError(f"Unknown source type: {source_type}")
|
||||||
|
|
||||||
|
if path and source_type == "git":
|
||||||
|
source["path"] = path
|
||||||
|
source["glob"] = glob
|
||||||
|
if base_url:
|
||||||
|
source["base_url"] = base_url
|
||||||
|
|
||||||
|
sources.append(source)
|
||||||
|
save_sources(sources)
|
||||||
|
|
||||||
|
return source
|
||||||
|
|
||||||
|
|
||||||
|
def remove_source(source_id: str) -> bool:
|
||||||
|
"""Remove a documentation source."""
|
||||||
|
sources = load_sources()
|
||||||
|
original_count = len(sources)
|
||||||
|
sources = [s for s in sources if s["id"] != source_id]
|
||||||
|
|
||||||
|
if len(sources) == original_count:
|
||||||
|
return False
|
||||||
|
|
||||||
|
save_sources(sources)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Add or manage documentation sources for RAG search",
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||||
|
epilog="""
|
||||||
|
Examples:
|
||||||
|
# Add k0s documentation from GitHub
|
||||||
|
%(prog)s --id k0s --name "k0s Documentation" --type git \\
|
||||||
|
--url "https://github.com/k0sproject/k0s.git" \\
|
||||||
|
--path "docs/" --version "v1.30.0"
|
||||||
|
|
||||||
|
# Add local documentation directory
|
||||||
|
%(prog)s --id internal --name "Internal Docs" --type local \\
|
||||||
|
--path "~/docs/internal" --glob "**/*.md"
|
||||||
|
|
||||||
|
# Remove a source
|
||||||
|
%(prog)s --remove k0s
|
||||||
|
|
||||||
|
# List sources
|
||||||
|
%(prog)s --list
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
parser.add_argument("--id", help="Unique source identifier")
|
||||||
|
parser.add_argument("--name", help="Human-readable name")
|
||||||
|
parser.add_argument(
|
||||||
|
"--type", "-t",
|
||||||
|
choices=["git", "local"],
|
||||||
|
default="git",
|
||||||
|
help="Source type (default: git)"
|
||||||
|
)
|
||||||
|
parser.add_argument("--url", help="Git repository URL")
|
||||||
|
parser.add_argument("--path", help="Path within repo or local directory")
|
||||||
|
parser.add_argument(
|
||||||
|
"--glob", "-g",
|
||||||
|
default="**/*.md",
|
||||||
|
help="File pattern to match (default: **/*.md)"
|
||||||
|
)
|
||||||
|
parser.add_argument("--version", "-v", help="Git tag or branch")
|
||||||
|
parser.add_argument("--base-url", help="Base URL for documentation links")
|
||||||
|
parser.add_argument(
|
||||||
|
"--remove", "-r",
|
||||||
|
metavar="ID",
|
||||||
|
help="Remove a source by ID"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--list", "-l",
|
||||||
|
action="store_true",
|
||||||
|
help="List configured sources"
|
||||||
|
)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.list:
|
||||||
|
sources = load_sources()
|
||||||
|
if sources:
|
||||||
|
print(json.dumps(sources, indent=2))
|
||||||
|
else:
|
||||||
|
print("No documentation sources configured")
|
||||||
|
return
|
||||||
|
|
||||||
|
if args.remove:
|
||||||
|
if remove_source(args.remove):
|
||||||
|
print(f"Removed source: {args.remove}")
|
||||||
|
else:
|
||||||
|
print(f"Source not found: {args.remove}", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Adding a new source
|
||||||
|
if not args.id or not args.name:
|
||||||
|
parser.error("--id and --name are required when adding a source")
|
||||||
|
|
||||||
|
try:
|
||||||
|
source = add_source(
|
||||||
|
source_id=args.id,
|
||||||
|
name=args.name,
|
||||||
|
source_type=args.type,
|
||||||
|
url=args.url,
|
||||||
|
path=args.path,
|
||||||
|
glob=args.glob,
|
||||||
|
version=args.version,
|
||||||
|
base_url=args.base_url,
|
||||||
|
)
|
||||||
|
print(f"Added source: {args.id}")
|
||||||
|
print(json.dumps(source, indent=2))
|
||||||
|
print(f"\nTo index this source, run:")
|
||||||
|
print(f" index_docs.py --source {args.id}")
|
||||||
|
except ValueError as e:
|
||||||
|
print(f"Error: {e}", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Executable
+419
@@ -0,0 +1,419 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
RAG Search - Documentation Index Builder
|
||||||
|
|
||||||
|
Indexes external documentation sources for semantic search.
|
||||||
|
Supports git repos and local directories.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Generator, Optional
|
||||||
|
|
||||||
|
# Add venv site-packages to path
|
||||||
|
VENV_PATH = Path(__file__).parent.parent / "venv" / "lib" / "python3.13" / "site-packages"
|
||||||
|
if str(VENV_PATH) not in sys.path:
|
||||||
|
sys.path.insert(0, str(VENV_PATH))
|
||||||
|
|
||||||
|
import chromadb
|
||||||
|
from sentence_transformers import SentenceTransformer
|
||||||
|
|
||||||
|
# Constants
|
||||||
|
SKILL_DIR = Path(__file__).parent.parent
|
||||||
|
SOURCES_FILE = SKILL_DIR / "references" / "sources.json"
|
||||||
|
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||||
|
CHROMA_DIR = DATA_DIR / "chroma"
|
||||||
|
DOCS_CACHE_DIR = DATA_DIR / "docs-cache"
|
||||||
|
MODEL_NAME = "all-MiniLM-L6-v2"
|
||||||
|
COLLECTION_NAME = "docs"
|
||||||
|
|
||||||
|
# Chunking parameters
|
||||||
|
CHUNK_SIZE = 500 # Target tokens (roughly 4 chars per token)
|
||||||
|
CHUNK_OVERLAP = 50
|
||||||
|
|
||||||
|
|
||||||
|
def load_sources() -> list[dict]:
|
||||||
|
"""Load configured documentation sources."""
|
||||||
|
if not SOURCES_FILE.exists():
|
||||||
|
return []
|
||||||
|
with open(SOURCES_FILE) as f:
|
||||||
|
data = json.load(f)
|
||||||
|
return data.get("sources", [])
|
||||||
|
|
||||||
|
|
||||||
|
def save_sources(sources: list[dict]) -> None:
|
||||||
|
"""Save documentation sources."""
|
||||||
|
SOURCES_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(SOURCES_FILE, "w") as f:
|
||||||
|
json.dump({"sources": sources}, f, indent=2)
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_git_source(source: dict, quiet: bool = False) -> Optional[Path]:
|
||||||
|
"""
|
||||||
|
Clone or update a git repository.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Path to the docs directory within the repo
|
||||||
|
"""
|
||||||
|
source_id = source["id"]
|
||||||
|
url = source["url"]
|
||||||
|
version = source.get("version", "HEAD")
|
||||||
|
doc_path = source.get("path", "")
|
||||||
|
|
||||||
|
cache_dir = DOCS_CACHE_DIR / source_id
|
||||||
|
|
||||||
|
if cache_dir.exists():
|
||||||
|
# Update existing repo
|
||||||
|
if not quiet:
|
||||||
|
print(f" Updating {source_id}...")
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
["git", "fetch", "--all"],
|
||||||
|
cwd=cache_dir,
|
||||||
|
capture_output=True,
|
||||||
|
check=True
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["git", "checkout", version],
|
||||||
|
cwd=cache_dir,
|
||||||
|
capture_output=True,
|
||||||
|
check=True
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["git", "pull", "--ff-only"],
|
||||||
|
cwd=cache_dir,
|
||||||
|
capture_output=True,
|
||||||
|
check=False # May fail on tags
|
||||||
|
)
|
||||||
|
except subprocess.CalledProcessError as e:
|
||||||
|
print(f" Warning: Could not update {source_id}: {e}", file=sys.stderr)
|
||||||
|
else:
|
||||||
|
# Clone new repo
|
||||||
|
if not quiet:
|
||||||
|
print(f" Cloning {source_id}...")
|
||||||
|
cache_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
["git", "clone", "--depth", "1", url, str(cache_dir)],
|
||||||
|
capture_output=True,
|
||||||
|
check=True
|
||||||
|
)
|
||||||
|
if version != "HEAD":
|
||||||
|
subprocess.run(
|
||||||
|
["git", "fetch", "--depth", "1", "origin", version],
|
||||||
|
cwd=cache_dir,
|
||||||
|
capture_output=True,
|
||||||
|
check=True
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["git", "checkout", version],
|
||||||
|
cwd=cache_dir,
|
||||||
|
capture_output=True,
|
||||||
|
check=True
|
||||||
|
)
|
||||||
|
except subprocess.CalledProcessError as e:
|
||||||
|
print(f" Error: Could not clone {source_id}: {e}", file=sys.stderr)
|
||||||
|
return None
|
||||||
|
|
||||||
|
docs_dir = cache_dir / doc_path if doc_path else cache_dir
|
||||||
|
return docs_dir if docs_dir.exists() else None
|
||||||
|
|
||||||
|
|
||||||
|
def chunk_markdown(content: str, file_path: str) -> Generator[tuple[str, dict], None, None]:
|
||||||
|
"""
|
||||||
|
Chunk markdown content for embedding.
|
||||||
|
|
||||||
|
Strategy:
|
||||||
|
- Split by headers to preserve context
|
||||||
|
- Chunk sections that are too long
|
||||||
|
- Preserve header hierarchy in metadata
|
||||||
|
"""
|
||||||
|
lines = content.split("\n")
|
||||||
|
current_chunk = []
|
||||||
|
current_headers = []
|
||||||
|
chunk_start_line = 0
|
||||||
|
|
||||||
|
def emit_chunk() -> Optional[tuple[str, dict]]:
|
||||||
|
if not current_chunk:
|
||||||
|
return None
|
||||||
|
text = "\n".join(current_chunk).strip()
|
||||||
|
if len(text) < 20:
|
||||||
|
return None
|
||||||
|
|
||||||
|
metadata = {
|
||||||
|
"file": file_path,
|
||||||
|
"headers": " > ".join(current_headers) if current_headers else ""
|
||||||
|
}
|
||||||
|
return (text, metadata)
|
||||||
|
|
||||||
|
for i, line in enumerate(lines):
|
||||||
|
# Check for header
|
||||||
|
header_match = re.match(r'^(#{1,6})\s+(.+)$', line)
|
||||||
|
|
||||||
|
if header_match:
|
||||||
|
# Emit current chunk before new header
|
||||||
|
chunk = emit_chunk()
|
||||||
|
if chunk:
|
||||||
|
yield chunk
|
||||||
|
current_chunk = []
|
||||||
|
|
||||||
|
# Update header hierarchy
|
||||||
|
level = len(header_match.group(1))
|
||||||
|
header_text = header_match.group(2).strip()
|
||||||
|
|
||||||
|
# Trim headers to current level
|
||||||
|
current_headers = current_headers[:level-1]
|
||||||
|
current_headers.append(header_text)
|
||||||
|
|
||||||
|
chunk_start_line = i
|
||||||
|
|
||||||
|
current_chunk.append(line)
|
||||||
|
|
||||||
|
# Check if chunk is getting too large (rough token estimate)
|
||||||
|
chunk_text = "\n".join(current_chunk)
|
||||||
|
if len(chunk_text) > CHUNK_SIZE * 4:
|
||||||
|
chunk = emit_chunk()
|
||||||
|
if chunk:
|
||||||
|
yield chunk
|
||||||
|
# Start new chunk with overlap
|
||||||
|
overlap_lines = current_chunk[-CHUNK_OVERLAP // 10:] if len(current_chunk) > CHUNK_OVERLAP // 10 else []
|
||||||
|
current_chunk = overlap_lines
|
||||||
|
|
||||||
|
# Emit final chunk
|
||||||
|
chunk = emit_chunk()
|
||||||
|
if chunk:
|
||||||
|
yield chunk
|
||||||
|
|
||||||
|
|
||||||
|
def index_source(
|
||||||
|
source: dict,
|
||||||
|
model: SentenceTransformer,
|
||||||
|
quiet: bool = False
|
||||||
|
) -> tuple[list[str], list[list[float]], list[dict], list[str]]:
|
||||||
|
"""
|
||||||
|
Index a single documentation source.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(chunks, embeddings, metadatas, ids)
|
||||||
|
"""
|
||||||
|
source_id = source["id"]
|
||||||
|
source_type = source.get("type", "git")
|
||||||
|
glob_pattern = source.get("glob", "**/*.md")
|
||||||
|
|
||||||
|
if source_type == "git":
|
||||||
|
docs_dir = fetch_git_source(source, quiet=quiet)
|
||||||
|
if not docs_dir:
|
||||||
|
return [], [], [], []
|
||||||
|
elif source_type == "local":
|
||||||
|
docs_dir = Path(source["path"]).expanduser()
|
||||||
|
if not docs_dir.exists():
|
||||||
|
print(f" Warning: Local path does not exist: {docs_dir}", file=sys.stderr)
|
||||||
|
return [], [], [], []
|
||||||
|
else:
|
||||||
|
print(f" Warning: Unknown source type: {source_type}", file=sys.stderr)
|
||||||
|
return [], [], [], []
|
||||||
|
|
||||||
|
chunks = []
|
||||||
|
metadatas = []
|
||||||
|
ids = []
|
||||||
|
|
||||||
|
# Find and process files
|
||||||
|
files = list(docs_dir.glob(glob_pattern))
|
||||||
|
if not quiet:
|
||||||
|
print(f" Found {len(files)} files matching {glob_pattern}")
|
||||||
|
|
||||||
|
for file_path in files:
|
||||||
|
try:
|
||||||
|
content = file_path.read_text(encoding="utf-8", errors="ignore")
|
||||||
|
except IOError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
rel_path = str(file_path.relative_to(docs_dir))
|
||||||
|
full_path = f"{source_id}/{rel_path}"
|
||||||
|
|
||||||
|
for chunk_text, metadata in chunk_markdown(content, full_path):
|
||||||
|
chunk_id = f"docs_{source_id}_{len(chunks)}"
|
||||||
|
chunks.append(chunk_text)
|
||||||
|
metadata["source_id"] = source_id
|
||||||
|
metadata["source_name"] = source.get("name", source_id)
|
||||||
|
if source.get("version"):
|
||||||
|
metadata["version"] = source["version"]
|
||||||
|
if source.get("base_url"):
|
||||||
|
metadata["url"] = source["base_url"]
|
||||||
|
metadatas.append(metadata)
|
||||||
|
ids.append(chunk_id)
|
||||||
|
|
||||||
|
if not quiet:
|
||||||
|
print(f" Indexed {len(chunks)} chunks from {source_id}")
|
||||||
|
|
||||||
|
return chunks, [], metadatas, ids
|
||||||
|
|
||||||
|
|
||||||
|
def index_docs(
|
||||||
|
source_id: Optional[str] = None,
|
||||||
|
all_sources: bool = False,
|
||||||
|
quiet: bool = False
|
||||||
|
) -> dict:
|
||||||
|
"""
|
||||||
|
Index documentation sources.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
source_id: Index only this source
|
||||||
|
all_sources: Index all configured sources
|
||||||
|
quiet: Suppress progress output
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Summary statistics
|
||||||
|
"""
|
||||||
|
sources = load_sources()
|
||||||
|
if not sources:
|
||||||
|
return {"error": "No documentation sources configured"}
|
||||||
|
|
||||||
|
# Filter sources
|
||||||
|
if source_id:
|
||||||
|
sources = [s for s in sources if s["id"] == source_id]
|
||||||
|
if not sources:
|
||||||
|
return {"error": f"Source not found: {source_id}"}
|
||||||
|
elif not all_sources:
|
||||||
|
return {"error": "Specify --source <id> or --all"}
|
||||||
|
|
||||||
|
if not quiet:
|
||||||
|
print(f"Indexing {len(sources)} documentation source(s)")
|
||||||
|
|
||||||
|
# Initialize model and client
|
||||||
|
model = SentenceTransformer(MODEL_NAME)
|
||||||
|
CHROMA_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
client = chromadb.PersistentClient(path=str(CHROMA_DIR))
|
||||||
|
|
||||||
|
# Get or create collection
|
||||||
|
try:
|
||||||
|
collection = client.get_collection(COLLECTION_NAME)
|
||||||
|
# If indexing all or specific source, we'll need to handle existing data
|
||||||
|
if all_sources:
|
||||||
|
client.delete_collection(COLLECTION_NAME)
|
||||||
|
collection = client.create_collection(
|
||||||
|
name=COLLECTION_NAME,
|
||||||
|
metadata={"description": "External documentation"}
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
collection = client.create_collection(
|
||||||
|
name=COLLECTION_NAME,
|
||||||
|
metadata={"description": "External documentation"}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Process each source
|
||||||
|
all_chunks = []
|
||||||
|
all_metadatas = []
|
||||||
|
all_ids = []
|
||||||
|
|
||||||
|
for source in sources:
|
||||||
|
if not quiet:
|
||||||
|
print(f"\nProcessing: {source['name']}")
|
||||||
|
|
||||||
|
chunks, _, metadatas, ids = index_source(source, model, quiet=quiet)
|
||||||
|
all_chunks.extend(chunks)
|
||||||
|
all_metadatas.extend(metadatas)
|
||||||
|
all_ids.extend(ids)
|
||||||
|
|
||||||
|
# Update last_indexed timestamp
|
||||||
|
source["last_indexed"] = datetime.now().isoformat()
|
||||||
|
|
||||||
|
# Batch embed and add to collection
|
||||||
|
if all_chunks:
|
||||||
|
if not quiet:
|
||||||
|
print(f"\nEmbedding {len(all_chunks)} chunks...")
|
||||||
|
|
||||||
|
embeddings = model.encode(all_chunks, show_progress_bar=not quiet).tolist()
|
||||||
|
|
||||||
|
# Add in batches
|
||||||
|
batch_size = 100
|
||||||
|
for i in range(0, len(all_chunks), batch_size):
|
||||||
|
end_idx = min(i + batch_size, len(all_chunks))
|
||||||
|
collection.add(
|
||||||
|
documents=all_chunks[i:end_idx],
|
||||||
|
embeddings=embeddings[i:end_idx],
|
||||||
|
metadatas=all_metadatas[i:end_idx],
|
||||||
|
ids=all_ids[i:end_idx]
|
||||||
|
)
|
||||||
|
|
||||||
|
# Save updated sources with timestamps
|
||||||
|
all_sources = load_sources()
|
||||||
|
for source in sources:
|
||||||
|
for s in all_sources:
|
||||||
|
if s["id"] == source["id"]:
|
||||||
|
s["last_indexed"] = source["last_indexed"]
|
||||||
|
break
|
||||||
|
save_sources(all_sources)
|
||||||
|
|
||||||
|
stats = {
|
||||||
|
"collection": COLLECTION_NAME,
|
||||||
|
"sources_processed": len(sources),
|
||||||
|
"chunks_indexed": len(all_chunks),
|
||||||
|
"indexed_at": datetime.now().isoformat()
|
||||||
|
}
|
||||||
|
|
||||||
|
if not quiet:
|
||||||
|
print(f"\nIndexed {len(all_chunks)} chunks from {len(sources)} source(s)")
|
||||||
|
|
||||||
|
return stats
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Index external documentation for RAG search"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--source", "-s",
|
||||||
|
help="Index only this source ID"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--all", "-a",
|
||||||
|
action="store_true",
|
||||||
|
dest="all_sources",
|
||||||
|
help="Index all configured sources"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--quiet", "-q",
|
||||||
|
action="store_true",
|
||||||
|
help="Suppress progress output"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--list", "-l",
|
||||||
|
action="store_true",
|
||||||
|
help="List configured sources"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--stats",
|
||||||
|
action="store_true",
|
||||||
|
help="Output stats as JSON"
|
||||||
|
)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.list:
|
||||||
|
sources = load_sources()
|
||||||
|
if sources:
|
||||||
|
print(json.dumps(sources, indent=2))
|
||||||
|
else:
|
||||||
|
print("No documentation sources configured")
|
||||||
|
print(f"Add sources with: add_doc_source.py")
|
||||||
|
return
|
||||||
|
|
||||||
|
stats = index_docs(
|
||||||
|
source_id=args.source,
|
||||||
|
all_sources=args.all_sources,
|
||||||
|
quiet=args.quiet
|
||||||
|
)
|
||||||
|
|
||||||
|
if args.stats or "error" in stats:
|
||||||
|
print(json.dumps(stats, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Executable
+286
@@ -0,0 +1,286 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
RAG Search - Personal Index Builder
|
||||||
|
|
||||||
|
Indexes ~/.claude/state files for semantic search.
|
||||||
|
Chunks JSON files by key for optimal retrieval.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Generator
|
||||||
|
|
||||||
|
# Add venv site-packages to path
|
||||||
|
VENV_PATH = Path(__file__).parent.parent / "venv" / "lib" / "python3.13" / "site-packages"
|
||||||
|
if str(VENV_PATH) not in sys.path:
|
||||||
|
sys.path.insert(0, str(VENV_PATH))
|
||||||
|
|
||||||
|
import chromadb
|
||||||
|
from sentence_transformers import SentenceTransformer
|
||||||
|
|
||||||
|
# Constants
|
||||||
|
STATE_DIR = Path.home() / ".claude" / "state"
|
||||||
|
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||||
|
CHROMA_DIR = DATA_DIR / "chroma"
|
||||||
|
MODEL_NAME = "all-MiniLM-L6-v2"
|
||||||
|
COLLECTION_NAME = "personal"
|
||||||
|
|
||||||
|
|
||||||
|
def chunk_json_file(file_path: Path) -> Generator[tuple[str, dict], None, None]:
|
||||||
|
"""
|
||||||
|
Chunk a JSON file into searchable segments.
|
||||||
|
|
||||||
|
Strategy:
|
||||||
|
- Arrays: Each item becomes a chunk
|
||||||
|
- Objects with arrays: Each array item with parent context
|
||||||
|
- Nested objects: Flatten with path prefix
|
||||||
|
|
||||||
|
Yields:
|
||||||
|
(chunk_text, metadata) tuples
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with open(file_path) as f:
|
||||||
|
data = json.load(f)
|
||||||
|
except (json.JSONDecodeError, IOError) as e:
|
||||||
|
print(f" Warning: Could not parse {file_path}: {e}", file=sys.stderr)
|
||||||
|
return
|
||||||
|
|
||||||
|
rel_path = str(file_path.relative_to(STATE_DIR))
|
||||||
|
base_metadata = {"file": rel_path}
|
||||||
|
|
||||||
|
def process_item(item: dict, context: str = "") -> Generator[tuple[str, dict], None, None]:
|
||||||
|
"""Process a single item from JSON structure."""
|
||||||
|
if isinstance(item, dict):
|
||||||
|
# Check for common patterns in our state files
|
||||||
|
|
||||||
|
# Memory items (decisions, preferences, facts, projects)
|
||||||
|
if "content" in item:
|
||||||
|
text_parts = []
|
||||||
|
if context:
|
||||||
|
text_parts.append(f"[{context}]")
|
||||||
|
text_parts.append(item.get("content", ""))
|
||||||
|
if item.get("context"):
|
||||||
|
text_parts.append(f"Context: {item['context']}")
|
||||||
|
if item.get("rationale"):
|
||||||
|
text_parts.append(f"Rationale: {item['rationale']}")
|
||||||
|
|
||||||
|
metadata = {**base_metadata}
|
||||||
|
if item.get("date"):
|
||||||
|
metadata["date"] = item["date"]
|
||||||
|
if item.get("id"):
|
||||||
|
metadata["id"] = item["id"]
|
||||||
|
if item.get("status"):
|
||||||
|
metadata["status"] = item["status"]
|
||||||
|
|
||||||
|
yield (" ".join(text_parts), metadata)
|
||||||
|
return
|
||||||
|
|
||||||
|
# General instructions (memory)
|
||||||
|
if "instruction" in item:
|
||||||
|
text_parts = [item["instruction"]]
|
||||||
|
metadata = {**base_metadata}
|
||||||
|
if item.get("added"):
|
||||||
|
metadata["date"] = item["added"]
|
||||||
|
if item.get("status"):
|
||||||
|
metadata["status"] = item["status"]
|
||||||
|
yield (" ".join(text_parts), metadata)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Knowledge base entries
|
||||||
|
if "fact" in item or "answer" in item:
|
||||||
|
text = item.get("fact") or item.get("answer", "")
|
||||||
|
if item.get("question"):
|
||||||
|
text = f"Q: {item['question']} A: {text}"
|
||||||
|
metadata = {**base_metadata}
|
||||||
|
if item.get("category"):
|
||||||
|
metadata["category"] = item["category"]
|
||||||
|
yield (text, metadata)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Component registry entries
|
||||||
|
if "name" in item and "description" in item:
|
||||||
|
text = f"{item['name']}: {item['description']}"
|
||||||
|
if item.get("triggers"):
|
||||||
|
text += f" Triggers: {', '.join(item['triggers'])}"
|
||||||
|
metadata = {**base_metadata, "type": item.get("type", "unknown")}
|
||||||
|
yield (text, metadata)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Future considerations
|
||||||
|
if "id" in item and "title" in item:
|
||||||
|
text = f"{item.get('id', '')}: {item['title']}"
|
||||||
|
if item.get("description"):
|
||||||
|
text += f" - {item['description']}"
|
||||||
|
if item.get("rationale"):
|
||||||
|
text += f" Rationale: {item['rationale']}"
|
||||||
|
metadata = {**base_metadata}
|
||||||
|
if item.get("date_added"):
|
||||||
|
metadata["date"] = item["date_added"]
|
||||||
|
if item.get("status"):
|
||||||
|
metadata["status"] = item["status"]
|
||||||
|
yield (text, metadata)
|
||||||
|
return
|
||||||
|
|
||||||
|
# System instructions - processes
|
||||||
|
if "process" in item or "name" in item:
|
||||||
|
parts = []
|
||||||
|
if item.get("name"):
|
||||||
|
parts.append(item["name"])
|
||||||
|
if item.get("description"):
|
||||||
|
parts.append(item["description"])
|
||||||
|
if item.get("steps"):
|
||||||
|
parts.append("Steps: " + " ".join(item["steps"]))
|
||||||
|
if parts:
|
||||||
|
yield (" - ".join(parts), {**base_metadata})
|
||||||
|
return
|
||||||
|
|
||||||
|
# Fallback: stringify the whole object
|
||||||
|
text = json.dumps(item, indent=None)
|
||||||
|
if len(text) > 50: # Only index if substantial
|
||||||
|
yield (text[:1000], {**base_metadata}) # Truncate very long items
|
||||||
|
|
||||||
|
elif isinstance(item, str) and len(item) > 20:
|
||||||
|
yield (item, {**base_metadata})
|
||||||
|
|
||||||
|
# Process top-level structure
|
||||||
|
if isinstance(data, list):
|
||||||
|
for item in data:
|
||||||
|
yield from process_item(item)
|
||||||
|
elif isinstance(data, dict):
|
||||||
|
# Handle nested arrays within objects
|
||||||
|
for key, value in data.items():
|
||||||
|
if isinstance(value, list):
|
||||||
|
for item in value:
|
||||||
|
yield from process_item(item, context=key)
|
||||||
|
elif isinstance(value, dict):
|
||||||
|
yield from process_item(value, context=key)
|
||||||
|
elif isinstance(value, str) and len(value) > 20:
|
||||||
|
yield (f"{key}: {value}", {**base_metadata})
|
||||||
|
|
||||||
|
|
||||||
|
def find_json_files() -> list[Path]:
|
||||||
|
"""Find all JSON files in the state directory."""
|
||||||
|
files = []
|
||||||
|
for pattern in ["*.json", "**/*.json"]:
|
||||||
|
files.extend(STATE_DIR.glob(pattern))
|
||||||
|
return sorted(set(files))
|
||||||
|
|
||||||
|
|
||||||
|
def index_personal(quiet: bool = False, force: bool = False) -> dict:
|
||||||
|
"""
|
||||||
|
Index all personal state files.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
quiet: Suppress progress output
|
||||||
|
force: Force reindex even if already exists
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Summary statistics
|
||||||
|
"""
|
||||||
|
if not quiet:
|
||||||
|
print(f"Indexing personal state from {STATE_DIR}")
|
||||||
|
|
||||||
|
# Initialize model and client
|
||||||
|
model = SentenceTransformer(MODEL_NAME)
|
||||||
|
CHROMA_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
client = chromadb.PersistentClient(path=str(CHROMA_DIR))
|
||||||
|
|
||||||
|
# Delete and recreate collection for clean reindex
|
||||||
|
try:
|
||||||
|
client.delete_collection(COLLECTION_NAME)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
collection = client.create_collection(
|
||||||
|
name=COLLECTION_NAME,
|
||||||
|
metadata={"description": "Personal state files from ~/.claude/state"}
|
||||||
|
)
|
||||||
|
|
||||||
|
# Find and process files
|
||||||
|
files = find_json_files()
|
||||||
|
if not quiet:
|
||||||
|
print(f"Found {len(files)} JSON files")
|
||||||
|
|
||||||
|
total_chunks = 0
|
||||||
|
chunks = []
|
||||||
|
metadatas = []
|
||||||
|
ids = []
|
||||||
|
|
||||||
|
for file_path in files:
|
||||||
|
if not quiet:
|
||||||
|
print(f" Processing: {file_path.relative_to(STATE_DIR)}")
|
||||||
|
|
||||||
|
for chunk_text, metadata in chunk_json_file(file_path):
|
||||||
|
# Skip empty or very short chunks
|
||||||
|
if not chunk_text or len(chunk_text.strip()) < 10:
|
||||||
|
continue
|
||||||
|
|
||||||
|
chunk_id = f"personal_{total_chunks}"
|
||||||
|
chunks.append(chunk_text)
|
||||||
|
metadatas.append(metadata)
|
||||||
|
ids.append(chunk_id)
|
||||||
|
total_chunks += 1
|
||||||
|
|
||||||
|
# Batch embed and add to collection
|
||||||
|
if chunks:
|
||||||
|
if not quiet:
|
||||||
|
print(f"Embedding {len(chunks)} chunks...")
|
||||||
|
|
||||||
|
embeddings = model.encode(chunks, show_progress_bar=not quiet).tolist()
|
||||||
|
|
||||||
|
# Add in batches (ChromaDB has limits)
|
||||||
|
batch_size = 100
|
||||||
|
for i in range(0, len(chunks), batch_size):
|
||||||
|
end_idx = min(i + batch_size, len(chunks))
|
||||||
|
collection.add(
|
||||||
|
documents=chunks[i:end_idx],
|
||||||
|
embeddings=embeddings[i:end_idx],
|
||||||
|
metadatas=metadatas[i:end_idx],
|
||||||
|
ids=ids[i:end_idx]
|
||||||
|
)
|
||||||
|
|
||||||
|
stats = {
|
||||||
|
"collection": COLLECTION_NAME,
|
||||||
|
"files_processed": len(files),
|
||||||
|
"chunks_indexed": total_chunks,
|
||||||
|
"indexed_at": datetime.now().isoformat()
|
||||||
|
}
|
||||||
|
|
||||||
|
if not quiet:
|
||||||
|
print(f"\nIndexed {total_chunks} chunks from {len(files)} files")
|
||||||
|
|
||||||
|
return stats
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Index personal state files for RAG search"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--quiet", "-q",
|
||||||
|
action="store_true",
|
||||||
|
help="Suppress progress output"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--force", "-f",
|
||||||
|
action="store_true",
|
||||||
|
help="Force reindex even if already indexed"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--stats",
|
||||||
|
action="store_true",
|
||||||
|
help="Output stats as JSON"
|
||||||
|
)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
stats = index_personal(quiet=args.quiet, force=args.force)
|
||||||
|
|
||||||
|
if args.stats:
|
||||||
|
print(json.dumps(stats, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Executable
+184
@@ -0,0 +1,184 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
RAG Search - Main search entry point
|
||||||
|
|
||||||
|
Searches personal and/or docs indexes for semantically similar content.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
# Add venv site-packages to path
|
||||||
|
VENV_PATH = Path(__file__).parent.parent / "venv" / "lib" / "python3.13" / "site-packages"
|
||||||
|
if str(VENV_PATH) not in sys.path:
|
||||||
|
sys.path.insert(0, str(VENV_PATH))
|
||||||
|
|
||||||
|
import chromadb
|
||||||
|
from sentence_transformers import SentenceTransformer
|
||||||
|
|
||||||
|
# Constants
|
||||||
|
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||||
|
CHROMA_DIR = DATA_DIR / "chroma"
|
||||||
|
MODEL_NAME = "all-MiniLM-L6-v2"
|
||||||
|
DEFAULT_TOP_K = 5
|
||||||
|
|
||||||
|
# Lazy-loaded globals
|
||||||
|
_model: Optional[SentenceTransformer] = None
|
||||||
|
_client: Optional[chromadb.PersistentClient] = None
|
||||||
|
|
||||||
|
|
||||||
|
def get_model() -> SentenceTransformer:
|
||||||
|
"""Lazy-load the embedding model."""
|
||||||
|
global _model
|
||||||
|
if _model is None:
|
||||||
|
_model = SentenceTransformer(MODEL_NAME)
|
||||||
|
return _model
|
||||||
|
|
||||||
|
|
||||||
|
def get_client() -> chromadb.PersistentClient:
|
||||||
|
"""Lazy-load the ChromaDB client."""
|
||||||
|
global _client
|
||||||
|
if _client is None:
|
||||||
|
CHROMA_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
|
_client = chromadb.PersistentClient(path=str(CHROMA_DIR))
|
||||||
|
return _client
|
||||||
|
|
||||||
|
|
||||||
|
def search(
|
||||||
|
query: str,
|
||||||
|
index: Optional[str] = None,
|
||||||
|
top_k: int = DEFAULT_TOP_K,
|
||||||
|
) -> dict:
|
||||||
|
"""
|
||||||
|
Search for semantically similar content.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
query: The search query
|
||||||
|
index: Which index to search ("personal", "docs", or None for both)
|
||||||
|
top_k: Number of results to return per collection
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
dict with query, results, and metadata
|
||||||
|
"""
|
||||||
|
client = get_client()
|
||||||
|
model = get_model()
|
||||||
|
|
||||||
|
# Embed the query
|
||||||
|
query_embedding = model.encode(query).tolist()
|
||||||
|
|
||||||
|
# Determine which collections to search
|
||||||
|
collections_to_search = []
|
||||||
|
if index is None or index == "personal":
|
||||||
|
try:
|
||||||
|
collections_to_search.append(("personal", client.get_collection("personal")))
|
||||||
|
except Exception:
|
||||||
|
pass # Collection doesn't exist
|
||||||
|
if index is None or index == "docs":
|
||||||
|
try:
|
||||||
|
collections_to_search.append(("docs", client.get_collection("docs")))
|
||||||
|
except Exception:
|
||||||
|
pass # Collection doesn't exist
|
||||||
|
|
||||||
|
if not collections_to_search:
|
||||||
|
return {
|
||||||
|
"query": query,
|
||||||
|
"results": [],
|
||||||
|
"searched_collections": [],
|
||||||
|
"total_chunks_searched": 0,
|
||||||
|
"error": f"No collections found for index: {index or 'any'}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Search each collection
|
||||||
|
all_results = []
|
||||||
|
total_chunks = 0
|
||||||
|
searched_collections = []
|
||||||
|
|
||||||
|
for coll_name, collection in collections_to_search:
|
||||||
|
searched_collections.append(coll_name)
|
||||||
|
count = collection.count()
|
||||||
|
total_chunks += count
|
||||||
|
|
||||||
|
if count == 0:
|
||||||
|
continue
|
||||||
|
|
||||||
|
results = collection.query(
|
||||||
|
query_embeddings=[query_embedding],
|
||||||
|
n_results=min(top_k, count),
|
||||||
|
include=["documents", "metadatas", "distances"]
|
||||||
|
)
|
||||||
|
|
||||||
|
# Process results
|
||||||
|
if results["documents"] and results["documents"][0]:
|
||||||
|
for i, (doc, metadata, distance) in enumerate(zip(
|
||||||
|
results["documents"][0],
|
||||||
|
results["metadatas"][0],
|
||||||
|
results["distances"][0]
|
||||||
|
)):
|
||||||
|
# Convert distance to similarity score (cosine distance to similarity)
|
||||||
|
score = 1 - (distance / 2) # Normalized for cosine distance
|
||||||
|
all_results.append({
|
||||||
|
"source": coll_name,
|
||||||
|
"file": metadata.get("file", "unknown"),
|
||||||
|
"chunk": doc,
|
||||||
|
"score": round(score, 3),
|
||||||
|
"metadata": {k: v for k, v in metadata.items() if k != "file"}
|
||||||
|
})
|
||||||
|
|
||||||
|
# Sort by score and add ranks
|
||||||
|
all_results.sort(key=lambda x: x["score"], reverse=True)
|
||||||
|
for i, result in enumerate(all_results[:top_k]):
|
||||||
|
result["rank"] = i + 1
|
||||||
|
|
||||||
|
return {
|
||||||
|
"query": query,
|
||||||
|
"results": all_results[:top_k],
|
||||||
|
"searched_collections": searched_collections,
|
||||||
|
"total_chunks_searched": total_chunks
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Search the RAG index for relevant content",
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||||
|
epilog="""
|
||||||
|
Examples:
|
||||||
|
%(prog)s "how did I configure ArgoCD sync?"
|
||||||
|
%(prog)s --index personal "past decisions about caching"
|
||||||
|
%(prog)s --index docs "k0s node maintenance"
|
||||||
|
%(prog)s --top-k 10 "prometheus alerting rules"
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
parser.add_argument("query", help="Search query")
|
||||||
|
parser.add_argument(
|
||||||
|
"--index", "-i",
|
||||||
|
choices=["personal", "docs"],
|
||||||
|
help="Search only this index (default: both)"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--top-k", "-k",
|
||||||
|
type=int,
|
||||||
|
default=DEFAULT_TOP_K,
|
||||||
|
help=f"Number of results to return (default: {DEFAULT_TOP_K})"
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--raw",
|
||||||
|
action="store_true",
|
||||||
|
help="Output raw JSON (default: formatted)"
|
||||||
|
)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
results = search(args.query, args.index, args.top_k)
|
||||||
|
|
||||||
|
if args.raw:
|
||||||
|
print(json.dumps(results))
|
||||||
|
else:
|
||||||
|
print(json.dumps(results, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Executable
+230
@@ -0,0 +1,230 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
RAG Search - Test Suite
|
||||||
|
|
||||||
|
Tests all components of the RAG search skill.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Constants
|
||||||
|
SKILL_DIR = Path(__file__).parent.parent
|
||||||
|
SCRIPTS_DIR = SKILL_DIR / "scripts"
|
||||||
|
VENV_PYTHON = SKILL_DIR / "venv" / "bin" / "python"
|
||||||
|
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||||
|
|
||||||
|
|
||||||
|
def run_script(script_name: str, args: list[str] = None) -> tuple[int, str, str]:
|
||||||
|
"""Run a script and return (returncode, stdout, stderr)."""
|
||||||
|
cmd = [str(VENV_PYTHON), str(SCRIPTS_DIR / script_name)]
|
||||||
|
if args:
|
||||||
|
cmd.extend(args)
|
||||||
|
|
||||||
|
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
return result.returncode, result.stdout, result.stderr
|
||||||
|
|
||||||
|
|
||||||
|
def test_chromadb_embeddings():
|
||||||
|
"""Test 1: ChromaDB + embeddings working."""
|
||||||
|
print("Test 1: ChromaDB + embeddings...")
|
||||||
|
|
||||||
|
# Add venv to path and test imports
|
||||||
|
venv_path = SKILL_DIR / "venv" / "lib" / "python3.13" / "site-packages"
|
||||||
|
sys.path.insert(0, str(venv_path))
|
||||||
|
|
||||||
|
try:
|
||||||
|
import chromadb
|
||||||
|
from sentence_transformers import SentenceTransformer
|
||||||
|
|
||||||
|
# Test ChromaDB
|
||||||
|
client = chromadb.PersistentClient(path=str(DATA_DIR / "chroma"))
|
||||||
|
assert client is not None, "Failed to create ChromaDB client"
|
||||||
|
|
||||||
|
# Test embedding model
|
||||||
|
model = SentenceTransformer("all-MiniLM-L6-v2")
|
||||||
|
embedding = model.encode("test query")
|
||||||
|
assert len(embedding) == 384, f"Expected 384 dimensions, got {len(embedding)}"
|
||||||
|
|
||||||
|
print(" PASS: ChromaDB and embeddings working")
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
print(f" FAIL: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_personal_index():
|
||||||
|
"""Test 2: Personal index populated from ~/.claude/state."""
|
||||||
|
print("Test 2: Personal index populated...")
|
||||||
|
|
||||||
|
# Check if collection exists and has data
|
||||||
|
venv_path = SKILL_DIR / "venv" / "lib" / "python3.13" / "site-packages"
|
||||||
|
if str(venv_path) not in sys.path:
|
||||||
|
sys.path.insert(0, str(venv_path))
|
||||||
|
|
||||||
|
try:
|
||||||
|
import chromadb
|
||||||
|
|
||||||
|
client = chromadb.PersistentClient(path=str(DATA_DIR / "chroma"))
|
||||||
|
collection = client.get_collection("personal")
|
||||||
|
count = collection.count()
|
||||||
|
|
||||||
|
assert count > 0, f"Personal collection is empty (count={count})"
|
||||||
|
print(f" PASS: Personal index has {count} chunks")
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
print(f" FAIL: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_docs_index():
|
||||||
|
"""Test 3: At least one external doc source indexed."""
|
||||||
|
print("Test 3: External docs indexed...")
|
||||||
|
|
||||||
|
# Check if collection exists and has data
|
||||||
|
venv_path = SKILL_DIR / "venv" / "lib" / "python3.13" / "site-packages"
|
||||||
|
if str(venv_path) not in sys.path:
|
||||||
|
sys.path.insert(0, str(venv_path))
|
||||||
|
|
||||||
|
try:
|
||||||
|
import chromadb
|
||||||
|
|
||||||
|
client = chromadb.PersistentClient(path=str(DATA_DIR / "chroma"))
|
||||||
|
collection = client.get_collection("docs")
|
||||||
|
count = collection.count()
|
||||||
|
|
||||||
|
assert count > 0, f"Docs collection is empty (count={count})"
|
||||||
|
|
||||||
|
# Also verify sources.json has at least one source
|
||||||
|
sources_file = SKILL_DIR / "references" / "sources.json"
|
||||||
|
with open(sources_file) as f:
|
||||||
|
sources = json.load(f)
|
||||||
|
assert len(sources.get("sources", [])) > 0, "No sources configured"
|
||||||
|
|
||||||
|
print(f" PASS: Docs index has {count} chunks from {len(sources['sources'])} source(s)")
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
print(f" FAIL: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_returns_results():
|
||||||
|
"""Test 4: search.py returns relevant results."""
|
||||||
|
print("Test 4: Search returns relevant results...")
|
||||||
|
|
||||||
|
# Test personal search
|
||||||
|
returncode, stdout, stderr = run_script("search.py", ["--index", "personal", "decisions"])
|
||||||
|
if returncode != 0:
|
||||||
|
print(f" FAIL: Personal search failed: {stderr}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = json.loads(stdout)
|
||||||
|
personal_results = result.get("results", [])
|
||||||
|
if not personal_results:
|
||||||
|
print(" WARN: No personal results found (may be expected if state is minimal)")
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
print(f" FAIL: Invalid JSON output: {stdout}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Test docs search
|
||||||
|
returncode, stdout, stderr = run_script("search.py", ["--index", "docs", "kubernetes"])
|
||||||
|
if returncode != 0:
|
||||||
|
print(f" FAIL: Docs search failed: {stderr}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = json.loads(stdout)
|
||||||
|
docs_results = result.get("results", [])
|
||||||
|
if not docs_results:
|
||||||
|
print(" FAIL: No docs results found for 'kubernetes'")
|
||||||
|
return False
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
print(f" FAIL: Invalid JSON output: {stdout}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Test combined search
|
||||||
|
returncode, stdout, stderr = run_script("search.py", ["configuration"])
|
||||||
|
if returncode != 0:
|
||||||
|
print(f" FAIL: Combined search failed: {stderr}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = json.loads(stdout)
|
||||||
|
assert "query" in result, "Missing 'query' in output"
|
||||||
|
assert "results" in result, "Missing 'results' in output"
|
||||||
|
assert "searched_collections" in result, "Missing 'searched_collections'"
|
||||||
|
assert len(result["searched_collections"]) == 2, "Should search both collections"
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
print(f" FAIL: Invalid JSON output: {stdout}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
print(f" PASS: Search returns properly formatted results")
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def test_skill_structure():
|
||||||
|
"""Test 5: All required files exist."""
|
||||||
|
print("Test 5: Skill structure complete...")
|
||||||
|
|
||||||
|
required_files = [
|
||||||
|
SKILL_DIR / "SKILL.md",
|
||||||
|
SCRIPTS_DIR / "search.py",
|
||||||
|
SCRIPTS_DIR / "index_personal.py",
|
||||||
|
SCRIPTS_DIR / "index_docs.py",
|
||||||
|
SCRIPTS_DIR / "add_doc_source.py",
|
||||||
|
SKILL_DIR / "references" / "sources.json",
|
||||||
|
]
|
||||||
|
|
||||||
|
missing = []
|
||||||
|
for f in required_files:
|
||||||
|
if not f.exists():
|
||||||
|
missing.append(str(f.relative_to(SKILL_DIR)))
|
||||||
|
|
||||||
|
if missing:
|
||||||
|
print(f" FAIL: Missing files: {', '.join(missing)}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
print(" PASS: All required files exist")
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
print("=" * 60)
|
||||||
|
print("RAG Search Test Suite")
|
||||||
|
print("=" * 60)
|
||||||
|
print()
|
||||||
|
|
||||||
|
tests = [
|
||||||
|
test_chromadb_embeddings,
|
||||||
|
test_personal_index,
|
||||||
|
test_docs_index,
|
||||||
|
test_search_returns_results,
|
||||||
|
test_skill_structure,
|
||||||
|
]
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for test in tests:
|
||||||
|
results.append(test())
|
||||||
|
print()
|
||||||
|
|
||||||
|
print("=" * 60)
|
||||||
|
print("Summary")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
passed = sum(results)
|
||||||
|
total = len(results)
|
||||||
|
print(f"Passed: {passed}/{total}")
|
||||||
|
|
||||||
|
if passed == total:
|
||||||
|
print("\nAll tests passed!")
|
||||||
|
return 0
|
||||||
|
else:
|
||||||
|
print(f"\n{total - passed} test(s) failed")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
+314
-154
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"version": "1.1",
|
"version": "1.0",
|
||||||
"generated": "2026-01-01T10:30:00.000000-08:00",
|
"generated": "2026-01-04T14:29:44.138959-08:00",
|
||||||
"description": "Component registry for PA session awareness. Read at session start for routing.",
|
"description": "Component registry for PA session awareness. Read at session start for routing.",
|
||||||
"skills": {
|
"skills": {
|
||||||
"sysadmin-health": {
|
"sysadmin-health": {
|
||||||
@@ -78,6 +78,18 @@
|
|||||||
"history"
|
"history"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
"morning-report": {
|
||||||
|
"description": "Generate daily morning dashboard with email, calendar, stocks, weather, tasks, infra, and news",
|
||||||
|
"script": "~/.claude/skills/morning-report/scripts/generate.py",
|
||||||
|
"triggers": [
|
||||||
|
"morning report",
|
||||||
|
"morning",
|
||||||
|
"daily report",
|
||||||
|
"dashboard",
|
||||||
|
"briefing",
|
||||||
|
"daily briefing"
|
||||||
|
]
|
||||||
|
},
|
||||||
"stock-lookup": {
|
"stock-lookup": {
|
||||||
"description": "Look up stock prices and quotes",
|
"description": "Look up stock prices and quotes",
|
||||||
"script": "~/.claude/skills/stock-lookup/scripts/quote.py",
|
"script": "~/.claude/skills/stock-lookup/scripts/quote.py",
|
||||||
@@ -92,23 +104,32 @@
|
|||||||
"performance"
|
"performance"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"morning-report": {
|
"rag-search": {
|
||||||
"description": "Generate daily morning dashboard with email, calendar, stocks, weather, tasks, infra, and news",
|
"description": "Semantic search across personal state files and external documentation (k0s, etc.)",
|
||||||
"script": "~/.claude/skills/morning-report/scripts/generate.py",
|
"script": "~/.claude/skills/rag-search/scripts/search.py",
|
||||||
"triggers": [
|
"triggers": [
|
||||||
"morning report",
|
"search",
|
||||||
"morning",
|
"find",
|
||||||
"daily report",
|
"lookup",
|
||||||
"dashboard",
|
"what did",
|
||||||
"briefing",
|
"how did",
|
||||||
"daily briefing"
|
"when did",
|
||||||
|
"past decisions",
|
||||||
|
"previous",
|
||||||
|
"documentation",
|
||||||
|
"docs",
|
||||||
|
"remember",
|
||||||
|
"history"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"commands": {
|
"commands": {
|
||||||
"/pa": {
|
"/pa": {
|
||||||
"description": "Personal assistant entrypoint",
|
"description": "Personal assistant entrypoint",
|
||||||
"aliases": ["/assistant", "/ask"],
|
"aliases": [
|
||||||
|
"/assistant",
|
||||||
|
"/ask"
|
||||||
|
],
|
||||||
"invokes": "agent:personal-assistant"
|
"invokes": "agent:personal-assistant"
|
||||||
},
|
},
|
||||||
"/programmer": {
|
"/programmer": {
|
||||||
@@ -118,24 +139,160 @@
|
|||||||
},
|
},
|
||||||
"/gcal": {
|
"/gcal": {
|
||||||
"description": "Google Calendar access",
|
"description": "Google Calendar access",
|
||||||
"aliases": ["/calendar", "/cal"],
|
"aliases": [
|
||||||
|
"/calendar",
|
||||||
|
"/cal"
|
||||||
|
],
|
||||||
"invokes": "skill:gcal"
|
"invokes": "skill:gcal"
|
||||||
},
|
},
|
||||||
"/stock": {
|
|
||||||
"description": "Stock price lookup",
|
|
||||||
"aliases": ["/quote", "/ticker"],
|
|
||||||
"invokes": "skill:stock-lookup"
|
|
||||||
},
|
|
||||||
"/morning": {
|
|
||||||
"description": "Generate morning report dashboard",
|
|
||||||
"aliases": ["/briefing", "/daily"],
|
|
||||||
"invokes": "skill:morning-report"
|
|
||||||
},
|
|
||||||
"/usage": {
|
"/usage": {
|
||||||
"description": "View usage statistics",
|
"description": "View usage statistics",
|
||||||
"aliases": ["/stats"],
|
"aliases": [
|
||||||
|
"/stats"
|
||||||
|
],
|
||||||
"invokes": "skill:usage"
|
"invokes": "skill:usage"
|
||||||
},
|
},
|
||||||
|
"/README": {
|
||||||
|
"description": "TODO",
|
||||||
|
"aliases": [],
|
||||||
|
"invokes": ""
|
||||||
|
},
|
||||||
|
"/agent-info": {
|
||||||
|
"description": "Show agent information",
|
||||||
|
"aliases": [
|
||||||
|
"/agent",
|
||||||
|
"/agents"
|
||||||
|
],
|
||||||
|
"invokes": "command:agent-info"
|
||||||
|
},
|
||||||
|
"/config": {
|
||||||
|
"description": "View and manage configuration settings",
|
||||||
|
"aliases": [
|
||||||
|
"/settings",
|
||||||
|
"/prefs"
|
||||||
|
],
|
||||||
|
"invokes": "command:config"
|
||||||
|
},
|
||||||
|
"/debug": {
|
||||||
|
"description": "Debug and troubleshoot configuration",
|
||||||
|
"aliases": [
|
||||||
|
"/diag",
|
||||||
|
"/diagnose"
|
||||||
|
],
|
||||||
|
"invokes": "command:debug"
|
||||||
|
},
|
||||||
|
"/diff": {
|
||||||
|
"description": "Compare config with backup",
|
||||||
|
"aliases": [
|
||||||
|
"/config-diff",
|
||||||
|
"/compare"
|
||||||
|
],
|
||||||
|
"invokes": "command:diff"
|
||||||
|
},
|
||||||
|
"/export": {
|
||||||
|
"description": "Export session data for sharing",
|
||||||
|
"aliases": [
|
||||||
|
"/session-export",
|
||||||
|
"/share"
|
||||||
|
],
|
||||||
|
"invokes": "command:export"
|
||||||
|
},
|
||||||
|
"/help": {
|
||||||
|
"description": "Show available commands and skills",
|
||||||
|
"aliases": [
|
||||||
|
"/commands",
|
||||||
|
"/skills"
|
||||||
|
],
|
||||||
|
"invokes": "command:help"
|
||||||
|
},
|
||||||
|
"/log": {
|
||||||
|
"description": "View and analyze logs",
|
||||||
|
"aliases": [
|
||||||
|
"/logs",
|
||||||
|
"/logview"
|
||||||
|
],
|
||||||
|
"invokes": "command:log"
|
||||||
|
},
|
||||||
|
"/maintain": {
|
||||||
|
"description": "Configuration maintenance (backup, validate, etc.)",
|
||||||
|
"aliases": [
|
||||||
|
"/maintenance",
|
||||||
|
"/admin"
|
||||||
|
],
|
||||||
|
"invokes": "command:maintain"
|
||||||
|
},
|
||||||
|
"/mcp-status": {
|
||||||
|
"description": "Check MCP integration status",
|
||||||
|
"aliases": [
|
||||||
|
"/mcp",
|
||||||
|
"/integrations"
|
||||||
|
],
|
||||||
|
"invokes": "command:mcp-status"
|
||||||
|
},
|
||||||
|
"/remember": {
|
||||||
|
"description": "Quick shortcut to save something to memory",
|
||||||
|
"aliases": [
|
||||||
|
"/save",
|
||||||
|
"/note"
|
||||||
|
],
|
||||||
|
"invokes": "command:remember"
|
||||||
|
},
|
||||||
|
"/search": {
|
||||||
|
"description": "Search memory, history, and configuration",
|
||||||
|
"aliases": [
|
||||||
|
"/find",
|
||||||
|
"/lookup"
|
||||||
|
],
|
||||||
|
"invokes": "command:search"
|
||||||
|
},
|
||||||
|
"/rag": {
|
||||||
|
"description": "Semantic search across state files and documentation",
|
||||||
|
"aliases": [
|
||||||
|
"/rag-search",
|
||||||
|
"/semantic-search"
|
||||||
|
],
|
||||||
|
"invokes": "skill:rag-search"
|
||||||
|
},
|
||||||
|
"/skill-info": {
|
||||||
|
"description": "Show skill information",
|
||||||
|
"aliases": [
|
||||||
|
"/skill",
|
||||||
|
"/skills-info"
|
||||||
|
],
|
||||||
|
"invokes": "command:skill-info"
|
||||||
|
},
|
||||||
|
"/status": {
|
||||||
|
"description": "Quick status overview across all domains",
|
||||||
|
"aliases": [
|
||||||
|
"/overview",
|
||||||
|
"/dashboard"
|
||||||
|
],
|
||||||
|
"invokes": "command:status"
|
||||||
|
},
|
||||||
|
"/summarize": {
|
||||||
|
"description": "Summarize and save session to memory",
|
||||||
|
"aliases": [
|
||||||
|
"/save-session",
|
||||||
|
"/session-summary"
|
||||||
|
],
|
||||||
|
"invokes": "command:summarize"
|
||||||
|
},
|
||||||
|
"/template": {
|
||||||
|
"description": "Manage session templates",
|
||||||
|
"aliases": [
|
||||||
|
"/templates",
|
||||||
|
"/session-template"
|
||||||
|
],
|
||||||
|
"invokes": "command:template"
|
||||||
|
},
|
||||||
|
"/workflow": {
|
||||||
|
"description": "List and describe workflows",
|
||||||
|
"aliases": [
|
||||||
|
"/workflows",
|
||||||
|
"/wf"
|
||||||
|
],
|
||||||
|
"invokes": "command:workflow"
|
||||||
|
},
|
||||||
"/sysadmin:health": {
|
"/sysadmin:health": {
|
||||||
"description": "System health check",
|
"description": "System health check",
|
||||||
"aliases": [],
|
"aliases": [],
|
||||||
@@ -166,137 +323,125 @@
|
|||||||
"aliases": [],
|
"aliases": [],
|
||||||
"invokes": "agent:k8s-diagnostician"
|
"invokes": "agent:k8s-diagnostician"
|
||||||
},
|
},
|
||||||
"/help": {
|
"/stock": {
|
||||||
"description": "Show available commands and skills",
|
"description": "Stock price lookup",
|
||||||
"aliases": ["/commands", "/skills"],
|
"aliases": [
|
||||||
"invokes": "command:help"
|
"/quote",
|
||||||
|
"/ticker"
|
||||||
|
],
|
||||||
|
"invokes": "skill:stock-lookup",
|
||||||
|
"status": "removed"
|
||||||
},
|
},
|
||||||
"/status": {
|
"/morning": {
|
||||||
"description": "Quick status overview across all domains",
|
"description": "Generate morning report dashboard",
|
||||||
"aliases": ["/overview", "/dashboard"],
|
"aliases": [
|
||||||
"invokes": "command:status"
|
"/briefing",
|
||||||
},
|
"/daily"
|
||||||
"/summarize": {
|
],
|
||||||
"description": "Summarize and save session to memory",
|
"invokes": "skill:morning-report",
|
||||||
"aliases": ["/save-session", "/session-summary"],
|
"status": "removed"
|
||||||
"invokes": "command:summarize"
|
|
||||||
},
|
|
||||||
"/maintain": {
|
|
||||||
"description": "Configuration maintenance (backup, validate, etc.)",
|
|
||||||
"aliases": ["/maintenance", "/admin"],
|
|
||||||
"invokes": "command:maintain"
|
|
||||||
},
|
|
||||||
"/remember": {
|
|
||||||
"description": "Quick shortcut to save something to memory",
|
|
||||||
"aliases": ["/save", "/note"],
|
|
||||||
"invokes": "command:remember"
|
|
||||||
},
|
|
||||||
"/config": {
|
|
||||||
"description": "View and manage configuration settings",
|
|
||||||
"aliases": ["/settings", "/prefs"],
|
|
||||||
"invokes": "command:config"
|
|
||||||
},
|
|
||||||
"/search": {
|
|
||||||
"description": "Search memory, history, and configuration",
|
|
||||||
"aliases": ["/find", "/lookup"],
|
|
||||||
"invokes": "command:search"
|
|
||||||
},
|
|
||||||
"/log": {
|
|
||||||
"description": "View and analyze logs",
|
|
||||||
"aliases": ["/logs", "/logview"],
|
|
||||||
"invokes": "command:log"
|
|
||||||
},
|
|
||||||
"/debug": {
|
|
||||||
"description": "Debug and troubleshoot configuration",
|
|
||||||
"aliases": ["/diag", "/diagnose"],
|
|
||||||
"invokes": "command:debug"
|
|
||||||
},
|
|
||||||
"/export": {
|
|
||||||
"description": "Export session data for sharing",
|
|
||||||
"aliases": ["/session-export", "/share"],
|
|
||||||
"invokes": "command:export"
|
|
||||||
},
|
|
||||||
"/mcp-status": {
|
|
||||||
"description": "Check MCP integration status",
|
|
||||||
"aliases": ["/mcp", "/integrations"],
|
|
||||||
"invokes": "command:mcp-status"
|
|
||||||
},
|
|
||||||
"/workflow": {
|
|
||||||
"description": "List and describe workflows",
|
|
||||||
"aliases": ["/workflows", "/wf"],
|
|
||||||
"invokes": "command:workflow"
|
|
||||||
},
|
|
||||||
"/skill-info": {
|
|
||||||
"description": "Show skill information",
|
|
||||||
"aliases": ["/skill", "/skills-info"],
|
|
||||||
"invokes": "command:skill-info"
|
|
||||||
},
|
|
||||||
"/agent-info": {
|
|
||||||
"description": "Show agent information",
|
|
||||||
"aliases": ["/agent", "/agents"],
|
|
||||||
"invokes": "command:agent-info"
|
|
||||||
},
|
|
||||||
"/diff": {
|
|
||||||
"description": "Compare config with backup",
|
|
||||||
"aliases": ["/config-diff", "/compare"],
|
|
||||||
"invokes": "command:diff"
|
|
||||||
},
|
|
||||||
"/template": {
|
|
||||||
"description": "Manage session templates",
|
|
||||||
"aliases": ["/templates", "/session-template"],
|
|
||||||
"invokes": "command:template"
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"agents": {
|
"agents": {
|
||||||
"linux-sysadmin": {
|
"linux-sysadmin": {
|
||||||
"description": "Workstation management",
|
"description": "Workstation management",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["system", "linux", "package", "service", "disk", "process"]
|
"triggers": [
|
||||||
|
"system",
|
||||||
|
"linux",
|
||||||
|
"package",
|
||||||
|
"service",
|
||||||
|
"disk",
|
||||||
|
"process"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"k8s-orchestrator": {
|
"k8s-orchestrator": {
|
||||||
"description": "Kubernetes cluster management",
|
"description": "Kubernetes cluster management",
|
||||||
"model": "opus",
|
"model": "opus",
|
||||||
"triggers": ["kubernetes", "k8s", "cluster", "deploy"]
|
"triggers": [
|
||||||
|
"kubernetes",
|
||||||
|
"k8s",
|
||||||
|
"cluster",
|
||||||
|
"deploy"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"k8s-diagnostician": {
|
"k8s-diagnostician": {
|
||||||
"description": "Kubernetes troubleshooting",
|
"description": "Kubernetes troubleshooting",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["pod issue", "crashloop", "k8s error", "deployment failed"]
|
"triggers": [
|
||||||
|
"pod issue",
|
||||||
|
"crashloop",
|
||||||
|
"k8s error",
|
||||||
|
"deployment failed"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"argocd-operator": {
|
"argocd-operator": {
|
||||||
"description": "ArgoCD GitOps operations",
|
"description": "ArgoCD GitOps operations",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["argocd", "gitops", "sync", "app sync"]
|
"triggers": [
|
||||||
|
"argocd",
|
||||||
|
"gitops",
|
||||||
|
"sync",
|
||||||
|
"app sync"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"prometheus-analyst": {
|
"prometheus-analyst": {
|
||||||
"description": "Metrics and alerting analysis",
|
"description": "Metrics and alerting analysis",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["metrics", "prometheus", "alert", "grafana"]
|
"triggers": [
|
||||||
|
"metrics",
|
||||||
|
"prometheus",
|
||||||
|
"alert",
|
||||||
|
"grafana"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"git-operator": {
|
"git-operator": {
|
||||||
"description": "Git repository operations",
|
"description": "Git repository operations",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["git", "commit", "branch", "merge", "repo"]
|
"triggers": [
|
||||||
|
"git",
|
||||||
|
"commit",
|
||||||
|
"branch",
|
||||||
|
"merge",
|
||||||
|
"repo"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"programmer-orchestrator": {
|
"programmer-orchestrator": {
|
||||||
"description": "Code development coordination",
|
"description": "Code development coordination",
|
||||||
"model": "opus",
|
"model": "opus",
|
||||||
"triggers": ["code", "develop", "implement", "program"]
|
"triggers": [
|
||||||
|
"code",
|
||||||
|
"develop",
|
||||||
|
"implement",
|
||||||
|
"program"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"code-planner": {
|
"code-planner": {
|
||||||
"description": "Code planning and design",
|
"description": "Code planning and design",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["plan code", "design", "architecture"]
|
"triggers": [
|
||||||
|
"plan code",
|
||||||
|
"design",
|
||||||
|
"architecture"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"code-implementer": {
|
"code-implementer": {
|
||||||
"description": "Code implementation",
|
"description": "Code implementation",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["write code", "implement", "build"]
|
"triggers": [
|
||||||
|
"write code",
|
||||||
|
"implement",
|
||||||
|
"build"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"code-reviewer": {
|
"code-reviewer": {
|
||||||
"description": "Code review",
|
"description": "Code review",
|
||||||
"model": "sonnet",
|
"model": "sonnet",
|
||||||
"triggers": ["review", "code review", "check code"]
|
"triggers": [
|
||||||
|
"review",
|
||||||
|
"code review",
|
||||||
|
"check code"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"master-orchestrator": {
|
"master-orchestrator": {
|
||||||
"description": "Coordinate and enforce policies",
|
"description": "Coordinate and enforce policies",
|
||||||
@@ -306,49 +451,94 @@
|
|||||||
"personal-assistant": {
|
"personal-assistant": {
|
||||||
"description": "User interface, ultimate oversight",
|
"description": "User interface, ultimate oversight",
|
||||||
"model": "opus",
|
"model": "opus",
|
||||||
"triggers": ["help", "assist", "question"]
|
"triggers": [
|
||||||
|
"help",
|
||||||
|
"assist",
|
||||||
|
"question"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"README": {
|
||||||
|
"description": "TODO",
|
||||||
|
"model": "sonnet",
|
||||||
|
"triggers": [
|
||||||
|
"TODO"
|
||||||
|
]
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"workflows": {
|
"workflows": {
|
||||||
"validate-agent-format": {
|
"validate-agent-format": {
|
||||||
"description": "Validate agent file format",
|
"description": "Validate agent file format",
|
||||||
"triggers": ["validate agent", "check agent format"]
|
"triggers": [
|
||||||
|
"validate agent",
|
||||||
|
"check agent format"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"health/cluster-health-check": {
|
"health/cluster-health-check": {
|
||||||
"description": "Kubernetes cluster health check",
|
"description": "Kubernetes cluster health check",
|
||||||
"triggers": ["cluster health", "k8s health"]
|
"triggers": [
|
||||||
|
"cluster health",
|
||||||
|
"k8s health"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"health/cluster-daily-summary": {
|
"health/cluster-daily-summary": {
|
||||||
"description": "Daily cluster health summary",
|
"description": "Daily cluster health summary",
|
||||||
"triggers": ["daily summary", "cluster summary"]
|
"triggers": [
|
||||||
|
"daily summary",
|
||||||
|
"cluster summary"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"deploy/deploy-app": {
|
"deploy/deploy-app": {
|
||||||
"description": "Deploy application to Kubernetes",
|
"description": "Deploy application to Kubernetes",
|
||||||
"triggers": ["deploy app", "deploy to k8s"]
|
"triggers": [
|
||||||
|
"deploy app",
|
||||||
|
"deploy to k8s"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"incidents/pod-crashloop": {
|
"incidents/pod-crashloop": {
|
||||||
"description": "Handle pod crashloop",
|
"description": "Handle pod crashloop",
|
||||||
"triggers": ["crashloop", "pod crashing", "restart loop"]
|
"triggers": [
|
||||||
|
"crashloop",
|
||||||
|
"pod crashing",
|
||||||
|
"restart loop"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"incidents/node-issue-response": {
|
"incidents/node-issue-response": {
|
||||||
"description": "Respond to node issues",
|
"description": "Respond to node issues",
|
||||||
"triggers": ["node issue", "node down", "node problem"]
|
"triggers": [
|
||||||
|
"node issue",
|
||||||
|
"node down",
|
||||||
|
"node problem"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"incidents/resource-pressure-response": {
|
"incidents/resource-pressure-response": {
|
||||||
"description": "Handle resource pressure",
|
"description": "Handle resource pressure",
|
||||||
"triggers": ["resource pressure", "out of memory", "disk full"]
|
"triggers": [
|
||||||
|
"resource pressure",
|
||||||
|
"out of memory",
|
||||||
|
"disk full"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"incidents/argocd-sync-failure": {
|
"incidents/argocd-sync-failure": {
|
||||||
"description": "Handle ArgoCD sync failures",
|
"description": "Handle ArgoCD sync failures",
|
||||||
"triggers": ["sync failed", "argocd error"]
|
"triggers": [
|
||||||
|
"sync failed",
|
||||||
|
"argocd error"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"sysadmin/health-check": {
|
"sysadmin/health-check": {
|
||||||
"description": "System health check workflow",
|
"description": "System health check workflow",
|
||||||
"triggers": ["system check", "health check"]
|
"triggers": [
|
||||||
|
"system check",
|
||||||
|
"health check"
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"sysadmin/system-update": {
|
"sysadmin/system-update": {
|
||||||
"description": "System update workflow",
|
"description": "System update workflow",
|
||||||
"triggers": ["system update", "update packages", "upgrade"]
|
"triggers": [
|
||||||
|
"system update",
|
||||||
|
"update packages",
|
||||||
|
"upgrade"
|
||||||
|
]
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"delegation_helpers": {
|
"delegation_helpers": {
|
||||||
@@ -360,35 +550,5 @@
|
|||||||
"description": "Calendar API with tiered delegation",
|
"description": "Calendar API with tiered delegation",
|
||||||
"location": "~/.claude/mcp/delegation/gcal_delegate.py"
|
"location": "~/.claude/mcp/delegation/gcal_delegate.py"
|
||||||
}
|
}
|
||||||
},
|
|
||||||
"automation": {
|
|
||||||
"scripts": {
|
|
||||||
"validate-setup": "~/.claude/automation/validate-setup.sh",
|
|
||||||
"quick-status": "~/.claude/automation/quick-status.sh",
|
|
||||||
"backup": "~/.claude/automation/backup.sh",
|
|
||||||
"restore": "~/.claude/automation/restore.sh",
|
|
||||||
"clean": "~/.claude/automation/clean.sh",
|
|
||||||
"install": "~/.claude/automation/install.sh",
|
|
||||||
"test": "~/.claude/automation/test-scripts.sh",
|
|
||||||
"memory-add": "~/.claude/automation/memory-add.py",
|
|
||||||
"memory-list": "~/.claude/automation/memory-list.py",
|
|
||||||
"search": "~/.claude/automation/search.py",
|
|
||||||
"history-browser": "~/.claude/automation/history-browser.py",
|
|
||||||
"log-viewer": "~/.claude/automation/log-viewer.py",
|
|
||||||
"debug": "~/.claude/automation/debug.sh",
|
|
||||||
"daily-maintenance": "~/.claude/automation/daily-maintenance.sh",
|
|
||||||
"session-export": "~/.claude/automation/session-export.py",
|
|
||||||
"mcp-status": "~/.claude/automation/mcp-status.sh",
|
|
||||||
"upgrade": "~/.claude/automation/upgrade.sh",
|
|
||||||
"workflow-info": "~/.claude/automation/workflow-info.py",
|
|
||||||
"skill-info": "~/.claude/automation/skill-info.py",
|
|
||||||
"agent-info": "~/.claude/automation/agent-info.py",
|
|
||||||
"config-diff": "~/.claude/automation/config-diff.py",
|
|
||||||
"session-template": "~/.claude/automation/session-template.py"
|
|
||||||
},
|
|
||||||
"completions": {
|
|
||||||
"bash": "~/.claude/automation/completions.bash",
|
|
||||||
"zsh": "~/.claude/automation/completions.zsh"
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
File diff suppressed because one or more lines are too long
@@ -189,6 +189,41 @@
|
|||||||
"ended": null,
|
"ended": null,
|
||||||
"summarized": false,
|
"summarized": false,
|
||||||
"topics": []
|
"topics": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "2026-01-04_13-30-26",
|
||||||
|
"started": "2026-01-04T13:30:26-08:00",
|
||||||
|
"ended": null,
|
||||||
|
"summarized": false,
|
||||||
|
"topics": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "2026-01-04_14-20-18",
|
||||||
|
"started": "2026-01-04T14:20:18-08:00",
|
||||||
|
"ended": null,
|
||||||
|
"summarized": false,
|
||||||
|
"topics": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "2026-01-04_23-21-33",
|
||||||
|
"started": "2026-01-04T23:21:33-08:00",
|
||||||
|
"ended": null,
|
||||||
|
"summarized": false,
|
||||||
|
"topics": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "2026-01-04_23-22-43",
|
||||||
|
"started": "2026-01-04T23:22:43-08:00",
|
||||||
|
"ended": null,
|
||||||
|
"summarized": false,
|
||||||
|
"topics": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "2026-01-04_23-22-43",
|
||||||
|
"started": "2026-01-04T23:22:43-08:00",
|
||||||
|
"ended": null,
|
||||||
|
"summarized": false,
|
||||||
|
"topics": []
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user