Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 OpenCode Test
|
f3cb082c36 | ||
|
OpenCode Test
|
db0d9f97b2 | ||
|
OpenCode Test
|
94603b19a5 | ||
|
OpenCode Test
|
45b7e4bcf7 | ||
|
OpenCode Test
|
7ca8caeecb | ||
|
OpenCode Test
|
c21b152de8 |
@@ -45,3 +45,7 @@ tmp_unused
|
||||
# Todos (managed by Claude Code)
|
||||
todos/
|
||||
repos/homelab
|
||||
|
||||
# RAG search data (generated vector stores and caches)
|
||||
data/
|
||||
skills/rag-search/venv/
|
||||
|
||||
@@ -0,0 +1,388 @@
|
||||
# Agentic RAG Design
|
||||
|
||||
**Date:** 2025-01-21
|
||||
**Status:** Ready for implementation
|
||||
**Category:** Agent memory / Knowledge retrieval
|
||||
|
||||
## Overview
|
||||
|
||||
Add semantic search to the existing Claude agent system, enabling multi-source reasoning that combines personal context (state files, memory, decisions) with external documentation.
|
||||
|
||||
### Goals
|
||||
|
||||
- Retrieve relevant past decisions and preferences when answering questions
|
||||
- Search external docs (k0s, ArgoCD, Prometheus, etc.) for technical reference
|
||||
- Cross-reference personal context with official documentation
|
||||
- Support iterative query refinement (agentic behavior)
|
||||
|
||||
### Non-Goals (Future Considerations)
|
||||
|
||||
Deferred to `future-considerations.json`:
|
||||
|
||||
- **fc-043**: Auto-sync on tool version change
|
||||
- **fc-044**: Broad doc indexing (hundreds of sources)
|
||||
- **fc-045**: K8s deployment
|
||||
- **fc-046**: Query caching
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
User question
|
||||
│
|
||||
▼
|
||||
Personal Assistant (existing)
|
||||
│
|
||||
├── Decides if RAG would help
|
||||
│
|
||||
▼
|
||||
rag-search skill (new)
|
||||
│
|
||||
├── Query embedding
|
||||
├── Vector similarity search
|
||||
├── Return ranked chunks with metadata
|
||||
│
|
||||
▼
|
||||
Claude reasons over results
|
||||
│
|
||||
├── Good enough? → Answer
|
||||
└── Need more? → Reformulate, search again
|
||||
```
|
||||
|
||||
### Two Indexes
|
||||
|
||||
| Index | Contents | Update Frequency |
|
||||
|-------|----------|------------------|
|
||||
| **personal** | `~/.claude/state/` files, memory, decisions, preferences | Daily |
|
||||
| **docs** | External documentation (k0s, ArgoCD, etc.) | Daily |
|
||||
|
||||
### Why Two Indexes
|
||||
|
||||
- Different update frequencies
|
||||
- Different retrieval strategies (personal may weight recency)
|
||||
- Can query one or both depending on the question
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ rag-search skill │
|
||||
│ (Claude invokes this) │
|
||||
└─────────────────────┬───────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┴─────────────┐
|
||||
▼ ▼
|
||||
┌───────────────────┐ ┌───────────────────┐
|
||||
│ Personal Index │ │ Docs Index │
|
||||
│ │ │ │
|
||||
│ ~/.claude/state/* │ │ External docs │
|
||||
│ memory/*.json │ │ (k0s, ArgoCD...) │
|
||||
│ kb.json │ │ │
|
||||
└────────┬──────────┘ └────────┬──────────┘
|
||||
│ │
|
||||
└──────────┬──────────────┘
|
||||
▼
|
||||
┌───────────────────┐
|
||||
│ Vector Store │
|
||||
│ (ChromaDB) │
|
||||
│ │
|
||||
│ Collections: │
|
||||
│ - personal │
|
||||
│ - docs │
|
||||
└────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────┐
|
||||
│ Embedding Model │
|
||||
│ (sentence- │
|
||||
│ transformers) │
|
||||
└───────────────────┘
|
||||
```
|
||||
|
||||
### Stack
|
||||
|
||||
| Component | Choice | Notes |
|
||||
|-----------|--------|-------|
|
||||
| Vector store | ChromaDB | Pure Python, no external deps |
|
||||
| Embeddings | sentence-transformers (all-MiniLM-L6-v2) | Runs on arm64, ~90MB |
|
||||
| Storage | `~/.claude/data/rag-search/` | Local to workstation |
|
||||
|
||||
## Skill Structure
|
||||
|
||||
**Location:** `~/.claude/skills/rag-search/`
|
||||
|
||||
```
|
||||
rag-search/
|
||||
├── SKILL.md # Instructions for Claude
|
||||
├── scripts/
|
||||
│ ├── search.py # Main search entry point
|
||||
│ ├── index_personal.py # Index state files
|
||||
│ ├── index_docs.py # Index external docs
|
||||
│ └── add_doc_source.py # Add new doc source
|
||||
└── references/
|
||||
└── sources.json # Configured doc sources
|
||||
```
|
||||
|
||||
## Skill Interface
|
||||
|
||||
### Invocation
|
||||
|
||||
```bash
|
||||
# Basic search (both indexes)
|
||||
~/.claude/skills/rag-search/scripts/search.py "how did I configure ArgoCD sync?"
|
||||
|
||||
# Search specific index
|
||||
~/.claude/skills/rag-search/scripts/search.py --index personal "past decisions about caching"
|
||||
~/.claude/skills/rag-search/scripts/search.py --index docs "k0s node maintenance"
|
||||
|
||||
# Control result count
|
||||
~/.claude/skills/rag-search/scripts/search.py --top-k 10 "prometheus alerting rules"
|
||||
```
|
||||
|
||||
### Output Format
|
||||
|
||||
```json
|
||||
{
|
||||
"query": "how did I configure ArgoCD sync?",
|
||||
"results": [
|
||||
{
|
||||
"rank": 1,
|
||||
"score": 0.847,
|
||||
"source": "personal",
|
||||
"file": "memory/decisions.json",
|
||||
"chunk": "Decided to use ArgoCD auto-sync with self-heal disabled...",
|
||||
"metadata": {"date": "2025-01-15", "context": "k8s setup"}
|
||||
},
|
||||
{
|
||||
"rank": 2,
|
||||
"score": 0.823,
|
||||
"source": "docs",
|
||||
"file": "argocd/sync-options.md",
|
||||
"chunk": "Auto-sync can be configured with selfHeal and prune options...",
|
||||
"metadata": {"doc_version": "2.9", "url": "https://..."}
|
||||
}
|
||||
],
|
||||
"searched_collections": ["personal", "docs"],
|
||||
"total_chunks_searched": 1847
|
||||
}
|
||||
```
|
||||
|
||||
### SKILL.md Guidance
|
||||
|
||||
- Start with broad query, refine if results aren't relevant
|
||||
- Cross-reference personal decisions with docs when both appear
|
||||
- Cite sources in answers (file + date for personal, URL for docs)
|
||||
|
||||
## External Docs Management
|
||||
|
||||
### Source Registry
|
||||
|
||||
**Location:** `~/.claude/skills/rag-search/references/sources.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"sources": [
|
||||
{
|
||||
"id": "k0s",
|
||||
"name": "k0s Documentation",
|
||||
"type": "git",
|
||||
"url": "https://github.com/k0sproject/k0s.git",
|
||||
"path": "docs/",
|
||||
"glob": "**/*.md",
|
||||
"version": "v1.30.0",
|
||||
"last_indexed": "2025-01-20T10:00:00Z"
|
||||
},
|
||||
{
|
||||
"id": "argocd",
|
||||
"name": "ArgoCD Documentation",
|
||||
"type": "web",
|
||||
"base_url": "https://argo-cd.readthedocs.io/en/stable/",
|
||||
"pages": ["user-guide/sync-options/", "operator-manual/"],
|
||||
"last_indexed": "2025-01-18T14:30:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Adding Sources
|
||||
|
||||
```bash
|
||||
~/.claude/skills/rag-search/scripts/add_doc_source.py \
|
||||
--id "cilium" \
|
||||
--name "Cilium Docs" \
|
||||
--type git \
|
||||
--url "https://github.com/cilium/cilium.git" \
|
||||
--path "Documentation/" \
|
||||
--glob "**/*.md"
|
||||
|
||||
# Then index it
|
||||
~/.claude/skills/rag-search/scripts/index_docs.py --source cilium
|
||||
```
|
||||
|
||||
### Update Strategies
|
||||
|
||||
| Strategy | Command | When |
|
||||
|----------|---------|------|
|
||||
| Manual | `index_docs.py --source <id>` | After version upgrade |
|
||||
| All sources | `index_docs.py --all` | Periodic refresh |
|
||||
|
||||
## Periodic Refresh
|
||||
|
||||
Daily systemd timer on workstation.
|
||||
|
||||
### Service
|
||||
|
||||
**Location:** `~/.config/systemd/user/rag-index.service`
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Refresh RAG search indexes
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=%h/.claude/skills/rag-search/scripts/index_docs.py --all --quiet
|
||||
ExecStartPost=%h/.claude/skills/rag-search/scripts/index_personal.py --quiet
|
||||
Environment=PATH=%h/.claude/skills/rag-search/venv/bin:/usr/bin
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
```
|
||||
|
||||
### Timer
|
||||
|
||||
**Location:** `~/.config/systemd/user/rag-index.timer`
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Daily RAG index refresh
|
||||
|
||||
[Timer]
|
||||
OnCalendar=daily
|
||||
Persistent=true
|
||||
RandomizedDelaySec=3600
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
```
|
||||
|
||||
### Enable
|
||||
|
||||
```bash
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now rag-index.timer
|
||||
```
|
||||
|
||||
### Manual Trigger
|
||||
|
||||
```bash
|
||||
systemctl --user start rag-index.service
|
||||
journalctl --user -u rag-index.service # View logs
|
||||
```
|
||||
|
||||
## Resource Requirements
|
||||
|
||||
**Target:** Workstation or Pi5 8GB
|
||||
|
||||
| Component | RAM | Disk | Notes |
|
||||
|-----------|-----|------|-------|
|
||||
| Embedding model (all-MiniLM-L6-v2) | ~256MB | ~90MB | Loaded on-demand |
|
||||
| ChromaDB | ~100-500MB | Varies | Scales with index size |
|
||||
| Index: personal (~50 files) | — | ~5MB | Small, fast to query |
|
||||
| Index: docs (10-20 sources) | — | ~100-500MB | Depends on doc volume |
|
||||
| Indexing process (peak) | ~1GB | — | During embedding generation |
|
||||
|
||||
**Pi3 1GB:** Not suitable for this workload.
|
||||
|
||||
## Chunking Strategy
|
||||
|
||||
| Index | Strategy |
|
||||
|-------|----------|
|
||||
| Personal | Per JSON key or logical section (decisions, preferences, facts as separate chunks) |
|
||||
| Docs | ~500 tokens per chunk with overlap, preserve headers as metadata |
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Recommended: Ralph Loop
|
||||
|
||||
This design is suitable for Ralph loop implementation:
|
||||
- Clear success criteria (tests, functional checks)
|
||||
- Iterative refinement expected (tuning chunking, embeddings)
|
||||
- Automatic verification possible
|
||||
|
||||
### Model Delegation
|
||||
|
||||
Use appropriate models for each phase:
|
||||
|
||||
| Phase | Task | Model |
|
||||
|-------|------|-------|
|
||||
| 1 | Set up ChromaDB + embedding model | Haiku |
|
||||
| 2 | Write `index_personal.py` | Sonnet |
|
||||
| 3 | Write `index_docs.py` | Sonnet |
|
||||
| 4 | Write `search.py` | Sonnet |
|
||||
| 5 | Write SKILL.md | Haiku |
|
||||
| 6 | Integration tests | Sonnet |
|
||||
| 7 | End-to-end validation | Sonnet |
|
||||
|
||||
### Ralph Invocation
|
||||
|
||||
```bash
|
||||
/ralph-loop "Implement rag-search skill per docs/plans/2025-01-21-agentic-rag-design.md.
|
||||
|
||||
Delegate to appropriate models:
|
||||
- Haiku: setup, docs, simple scripts
|
||||
- Sonnet: implementation, tests, debugging
|
||||
- Opus: only if stuck on complex reasoning
|
||||
|
||||
Success criteria:
|
||||
1. ChromaDB + embeddings working
|
||||
2. Personal index populated from ~/.claude/state
|
||||
3. At least one external doc source indexed
|
||||
4. search.py returns relevant results
|
||||
5. All tests pass
|
||||
|
||||
Output <promise>COMPLETE</promise> when done." --max-iterations 30 --completion-promise "COMPLETE"
|
||||
```
|
||||
|
||||
### When NOT to use Ralph
|
||||
|
||||
- Design decisions still needed (use brainstorming first)
|
||||
- Requires human judgment mid-implementation
|
||||
- One-shot simple tasks
|
||||
|
||||
## Workflow Integration
|
||||
|
||||
```
|
||||
/superpowers:brainstorm
|
||||
│
|
||||
▼
|
||||
Design doc created
|
||||
(docs/plans/YYYY-MM-DD-*-design.md)
|
||||
│
|
||||
▼
|
||||
"Ready to implement?"
|
||||
│
|
||||
┌────┴────┐
|
||||
│ │
|
||||
▼ ▼
|
||||
Simple Complex/Iterative
|
||||
│ │
|
||||
▼ ▼
|
||||
Manual /ralph-loop
|
||||
or TDD with design doc
|
||||
as spec
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
| Aspect | Decision |
|
||||
|--------|----------|
|
||||
| **Architecture** | Extend existing Claude skill system with semantic search |
|
||||
| **Indexes** | Two: personal (state files) + docs (external) |
|
||||
| **Vector store** | ChromaDB (local, no deps) |
|
||||
| **Embeddings** | sentence-transformers (all-MiniLM-L6-v2) |
|
||||
| **Skill interface** | `rag-search` skill with `search.py` CLI |
|
||||
| **Doc management** | `sources.json` registry, git/web fetching |
|
||||
| **Refresh** | systemd user timer, daily |
|
||||
| **Storage** | `~/.claude/data/rag-search/` |
|
||||
| **Hardware** | Runs on workstation (Pi5 8GB capable if needed) |
|
||||
| **Implementation** | Ralph loop with Haiku/Sonnet subagent delegation |
|
||||
@@ -4,10 +4,10 @@
|
||||
"frontend-design@claude-plugins-official": [
|
||||
{
|
||||
"scope": "user",
|
||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/frontend-design/6d3752c000e2",
|
||||
"version": "6d3752c000e2",
|
||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/frontend-design/15b07b46dab3",
|
||||
"version": "15b07b46dab3",
|
||||
"installedAt": "2025-12-24T19:08:12.422Z",
|
||||
"lastUpdated": "2025-12-24T19:08:12.422Z",
|
||||
"lastUpdated": "2026-01-05T07:21:36.978Z",
|
||||
"gitCommitSha": "6d3752c000e2b3d0e6137bd7adb04895d6f40f14",
|
||||
"isLocal": true
|
||||
}
|
||||
@@ -26,10 +26,10 @@
|
||||
"commit-commands@claude-plugins-official": [
|
||||
{
|
||||
"scope": "user",
|
||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/commit-commands/6d3752c000e2",
|
||||
"version": "6d3752c000e2",
|
||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/commit-commands/15b07b46dab3",
|
||||
"version": "15b07b46dab3",
|
||||
"installedAt": "2025-12-24T19:10:05.451Z",
|
||||
"lastUpdated": "2025-12-24T19:10:36.843Z",
|
||||
"lastUpdated": "2026-01-05T07:21:36.984Z",
|
||||
"isLocal": true
|
||||
}
|
||||
],
|
||||
@@ -69,10 +69,10 @@
|
||||
"ralph-wiggum@claude-plugins-official": [
|
||||
{
|
||||
"scope": "user",
|
||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/ralph-wiggum/6d3752c000e2",
|
||||
"version": "6d3752c000e2",
|
||||
"installPath": "/home/will/.claude/plugins/cache/claude-plugins-official/ralph-wiggum/15b07b46dab3",
|
||||
"version": "15b07b46dab3",
|
||||
"installedAt": "2026-01-02T19:47:02.395Z",
|
||||
"lastUpdated": "2026-01-02T19:47:11.472Z",
|
||||
"lastUpdated": "2026-01-05T07:21:36.997Z",
|
||||
"gitCommitSha": "de89f3066c68d7a2f2d4190173fa46c26e2f30fd",
|
||||
"isLocal": true
|
||||
}
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
"repo": "anthropics/claude-plugins-official"
|
||||
},
|
||||
"installLocation": "/home/will/.claude/plugins/marketplaces/claude-plugins-official",
|
||||
"lastUpdated": "2026-01-04T20:00:29.464Z"
|
||||
"lastUpdated": "2026-01-05T07:22:46.460Z"
|
||||
},
|
||||
"superpowers-marketplace": {
|
||||
"source": {
|
||||
|
||||
@@ -1,16 +1,15 @@
|
||||
# Morning Report - Sun Jan 04, 2026
|
||||
|
||||
## 🌤 Weather
|
||||
Weather unavailable: <urlopen error timed out>
|
||||
Seattle: 51°F, Partly cloudy | High 52° Low 43°
|
||||
|
||||
## 📧 Email
|
||||
10 unread, 0 urgent
|
||||
|
||||
- Chase - Your Chase Freedom Unlimited Visa balance is...
|
||||
- Chase - Your rewards balance has reached 0 POINTS
|
||||
- USPS - PO Box Price Changes Coming
|
||||
- Uber Receipts - Your Saturday evening trip with Uber
|
||||
- Chase - You made an online, phone, or mail transaction...
|
||||
15 unread
|
||||
• Capital One | Quicks - Your requested balance summary
|
||||
• Uber Receipts - [Personal] Your Saturday evening trip wi
|
||||
• Experian - William, it's time to check your utiliza
|
||||
• Experteer Search Age - William, we have 2 new opportunities for
|
||||
• Chase - You can start your mortgage preapproval
|
||||
|
||||
## 📅 Today
|
||||
• 2:00 PM - Seattle Saturday (SAM + QED + Lecosho) (5h)
|
||||
@@ -18,17 +17,14 @@ Weather unavailable: <urlopen error timed out>
|
||||
## 📈 Stocks
|
||||
CRWV $79.32 +10.8% ▲ NVDA $188.85 +1.3% ▲ MSFT $472.94 -2.2% ▼
|
||||
|
||||
## ✅ Tasks
|
||||
⚠️ Could not fetch tasks: ('invalid_scope: Bad Request', {'error': 'invalid_scope', 'error_description': 'Bad Request'})
|
||||
|
||||
## 🖥 Infrastructure
|
||||
K8s: 🟢 | Workstation: 🟢
|
||||
|
||||
## 📰 Tech News
|
||||
• Anti-Aging Injection Regrows Knee Cartilage and Prevents Art... (Hacker News)
|
||||
• How I archived 10 years of memories using Spotify (Hacker News)
|
||||
• C-Sentinel: System prober that captures "system fingerprints... (Hacker News)
|
||||
• Show HN: An LLM-Powered PCB Schematic Checker (Major Update) (Hacker News)
|
||||
• Can I finally start using Wayland in 2026? (Lobsters)
|
||||
• Saying goodbye to the servers at our physical datacenter - S... (Lobsters)
|
||||
• Saying goodbye to the servers at our physical datacenter (Lobsters)
|
||||
|
||||
---
|
||||
*Generated: 2026-01-04 08:00:33 PT*
|
||||
*Generated: 2026-01-04 14:40:48 PT*
|
||||
+11
-15
@@ -1,16 +1,15 @@
|
||||
# Morning Report - Sun Jan 04, 2026
|
||||
|
||||
## 🌤 Weather
|
||||
Weather unavailable: <urlopen error timed out>
|
||||
Seattle: 51°F, Partly cloudy | High 52° Low 43°
|
||||
|
||||
## 📧 Email
|
||||
10 unread, 0 urgent
|
||||
|
||||
- Chase - Your Chase Freedom Unlimited Visa balance is...
|
||||
- Chase - Your rewards balance has reached 0 POINTS
|
||||
- USPS - PO Box Price Changes Coming
|
||||
- Uber Receipts - Your Saturday evening trip with Uber
|
||||
- Chase - You made an online, phone, or mail transaction...
|
||||
15 unread
|
||||
• Capital One | Quicks - Your requested balance summary
|
||||
• Uber Receipts - [Personal] Your Saturday evening trip wi
|
||||
• Experian - William, it's time to check your utiliza
|
||||
• Experteer Search Age - William, we have 2 new opportunities for
|
||||
• Chase - You can start your mortgage preapproval
|
||||
|
||||
## 📅 Today
|
||||
• 2:00 PM - Seattle Saturday (SAM + QED + Lecosho) (5h)
|
||||
@@ -18,17 +17,14 @@ Weather unavailable: <urlopen error timed out>
|
||||
## 📈 Stocks
|
||||
CRWV $79.32 +10.8% ▲ NVDA $188.85 +1.3% ▲ MSFT $472.94 -2.2% ▼
|
||||
|
||||
## ✅ Tasks
|
||||
⚠️ Could not fetch tasks: ('invalid_scope: Bad Request', {'error': 'invalid_scope', 'error_description': 'Bad Request'})
|
||||
|
||||
## 🖥 Infrastructure
|
||||
K8s: 🟢 | Workstation: 🟢
|
||||
|
||||
## 📰 Tech News
|
||||
• Anti-Aging Injection Regrows Knee Cartilage and Prevents Art... (Hacker News)
|
||||
• How I archived 10 years of memories using Spotify (Hacker News)
|
||||
• C-Sentinel: System prober that captures "system fingerprints... (Hacker News)
|
||||
• Show HN: An LLM-Powered PCB Schematic Checker (Major Update) (Hacker News)
|
||||
• Can I finally start using Wayland in 2026? (Lobsters)
|
||||
• Saying goodbye to the servers at our physical datacenter - S... (Lobsters)
|
||||
• Saying goodbye to the servers at our physical datacenter (Lobsters)
|
||||
|
||||
---
|
||||
*Generated: 2026-01-04 08:00:33 PT*
|
||||
*Generated: 2026-01-04 14:40:48 PT*
|
||||
@@ -12,6 +12,7 @@ Agent skills that extend Claude's capabilities. Model-invoked (Claude decides wh
|
||||
| `sysadmin-health` | Arch Linux health check | `health-check.sh` |
|
||||
| `usage` | Session usage tracking | `usage_report.py` |
|
||||
| `programmer-add-project` | Register projects | (workflow only) |
|
||||
| `rag-search` | Semantic search (state + docs) | `search.py`, `index_personal.py`, `index_docs.py` |
|
||||
|
||||
## Skill Structure
|
||||
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
"show_tomorrow": true
|
||||
},
|
||||
"tasks": {
|
||||
"enabled": false,
|
||||
"max_display": 5,
|
||||
"show_due_dates": true
|
||||
},
|
||||
|
||||
@@ -9,11 +9,13 @@ from pathlib import Path
|
||||
|
||||
def fetch_events(mode: str = "today") -> list:
|
||||
"""Fetch calendar events directly using gmail_mcp library."""
|
||||
os.environ.setdefault('GMAIL_CREDENTIALS_PATH', os.path.expanduser('~/.gmail-mcp/credentials.json'))
|
||||
os.environ.setdefault(
|
||||
"GMAIL_CREDENTIALS_PATH", os.path.expanduser("~/.gmail-mcp/credentials.json")
|
||||
)
|
||||
|
||||
try:
|
||||
# Add gmail venv to path
|
||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.13/site-packages"
|
||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.14/site-packages"
|
||||
if str(venv_site) not in sys.path:
|
||||
sys.path.insert(0, str(venv_site))
|
||||
|
||||
@@ -22,26 +24,32 @@ def fetch_events(mode: str = "today") -> list:
|
||||
service = get_calendar_service()
|
||||
now = datetime.utcnow()
|
||||
|
||||
if mode == 'today':
|
||||
if mode == "today":
|
||||
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
end = start + timedelta(days=1)
|
||||
elif mode == 'tomorrow':
|
||||
start = (now + timedelta(days=1)).replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
elif mode == "tomorrow":
|
||||
start = (now + timedelta(days=1)).replace(
|
||||
hour=0, minute=0, second=0, microsecond=0
|
||||
)
|
||||
end = start + timedelta(days=1)
|
||||
else:
|
||||
start = now
|
||||
end = now + timedelta(days=7)
|
||||
|
||||
events_result = service.events().list(
|
||||
calendarId='primary',
|
||||
timeMin=start.isoformat() + 'Z',
|
||||
timeMax=end.isoformat() + 'Z',
|
||||
events_result = (
|
||||
service.events()
|
||||
.list(
|
||||
calendarId="primary",
|
||||
timeMin=start.isoformat() + "Z",
|
||||
timeMax=end.isoformat() + "Z",
|
||||
singleEvents=True,
|
||||
orderBy='startTime',
|
||||
maxResults=20
|
||||
).execute()
|
||||
orderBy="startTime",
|
||||
maxResults=20,
|
||||
)
|
||||
.execute()
|
||||
)
|
||||
|
||||
return events_result.get('items', [])
|
||||
return events_result.get("items", [])
|
||||
|
||||
except Exception as e:
|
||||
return [{"error": str(e)}]
|
||||
@@ -62,7 +70,9 @@ def format_events(today_events: list, tomorrow_events: list = None) -> str:
|
||||
|
||||
if "dateTime" in start:
|
||||
# Timed event
|
||||
dt = datetime.fromisoformat(start["dateTime"].replace("Z", "+00:00"))
|
||||
dt = datetime.fromisoformat(
|
||||
start["dateTime"].replace("Z", "+00:00")
|
||||
)
|
||||
time_str = dt.strftime("%I:%M %p").lstrip("0")
|
||||
elif "date" in start:
|
||||
time_str = "All day"
|
||||
@@ -73,13 +83,19 @@ def format_events(today_events: list, tomorrow_events: list = None) -> str:
|
||||
# Calculate duration if end time available
|
||||
end = event.get("end", {})
|
||||
if "dateTime" in start and "dateTime" in end:
|
||||
start_dt = datetime.fromisoformat(start["dateTime"].replace("Z", "+00:00"))
|
||||
end_dt = datetime.fromisoformat(end["dateTime"].replace("Z", "+00:00"))
|
||||
start_dt = datetime.fromisoformat(
|
||||
start["dateTime"].replace("Z", "+00:00")
|
||||
)
|
||||
end_dt = datetime.fromisoformat(
|
||||
end["dateTime"].replace("Z", "+00:00")
|
||||
)
|
||||
mins = int((end_dt - start_dt).total_seconds() / 60)
|
||||
if mins >= 60:
|
||||
hours = mins // 60
|
||||
remaining = mins % 60
|
||||
duration = f" ({hours}h{remaining}m)" if remaining else f" ({hours}h)"
|
||||
duration = (
|
||||
f" ({hours}h{remaining}m)" if remaining else f" ({hours}h)"
|
||||
)
|
||||
else:
|
||||
duration = f" ({mins}m)"
|
||||
|
||||
@@ -92,17 +108,23 @@ def format_events(today_events: list, tomorrow_events: list = None) -> str:
|
||||
|
||||
# Tomorrow preview
|
||||
if tomorrow_events is not None:
|
||||
if tomorrow_events and (len(tomorrow_events) == 0 or "error" not in tomorrow_events[0]):
|
||||
if tomorrow_events and (
|
||||
len(tomorrow_events) == 0 or "error" not in tomorrow_events[0]
|
||||
):
|
||||
count = len(tomorrow_events)
|
||||
if count > 0:
|
||||
first = tomorrow_events[0]
|
||||
start = first.get("start", {})
|
||||
if "dateTime" in start:
|
||||
dt = datetime.fromisoformat(start["dateTime"].replace("Z", "+00:00"))
|
||||
dt = datetime.fromisoformat(
|
||||
start["dateTime"].replace("Z", "+00:00")
|
||||
)
|
||||
first_time = dt.strftime("%I:%M %p").lstrip("0")
|
||||
else:
|
||||
first_time = "All day"
|
||||
lines.append(f"Tomorrow: {count} event{'s' if count > 1 else ''}, first at {first_time}")
|
||||
lines.append(
|
||||
f"Tomorrow: {count} event{'s' if count > 1 else ''}, first at {first_time}"
|
||||
)
|
||||
else:
|
||||
lines.append("Tomorrow: No events")
|
||||
|
||||
@@ -126,7 +148,7 @@ def collect(config: dict) -> dict:
|
||||
"icon": "📅",
|
||||
"content": formatted,
|
||||
"raw": {"today": today_events, "tomorrow": tomorrow_events},
|
||||
"error": today_events[0].get("error") if has_error else None
|
||||
"error": today_events[0].get("error") if has_error else None,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -11,37 +11,49 @@ from pathlib import Path
|
||||
def fetch_unread_emails(days: int = 7, max_results: int = 15) -> list:
|
||||
"""Fetch unread emails directly using gmail_mcp library."""
|
||||
# Set credentials path
|
||||
os.environ.setdefault('GMAIL_CREDENTIALS_PATH', os.path.expanduser('~/.gmail-mcp/credentials.json'))
|
||||
os.environ.setdefault(
|
||||
"GMAIL_CREDENTIALS_PATH", os.path.expanduser("~/.gmail-mcp/credentials.json")
|
||||
)
|
||||
|
||||
try:
|
||||
# Add gmail venv to path
|
||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.13/site-packages"
|
||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.14/site-packages"
|
||||
if str(venv_site) not in sys.path:
|
||||
sys.path.insert(0, str(venv_site))
|
||||
|
||||
from gmail_mcp.utils.GCP.gmail_auth import get_gmail_service
|
||||
|
||||
service = get_gmail_service()
|
||||
results = service.users().messages().list(
|
||||
userId='me',
|
||||
q=f'is:unread newer_than:{days}d',
|
||||
maxResults=max_results
|
||||
).execute()
|
||||
results = (
|
||||
service.users()
|
||||
.messages()
|
||||
.list(
|
||||
userId="me", q=f"is:unread newer_than:{days}d", maxResults=max_results
|
||||
)
|
||||
.execute()
|
||||
)
|
||||
|
||||
emails = []
|
||||
for msg in results.get('messages', []):
|
||||
detail = service.users().messages().get(
|
||||
userId='me',
|
||||
id=msg['id'],
|
||||
format='metadata',
|
||||
metadataHeaders=['From', 'Subject']
|
||||
).execute()
|
||||
headers = {h['name']: h['value'] for h in detail['payload']['headers']}
|
||||
emails.append({
|
||||
'from': headers.get('From', 'Unknown'),
|
||||
'subject': headers.get('Subject', '(no subject)'),
|
||||
'id': msg['id']
|
||||
})
|
||||
for msg in results.get("messages", []):
|
||||
detail = (
|
||||
service.users()
|
||||
.messages()
|
||||
.get(
|
||||
userId="me",
|
||||
id=msg["id"],
|
||||
format="metadata",
|
||||
metadataHeaders=["From", "Subject"],
|
||||
)
|
||||
.execute()
|
||||
)
|
||||
headers = {h["name"]: h["value"] for h in detail["payload"]["headers"]}
|
||||
emails.append(
|
||||
{
|
||||
"from": headers.get("From", "Unknown"),
|
||||
"subject": headers.get("Subject", "(no subject)"),
|
||||
"id": msg["id"],
|
||||
}
|
||||
)
|
||||
|
||||
return emails
|
||||
|
||||
@@ -79,10 +91,17 @@ Output the formatted email section, nothing else."""
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["/home/will/.local/bin/claude", "--print", "--model", "sonnet", "-p", prompt],
|
||||
[
|
||||
"/home/will/.local/bin/claude",
|
||||
"--print",
|
||||
"--model",
|
||||
"sonnet",
|
||||
"-p",
|
||||
prompt,
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60
|
||||
timeout=60,
|
||||
)
|
||||
|
||||
if result.returncode == 0 and result.stdout.strip():
|
||||
@@ -131,7 +150,7 @@ def collect(config: dict) -> dict:
|
||||
"content": formatted,
|
||||
"raw": emails if not has_error else None,
|
||||
"count": len(emails) if not has_error else 0,
|
||||
"error": emails[0].get("error") if has_error else None
|
||||
"error": emails[0].get("error") if has_error else None,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
# Add gmail venv to path for Google API libraries
|
||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.13/site-packages"
|
||||
venv_site = Path.home() / ".claude/mcp/gmail/venv/lib/python3.14/site-packages"
|
||||
if str(venv_site) not in sys.path:
|
||||
sys.path.insert(0, str(venv_site))
|
||||
|
||||
@@ -18,6 +18,7 @@ try:
|
||||
from google_auth_oauthlib.flow import InstalledAppFlow
|
||||
from google.auth.transport.requests import Request
|
||||
from googleapiclient.discovery import build
|
||||
|
||||
GOOGLE_API_AVAILABLE = True
|
||||
except ImportError:
|
||||
GOOGLE_API_AVAILABLE = False
|
||||
@@ -57,7 +58,11 @@ def fetch_tasks(max_results: int = 10) -> list:
|
||||
try:
|
||||
creds = get_credentials()
|
||||
if not creds:
|
||||
return [{"error": "Tasks API not authenticated - run: ~/.claude/mcp/gmail/venv/bin/python ~/.claude/skills/morning-report/scripts/collectors/gtasks.py --auth"}]
|
||||
return [
|
||||
{
|
||||
"error": "Tasks API not authenticated - run: ~/.claude/mcp/gmail/venv/bin/python ~/.claude/skills/morning-report/scripts/collectors/gtasks.py --auth"
|
||||
}
|
||||
]
|
||||
|
||||
service = build("tasks", "v1", credentials=creds)
|
||||
|
||||
@@ -69,12 +74,16 @@ def fetch_tasks(max_results: int = 10) -> list:
|
||||
tasklist_id = tasklists["items"][0]["id"]
|
||||
|
||||
# Get tasks
|
||||
results = service.tasks().list(
|
||||
results = (
|
||||
service.tasks()
|
||||
.list(
|
||||
tasklist=tasklist_id,
|
||||
maxResults=max_results,
|
||||
showCompleted=False,
|
||||
showHidden=False
|
||||
).execute()
|
||||
showHidden=False,
|
||||
)
|
||||
.execute()
|
||||
)
|
||||
|
||||
tasks = results.get("items", [])
|
||||
return tasks
|
||||
@@ -150,7 +159,7 @@ def collect(config: dict) -> dict:
|
||||
"content": formatted,
|
||||
"raw": tasks if not has_error else None,
|
||||
"count": len(tasks) if not has_error else 0,
|
||||
"error": tasks[0].get("error") if has_error else None
|
||||
"error": tasks[0].get("error") if has_error else None,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ from collectors import weather, stocks, infra, news
|
||||
# These may fail if gmail venv not activated
|
||||
try:
|
||||
from collectors import gmail, gcal, gtasks
|
||||
|
||||
GOOGLE_COLLECTORS = True
|
||||
except ImportError:
|
||||
GOOGLE_COLLECTORS = False
|
||||
@@ -29,10 +30,7 @@ LOG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||
handlers=[
|
||||
logging.FileHandler(LOG_PATH),
|
||||
logging.StreamHandler()
|
||||
]
|
||||
handlers=[logging.FileHandler(LOG_PATH), logging.StreamHandler()],
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -58,10 +56,17 @@ def collect_section(name: str, collector_func, config: dict) -> dict:
|
||||
"section": name,
|
||||
"icon": "❓",
|
||||
"content": f"⚠️ {name} unavailable: {e}",
|
||||
"error": str(e)
|
||||
"error": str(e),
|
||||
}
|
||||
|
||||
|
||||
def is_section_enabled(name: str, config: dict) -> bool:
|
||||
"""Check if a section is enabled in config."""
|
||||
section_key = name.lower()
|
||||
section_config = config.get(section_key, {})
|
||||
return section_config.get("enabled", True)
|
||||
|
||||
|
||||
def collect_all(config: dict) -> list:
|
||||
"""Collect all sections in parallel."""
|
||||
collectors = [
|
||||
@@ -72,11 +77,12 @@ def collect_all(config: dict) -> list:
|
||||
]
|
||||
|
||||
if GOOGLE_COLLECTORS:
|
||||
collectors.extend([
|
||||
("Email", gmail.collect),
|
||||
("Calendar", gcal.collect),
|
||||
("Tasks", gtasks.collect),
|
||||
])
|
||||
if is_section_enabled("email", config):
|
||||
collectors.append(("Email", gmail.collect))
|
||||
if is_section_enabled("calendar", config):
|
||||
collectors.append(("Calendar", gcal.collect))
|
||||
if is_section_enabled("tasks", config):
|
||||
collectors.append(("Tasks", gtasks.collect))
|
||||
else:
|
||||
logger.warning("Google collectors not available - run with gmail venv")
|
||||
|
||||
@@ -95,12 +101,14 @@ def collect_all(config: dict) -> list:
|
||||
results.append(result)
|
||||
except Exception as e:
|
||||
logger.error(f"Future {name} exception: {e}")
|
||||
results.append({
|
||||
results.append(
|
||||
{
|
||||
"section": name,
|
||||
"icon": "❓",
|
||||
"content": f"⚠️ {name} failed: {e}",
|
||||
"error": str(e)
|
||||
})
|
||||
"error": str(e),
|
||||
}
|
||||
)
|
||||
|
||||
return results
|
||||
|
||||
@@ -111,13 +119,21 @@ def render_report(sections: list, config: dict) -> str:
|
||||
date_str = now.strftime("%a %b %d, %Y")
|
||||
time_str = now.strftime("%I:%M %p %Z").strip()
|
||||
|
||||
lines = [
|
||||
f"# Morning Report - {date_str}",
|
||||
""
|
||||
]
|
||||
lines = [f"# Morning Report - {date_str}", ""]
|
||||
|
||||
# Order sections
|
||||
order = ["Weather", "Email", "Calendar", "Today", "Stocks", "Tasks", "Infra", "Infrastructure", "News", "Tech News"]
|
||||
order = [
|
||||
"Weather",
|
||||
"Email",
|
||||
"Calendar",
|
||||
"Today",
|
||||
"Stocks",
|
||||
"Tasks",
|
||||
"Infra",
|
||||
"Infrastructure",
|
||||
"News",
|
||||
"Tech News",
|
||||
]
|
||||
|
||||
# Sort by order
|
||||
section_map = {s.get("section", ""): s for s in sections}
|
||||
@@ -137,10 +153,7 @@ def render_report(sections: list, config: dict) -> str:
|
||||
lines.append("")
|
||||
|
||||
# Footer
|
||||
lines.extend([
|
||||
"---",
|
||||
f"*Generated: {now.strftime('%Y-%m-%d %H:%M:%S')} PT*"
|
||||
])
|
||||
lines.extend(["---", f"*Generated: {now.strftime('%Y-%m-%d %H:%M:%S')} PT*"])
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
@@ -148,7 +161,9 @@ def render_report(sections: list, config: dict) -> str:
|
||||
def save_report(content: str, config: dict) -> Path:
|
||||
"""Save report to file and archive."""
|
||||
output_config = config.get("output", {})
|
||||
output_path = Path(output_config.get("path", "~/.claude/reports/morning.md")).expanduser()
|
||||
output_path = Path(
|
||||
output_config.get("path", "~/.claude/reports/morning.md")
|
||||
).expanduser()
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Write main report
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
---
|
||||
name: rag-search
|
||||
description: Semantic search across personal state files and external documentation
|
||||
triggers: [search, find, lookup, what did, how did, when did, past decisions, previous, documentation, docs]
|
||||
---
|
||||
|
||||
# RAG Search Skill
|
||||
|
||||
Semantic search across two indexes:
|
||||
- **personal**: Your state files, memory, decisions, preferences
|
||||
- **docs**: External documentation (k0s, ArgoCD, etc.)
|
||||
|
||||
## When to Use
|
||||
|
||||
- "What decisions did I make about X?"
|
||||
- "How did I configure Y?"
|
||||
- "What does the k0s documentation say about Z?"
|
||||
- "Find my past notes on..."
|
||||
- Cross-referencing personal context with official docs
|
||||
|
||||
## Scripts
|
||||
|
||||
All scripts use the venv at `~/.claude/skills/rag-search/venv/`.
|
||||
|
||||
### Search (Primary Interface)
|
||||
|
||||
```bash
|
||||
# Search both indexes
|
||||
~/.claude/skills/rag-search/venv/bin/python \
|
||||
~/.claude/skills/rag-search/scripts/search.py "query"
|
||||
|
||||
# Search specific index
|
||||
~/.claude/skills/rag-search/scripts/search.py --index personal "query"
|
||||
~/.claude/skills/rag-search/scripts/search.py --index docs "query"
|
||||
|
||||
# Control result count
|
||||
~/.claude/skills/rag-search/scripts/search.py --top-k 10 "query"
|
||||
```
|
||||
|
||||
### Index Management
|
||||
|
||||
```bash
|
||||
# Reindex personal state files
|
||||
~/.claude/skills/rag-search/venv/bin/python \
|
||||
~/.claude/skills/rag-search/scripts/index_personal.py
|
||||
|
||||
# Index all doc sources
|
||||
~/.claude/skills/rag-search/venv/bin/python \
|
||||
~/.claude/skills/rag-search/scripts/index_docs.py --all
|
||||
|
||||
# Index specific doc source
|
||||
~/.claude/skills/rag-search/scripts/index_docs.py --source k0s
|
||||
```
|
||||
|
||||
### Adding Doc Sources
|
||||
|
||||
```bash
|
||||
# Add a git-based doc source
|
||||
~/.claude/skills/rag-search/venv/bin/python \
|
||||
~/.claude/skills/rag-search/scripts/add_doc_source.py \
|
||||
--id "argocd" \
|
||||
--name "ArgoCD Documentation" \
|
||||
--type git \
|
||||
--url "https://github.com/argoproj/argo-cd.git" \
|
||||
--path "docs/" \
|
||||
--glob "**/*.md"
|
||||
|
||||
# List configured sources
|
||||
~/.claude/skills/rag-search/scripts/add_doc_source.py --list
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
Search returns JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"query": "your search query",
|
||||
"results": [
|
||||
{
|
||||
"rank": 1,
|
||||
"score": 0.847,
|
||||
"source": "personal",
|
||||
"file": "memory/decisions.json",
|
||||
"chunk": "Relevant text content...",
|
||||
"metadata": {"date": "2025-01-15"}
|
||||
}
|
||||
],
|
||||
"searched_collections": ["personal", "docs"],
|
||||
"total_chunks_searched": 1847
|
||||
}
|
||||
```
|
||||
|
||||
## Search Strategy
|
||||
|
||||
1. **Start broad** - Use general terms first
|
||||
2. **Refine if needed** - Add specific keywords if results aren't relevant
|
||||
3. **Cross-reference** - When both personal and docs results appear, synthesize them
|
||||
4. **Cite sources** - Include file paths and dates in your answers
|
||||
|
||||
## Example Workflow
|
||||
|
||||
User asks: "How should I configure ArgoCD sync?"
|
||||
|
||||
1. Search both indexes:
|
||||
```bash
|
||||
search.py "ArgoCD sync configuration"
|
||||
```
|
||||
|
||||
2. If personal results exist, prioritize those (user's past decisions)
|
||||
|
||||
3. Supplement with docs results for official guidance
|
||||
|
||||
4. Synthesize answer:
|
||||
> Based on your previous decision (decisions.json, 2025-01-15), you configured ArgoCD with auto-sync enabled but self-heal disabled. The ArgoCD docs recommend this for production environments where you want automatic deployment but manual intervention for drift correction.
|
||||
|
||||
## Maintenance
|
||||
|
||||
Indexes should be refreshed periodically:
|
||||
- Personal: After significant state changes
|
||||
- Docs: After tool version upgrades
|
||||
|
||||
A systemd timer can automate this (see design doc for setup).
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"sources": [
|
||||
{
|
||||
"id": "k0s",
|
||||
"name": "k0s Documentation",
|
||||
"type": "git",
|
||||
"url": "https://github.com/k0sproject/k0s.git",
|
||||
"path": "docs/",
|
||||
"glob": "**/*.md",
|
||||
"version": "main",
|
||||
"last_indexed": "2026-01-04T23:27:40.175671"
|
||||
}
|
||||
]
|
||||
}
|
||||
Executable
+205
@@ -0,0 +1,205 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
RAG Search - Add Documentation Source
|
||||
|
||||
Adds a new documentation source to the registry.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Constants
|
||||
SKILL_DIR = Path(__file__).parent.parent
|
||||
SOURCES_FILE = SKILL_DIR / "references" / "sources.json"
|
||||
|
||||
|
||||
def load_sources() -> list[dict]:
|
||||
"""Load configured documentation sources."""
|
||||
if not SOURCES_FILE.exists():
|
||||
return []
|
||||
with open(SOURCES_FILE) as f:
|
||||
data = json.load(f)
|
||||
return data.get("sources", [])
|
||||
|
||||
|
||||
def save_sources(sources: list[dict]) -> None:
|
||||
"""Save documentation sources."""
|
||||
SOURCES_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(SOURCES_FILE, "w") as f:
|
||||
json.dump({"sources": sources}, f, indent=2)
|
||||
|
||||
|
||||
def add_source(
|
||||
source_id: str,
|
||||
name: str,
|
||||
source_type: str,
|
||||
url: str = None,
|
||||
path: str = None,
|
||||
glob: str = "**/*.md",
|
||||
version: str = None,
|
||||
base_url: str = None,
|
||||
) -> dict:
|
||||
"""
|
||||
Add a new documentation source.
|
||||
|
||||
Args:
|
||||
source_id: Unique identifier for the source
|
||||
name: Human-readable name
|
||||
source_type: "git" or "local"
|
||||
url: Git repository URL (for git type)
|
||||
path: Path within repo or local path
|
||||
glob: File pattern to match
|
||||
version: Git tag/branch (for git type)
|
||||
base_url: Base URL for documentation links
|
||||
|
||||
Returns:
|
||||
The created source configuration
|
||||
"""
|
||||
sources = load_sources()
|
||||
|
||||
# Check for existing source
|
||||
existing = [s for s in sources if s["id"] == source_id]
|
||||
if existing:
|
||||
raise ValueError(f"Source already exists: {source_id}")
|
||||
|
||||
# Build source config
|
||||
source = {
|
||||
"id": source_id,
|
||||
"name": name,
|
||||
"type": source_type,
|
||||
}
|
||||
|
||||
if source_type == "git":
|
||||
if not url:
|
||||
raise ValueError("Git sources require --url")
|
||||
source["url"] = url
|
||||
if version:
|
||||
source["version"] = version
|
||||
elif source_type == "local":
|
||||
if not path:
|
||||
raise ValueError("Local sources require --path")
|
||||
source["path"] = str(Path(path).expanduser())
|
||||
else:
|
||||
raise ValueError(f"Unknown source type: {source_type}")
|
||||
|
||||
if path and source_type == "git":
|
||||
source["path"] = path
|
||||
source["glob"] = glob
|
||||
if base_url:
|
||||
source["base_url"] = base_url
|
||||
|
||||
sources.append(source)
|
||||
save_sources(sources)
|
||||
|
||||
return source
|
||||
|
||||
|
||||
def remove_source(source_id: str) -> bool:
|
||||
"""Remove a documentation source."""
|
||||
sources = load_sources()
|
||||
original_count = len(sources)
|
||||
sources = [s for s in sources if s["id"] != source_id]
|
||||
|
||||
if len(sources) == original_count:
|
||||
return False
|
||||
|
||||
save_sources(sources)
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Add or manage documentation sources for RAG search",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
# Add k0s documentation from GitHub
|
||||
%(prog)s --id k0s --name "k0s Documentation" --type git \\
|
||||
--url "https://github.com/k0sproject/k0s.git" \\
|
||||
--path "docs/" --version "v1.30.0"
|
||||
|
||||
# Add local documentation directory
|
||||
%(prog)s --id internal --name "Internal Docs" --type local \\
|
||||
--path "~/docs/internal" --glob "**/*.md"
|
||||
|
||||
# Remove a source
|
||||
%(prog)s --remove k0s
|
||||
|
||||
# List sources
|
||||
%(prog)s --list
|
||||
"""
|
||||
)
|
||||
parser.add_argument("--id", help="Unique source identifier")
|
||||
parser.add_argument("--name", help="Human-readable name")
|
||||
parser.add_argument(
|
||||
"--type", "-t",
|
||||
choices=["git", "local"],
|
||||
default="git",
|
||||
help="Source type (default: git)"
|
||||
)
|
||||
parser.add_argument("--url", help="Git repository URL")
|
||||
parser.add_argument("--path", help="Path within repo or local directory")
|
||||
parser.add_argument(
|
||||
"--glob", "-g",
|
||||
default="**/*.md",
|
||||
help="File pattern to match (default: **/*.md)"
|
||||
)
|
||||
parser.add_argument("--version", "-v", help="Git tag or branch")
|
||||
parser.add_argument("--base-url", help="Base URL for documentation links")
|
||||
parser.add_argument(
|
||||
"--remove", "-r",
|
||||
metavar="ID",
|
||||
help="Remove a source by ID"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--list", "-l",
|
||||
action="store_true",
|
||||
help="List configured sources"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.list:
|
||||
sources = load_sources()
|
||||
if sources:
|
||||
print(json.dumps(sources, indent=2))
|
||||
else:
|
||||
print("No documentation sources configured")
|
||||
return
|
||||
|
||||
if args.remove:
|
||||
if remove_source(args.remove):
|
||||
print(f"Removed source: {args.remove}")
|
||||
else:
|
||||
print(f"Source not found: {args.remove}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
return
|
||||
|
||||
# Adding a new source
|
||||
if not args.id or not args.name:
|
||||
parser.error("--id and --name are required when adding a source")
|
||||
|
||||
try:
|
||||
source = add_source(
|
||||
source_id=args.id,
|
||||
name=args.name,
|
||||
source_type=args.type,
|
||||
url=args.url,
|
||||
path=args.path,
|
||||
glob=args.glob,
|
||||
version=args.version,
|
||||
base_url=args.base_url,
|
||||
)
|
||||
print(f"Added source: {args.id}")
|
||||
print(json.dumps(source, indent=2))
|
||||
print(f"\nTo index this source, run:")
|
||||
print(f" index_docs.py --source {args.id}")
|
||||
except ValueError as e:
|
||||
print(f"Error: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+419
@@ -0,0 +1,419 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
RAG Search - Documentation Index Builder
|
||||
|
||||
Indexes external documentation sources for semantic search.
|
||||
Supports git repos and local directories.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Generator, Optional
|
||||
|
||||
# Add venv site-packages to path
|
||||
VENV_PATH = Path(__file__).parent.parent / "venv" / "lib" / "python3.13" / "site-packages"
|
||||
if str(VENV_PATH) not in sys.path:
|
||||
sys.path.insert(0, str(VENV_PATH))
|
||||
|
||||
import chromadb
|
||||
from sentence_transformers import SentenceTransformer
|
||||
|
||||
# Constants
|
||||
SKILL_DIR = Path(__file__).parent.parent
|
||||
SOURCES_FILE = SKILL_DIR / "references" / "sources.json"
|
||||
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||
CHROMA_DIR = DATA_DIR / "chroma"
|
||||
DOCS_CACHE_DIR = DATA_DIR / "docs-cache"
|
||||
MODEL_NAME = "all-MiniLM-L6-v2"
|
||||
COLLECTION_NAME = "docs"
|
||||
|
||||
# Chunking parameters
|
||||
CHUNK_SIZE = 500 # Target tokens (roughly 4 chars per token)
|
||||
CHUNK_OVERLAP = 50
|
||||
|
||||
|
||||
def load_sources() -> list[dict]:
|
||||
"""Load configured documentation sources."""
|
||||
if not SOURCES_FILE.exists():
|
||||
return []
|
||||
with open(SOURCES_FILE) as f:
|
||||
data = json.load(f)
|
||||
return data.get("sources", [])
|
||||
|
||||
|
||||
def save_sources(sources: list[dict]) -> None:
|
||||
"""Save documentation sources."""
|
||||
SOURCES_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(SOURCES_FILE, "w") as f:
|
||||
json.dump({"sources": sources}, f, indent=2)
|
||||
|
||||
|
||||
def fetch_git_source(source: dict, quiet: bool = False) -> Optional[Path]:
|
||||
"""
|
||||
Clone or update a git repository.
|
||||
|
||||
Returns:
|
||||
Path to the docs directory within the repo
|
||||
"""
|
||||
source_id = source["id"]
|
||||
url = source["url"]
|
||||
version = source.get("version", "HEAD")
|
||||
doc_path = source.get("path", "")
|
||||
|
||||
cache_dir = DOCS_CACHE_DIR / source_id
|
||||
|
||||
if cache_dir.exists():
|
||||
# Update existing repo
|
||||
if not quiet:
|
||||
print(f" Updating {source_id}...")
|
||||
try:
|
||||
subprocess.run(
|
||||
["git", "fetch", "--all"],
|
||||
cwd=cache_dir,
|
||||
capture_output=True,
|
||||
check=True
|
||||
)
|
||||
subprocess.run(
|
||||
["git", "checkout", version],
|
||||
cwd=cache_dir,
|
||||
capture_output=True,
|
||||
check=True
|
||||
)
|
||||
subprocess.run(
|
||||
["git", "pull", "--ff-only"],
|
||||
cwd=cache_dir,
|
||||
capture_output=True,
|
||||
check=False # May fail on tags
|
||||
)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f" Warning: Could not update {source_id}: {e}", file=sys.stderr)
|
||||
else:
|
||||
# Clone new repo
|
||||
if not quiet:
|
||||
print(f" Cloning {source_id}...")
|
||||
cache_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
subprocess.run(
|
||||
["git", "clone", "--depth", "1", url, str(cache_dir)],
|
||||
capture_output=True,
|
||||
check=True
|
||||
)
|
||||
if version != "HEAD":
|
||||
subprocess.run(
|
||||
["git", "fetch", "--depth", "1", "origin", version],
|
||||
cwd=cache_dir,
|
||||
capture_output=True,
|
||||
check=True
|
||||
)
|
||||
subprocess.run(
|
||||
["git", "checkout", version],
|
||||
cwd=cache_dir,
|
||||
capture_output=True,
|
||||
check=True
|
||||
)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f" Error: Could not clone {source_id}: {e}", file=sys.stderr)
|
||||
return None
|
||||
|
||||
docs_dir = cache_dir / doc_path if doc_path else cache_dir
|
||||
return docs_dir if docs_dir.exists() else None
|
||||
|
||||
|
||||
def chunk_markdown(content: str, file_path: str) -> Generator[tuple[str, dict], None, None]:
|
||||
"""
|
||||
Chunk markdown content for embedding.
|
||||
|
||||
Strategy:
|
||||
- Split by headers to preserve context
|
||||
- Chunk sections that are too long
|
||||
- Preserve header hierarchy in metadata
|
||||
"""
|
||||
lines = content.split("\n")
|
||||
current_chunk = []
|
||||
current_headers = []
|
||||
chunk_start_line = 0
|
||||
|
||||
def emit_chunk() -> Optional[tuple[str, dict]]:
|
||||
if not current_chunk:
|
||||
return None
|
||||
text = "\n".join(current_chunk).strip()
|
||||
if len(text) < 20:
|
||||
return None
|
||||
|
||||
metadata = {
|
||||
"file": file_path,
|
||||
"headers": " > ".join(current_headers) if current_headers else ""
|
||||
}
|
||||
return (text, metadata)
|
||||
|
||||
for i, line in enumerate(lines):
|
||||
# Check for header
|
||||
header_match = re.match(r'^(#{1,6})\s+(.+)$', line)
|
||||
|
||||
if header_match:
|
||||
# Emit current chunk before new header
|
||||
chunk = emit_chunk()
|
||||
if chunk:
|
||||
yield chunk
|
||||
current_chunk = []
|
||||
|
||||
# Update header hierarchy
|
||||
level = len(header_match.group(1))
|
||||
header_text = header_match.group(2).strip()
|
||||
|
||||
# Trim headers to current level
|
||||
current_headers = current_headers[:level-1]
|
||||
current_headers.append(header_text)
|
||||
|
||||
chunk_start_line = i
|
||||
|
||||
current_chunk.append(line)
|
||||
|
||||
# Check if chunk is getting too large (rough token estimate)
|
||||
chunk_text = "\n".join(current_chunk)
|
||||
if len(chunk_text) > CHUNK_SIZE * 4:
|
||||
chunk = emit_chunk()
|
||||
if chunk:
|
||||
yield chunk
|
||||
# Start new chunk with overlap
|
||||
overlap_lines = current_chunk[-CHUNK_OVERLAP // 10:] if len(current_chunk) > CHUNK_OVERLAP // 10 else []
|
||||
current_chunk = overlap_lines
|
||||
|
||||
# Emit final chunk
|
||||
chunk = emit_chunk()
|
||||
if chunk:
|
||||
yield chunk
|
||||
|
||||
|
||||
def index_source(
|
||||
source: dict,
|
||||
model: SentenceTransformer,
|
||||
quiet: bool = False
|
||||
) -> tuple[list[str], list[list[float]], list[dict], list[str]]:
|
||||
"""
|
||||
Index a single documentation source.
|
||||
|
||||
Returns:
|
||||
(chunks, embeddings, metadatas, ids)
|
||||
"""
|
||||
source_id = source["id"]
|
||||
source_type = source.get("type", "git")
|
||||
glob_pattern = source.get("glob", "**/*.md")
|
||||
|
||||
if source_type == "git":
|
||||
docs_dir = fetch_git_source(source, quiet=quiet)
|
||||
if not docs_dir:
|
||||
return [], [], [], []
|
||||
elif source_type == "local":
|
||||
docs_dir = Path(source["path"]).expanduser()
|
||||
if not docs_dir.exists():
|
||||
print(f" Warning: Local path does not exist: {docs_dir}", file=sys.stderr)
|
||||
return [], [], [], []
|
||||
else:
|
||||
print(f" Warning: Unknown source type: {source_type}", file=sys.stderr)
|
||||
return [], [], [], []
|
||||
|
||||
chunks = []
|
||||
metadatas = []
|
||||
ids = []
|
||||
|
||||
# Find and process files
|
||||
files = list(docs_dir.glob(glob_pattern))
|
||||
if not quiet:
|
||||
print(f" Found {len(files)} files matching {glob_pattern}")
|
||||
|
||||
for file_path in files:
|
||||
try:
|
||||
content = file_path.read_text(encoding="utf-8", errors="ignore")
|
||||
except IOError:
|
||||
continue
|
||||
|
||||
rel_path = str(file_path.relative_to(docs_dir))
|
||||
full_path = f"{source_id}/{rel_path}"
|
||||
|
||||
for chunk_text, metadata in chunk_markdown(content, full_path):
|
||||
chunk_id = f"docs_{source_id}_{len(chunks)}"
|
||||
chunks.append(chunk_text)
|
||||
metadata["source_id"] = source_id
|
||||
metadata["source_name"] = source.get("name", source_id)
|
||||
if source.get("version"):
|
||||
metadata["version"] = source["version"]
|
||||
if source.get("base_url"):
|
||||
metadata["url"] = source["base_url"]
|
||||
metadatas.append(metadata)
|
||||
ids.append(chunk_id)
|
||||
|
||||
if not quiet:
|
||||
print(f" Indexed {len(chunks)} chunks from {source_id}")
|
||||
|
||||
return chunks, [], metadatas, ids
|
||||
|
||||
|
||||
def index_docs(
|
||||
source_id: Optional[str] = None,
|
||||
all_sources: bool = False,
|
||||
quiet: bool = False
|
||||
) -> dict:
|
||||
"""
|
||||
Index documentation sources.
|
||||
|
||||
Args:
|
||||
source_id: Index only this source
|
||||
all_sources: Index all configured sources
|
||||
quiet: Suppress progress output
|
||||
|
||||
Returns:
|
||||
Summary statistics
|
||||
"""
|
||||
sources = load_sources()
|
||||
if not sources:
|
||||
return {"error": "No documentation sources configured"}
|
||||
|
||||
# Filter sources
|
||||
if source_id:
|
||||
sources = [s for s in sources if s["id"] == source_id]
|
||||
if not sources:
|
||||
return {"error": f"Source not found: {source_id}"}
|
||||
elif not all_sources:
|
||||
return {"error": "Specify --source <id> or --all"}
|
||||
|
||||
if not quiet:
|
||||
print(f"Indexing {len(sources)} documentation source(s)")
|
||||
|
||||
# Initialize model and client
|
||||
model = SentenceTransformer(MODEL_NAME)
|
||||
CHROMA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
client = chromadb.PersistentClient(path=str(CHROMA_DIR))
|
||||
|
||||
# Get or create collection
|
||||
try:
|
||||
collection = client.get_collection(COLLECTION_NAME)
|
||||
# If indexing all or specific source, we'll need to handle existing data
|
||||
if all_sources:
|
||||
client.delete_collection(COLLECTION_NAME)
|
||||
collection = client.create_collection(
|
||||
name=COLLECTION_NAME,
|
||||
metadata={"description": "External documentation"}
|
||||
)
|
||||
except Exception:
|
||||
collection = client.create_collection(
|
||||
name=COLLECTION_NAME,
|
||||
metadata={"description": "External documentation"}
|
||||
)
|
||||
|
||||
# Process each source
|
||||
all_chunks = []
|
||||
all_metadatas = []
|
||||
all_ids = []
|
||||
|
||||
for source in sources:
|
||||
if not quiet:
|
||||
print(f"\nProcessing: {source['name']}")
|
||||
|
||||
chunks, _, metadatas, ids = index_source(source, model, quiet=quiet)
|
||||
all_chunks.extend(chunks)
|
||||
all_metadatas.extend(metadatas)
|
||||
all_ids.extend(ids)
|
||||
|
||||
# Update last_indexed timestamp
|
||||
source["last_indexed"] = datetime.now().isoformat()
|
||||
|
||||
# Batch embed and add to collection
|
||||
if all_chunks:
|
||||
if not quiet:
|
||||
print(f"\nEmbedding {len(all_chunks)} chunks...")
|
||||
|
||||
embeddings = model.encode(all_chunks, show_progress_bar=not quiet).tolist()
|
||||
|
||||
# Add in batches
|
||||
batch_size = 100
|
||||
for i in range(0, len(all_chunks), batch_size):
|
||||
end_idx = min(i + batch_size, len(all_chunks))
|
||||
collection.add(
|
||||
documents=all_chunks[i:end_idx],
|
||||
embeddings=embeddings[i:end_idx],
|
||||
metadatas=all_metadatas[i:end_idx],
|
||||
ids=all_ids[i:end_idx]
|
||||
)
|
||||
|
||||
# Save updated sources with timestamps
|
||||
all_sources = load_sources()
|
||||
for source in sources:
|
||||
for s in all_sources:
|
||||
if s["id"] == source["id"]:
|
||||
s["last_indexed"] = source["last_indexed"]
|
||||
break
|
||||
save_sources(all_sources)
|
||||
|
||||
stats = {
|
||||
"collection": COLLECTION_NAME,
|
||||
"sources_processed": len(sources),
|
||||
"chunks_indexed": len(all_chunks),
|
||||
"indexed_at": datetime.now().isoformat()
|
||||
}
|
||||
|
||||
if not quiet:
|
||||
print(f"\nIndexed {len(all_chunks)} chunks from {len(sources)} source(s)")
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Index external documentation for RAG search"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--source", "-s",
|
||||
help="Index only this source ID"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--all", "-a",
|
||||
action="store_true",
|
||||
dest="all_sources",
|
||||
help="Index all configured sources"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--quiet", "-q",
|
||||
action="store_true",
|
||||
help="Suppress progress output"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--list", "-l",
|
||||
action="store_true",
|
||||
help="List configured sources"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--stats",
|
||||
action="store_true",
|
||||
help="Output stats as JSON"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.list:
|
||||
sources = load_sources()
|
||||
if sources:
|
||||
print(json.dumps(sources, indent=2))
|
||||
else:
|
||||
print("No documentation sources configured")
|
||||
print(f"Add sources with: add_doc_source.py")
|
||||
return
|
||||
|
||||
stats = index_docs(
|
||||
source_id=args.source,
|
||||
all_sources=args.all_sources,
|
||||
quiet=args.quiet
|
||||
)
|
||||
|
||||
if args.stats or "error" in stats:
|
||||
print(json.dumps(stats, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+286
@@ -0,0 +1,286 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
RAG Search - Personal Index Builder
|
||||
|
||||
Indexes ~/.claude/state files for semantic search.
|
||||
Chunks JSON files by key for optimal retrieval.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Generator
|
||||
|
||||
# Add venv site-packages to path
|
||||
VENV_PATH = Path(__file__).parent.parent / "venv" / "lib" / "python3.13" / "site-packages"
|
||||
if str(VENV_PATH) not in sys.path:
|
||||
sys.path.insert(0, str(VENV_PATH))
|
||||
|
||||
import chromadb
|
||||
from sentence_transformers import SentenceTransformer
|
||||
|
||||
# Constants
|
||||
STATE_DIR = Path.home() / ".claude" / "state"
|
||||
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||
CHROMA_DIR = DATA_DIR / "chroma"
|
||||
MODEL_NAME = "all-MiniLM-L6-v2"
|
||||
COLLECTION_NAME = "personal"
|
||||
|
||||
|
||||
def chunk_json_file(file_path: Path) -> Generator[tuple[str, dict], None, None]:
|
||||
"""
|
||||
Chunk a JSON file into searchable segments.
|
||||
|
||||
Strategy:
|
||||
- Arrays: Each item becomes a chunk
|
||||
- Objects with arrays: Each array item with parent context
|
||||
- Nested objects: Flatten with path prefix
|
||||
|
||||
Yields:
|
||||
(chunk_text, metadata) tuples
|
||||
"""
|
||||
try:
|
||||
with open(file_path) as f:
|
||||
data = json.load(f)
|
||||
except (json.JSONDecodeError, IOError) as e:
|
||||
print(f" Warning: Could not parse {file_path}: {e}", file=sys.stderr)
|
||||
return
|
||||
|
||||
rel_path = str(file_path.relative_to(STATE_DIR))
|
||||
base_metadata = {"file": rel_path}
|
||||
|
||||
def process_item(item: dict, context: str = "") -> Generator[tuple[str, dict], None, None]:
|
||||
"""Process a single item from JSON structure."""
|
||||
if isinstance(item, dict):
|
||||
# Check for common patterns in our state files
|
||||
|
||||
# Memory items (decisions, preferences, facts, projects)
|
||||
if "content" in item:
|
||||
text_parts = []
|
||||
if context:
|
||||
text_parts.append(f"[{context}]")
|
||||
text_parts.append(item.get("content", ""))
|
||||
if item.get("context"):
|
||||
text_parts.append(f"Context: {item['context']}")
|
||||
if item.get("rationale"):
|
||||
text_parts.append(f"Rationale: {item['rationale']}")
|
||||
|
||||
metadata = {**base_metadata}
|
||||
if item.get("date"):
|
||||
metadata["date"] = item["date"]
|
||||
if item.get("id"):
|
||||
metadata["id"] = item["id"]
|
||||
if item.get("status"):
|
||||
metadata["status"] = item["status"]
|
||||
|
||||
yield (" ".join(text_parts), metadata)
|
||||
return
|
||||
|
||||
# General instructions (memory)
|
||||
if "instruction" in item:
|
||||
text_parts = [item["instruction"]]
|
||||
metadata = {**base_metadata}
|
||||
if item.get("added"):
|
||||
metadata["date"] = item["added"]
|
||||
if item.get("status"):
|
||||
metadata["status"] = item["status"]
|
||||
yield (" ".join(text_parts), metadata)
|
||||
return
|
||||
|
||||
# Knowledge base entries
|
||||
if "fact" in item or "answer" in item:
|
||||
text = item.get("fact") or item.get("answer", "")
|
||||
if item.get("question"):
|
||||
text = f"Q: {item['question']} A: {text}"
|
||||
metadata = {**base_metadata}
|
||||
if item.get("category"):
|
||||
metadata["category"] = item["category"]
|
||||
yield (text, metadata)
|
||||
return
|
||||
|
||||
# Component registry entries
|
||||
if "name" in item and "description" in item:
|
||||
text = f"{item['name']}: {item['description']}"
|
||||
if item.get("triggers"):
|
||||
text += f" Triggers: {', '.join(item['triggers'])}"
|
||||
metadata = {**base_metadata, "type": item.get("type", "unknown")}
|
||||
yield (text, metadata)
|
||||
return
|
||||
|
||||
# Future considerations
|
||||
if "id" in item and "title" in item:
|
||||
text = f"{item.get('id', '')}: {item['title']}"
|
||||
if item.get("description"):
|
||||
text += f" - {item['description']}"
|
||||
if item.get("rationale"):
|
||||
text += f" Rationale: {item['rationale']}"
|
||||
metadata = {**base_metadata}
|
||||
if item.get("date_added"):
|
||||
metadata["date"] = item["date_added"]
|
||||
if item.get("status"):
|
||||
metadata["status"] = item["status"]
|
||||
yield (text, metadata)
|
||||
return
|
||||
|
||||
# System instructions - processes
|
||||
if "process" in item or "name" in item:
|
||||
parts = []
|
||||
if item.get("name"):
|
||||
parts.append(item["name"])
|
||||
if item.get("description"):
|
||||
parts.append(item["description"])
|
||||
if item.get("steps"):
|
||||
parts.append("Steps: " + " ".join(item["steps"]))
|
||||
if parts:
|
||||
yield (" - ".join(parts), {**base_metadata})
|
||||
return
|
||||
|
||||
# Fallback: stringify the whole object
|
||||
text = json.dumps(item, indent=None)
|
||||
if len(text) > 50: # Only index if substantial
|
||||
yield (text[:1000], {**base_metadata}) # Truncate very long items
|
||||
|
||||
elif isinstance(item, str) and len(item) > 20:
|
||||
yield (item, {**base_metadata})
|
||||
|
||||
# Process top-level structure
|
||||
if isinstance(data, list):
|
||||
for item in data:
|
||||
yield from process_item(item)
|
||||
elif isinstance(data, dict):
|
||||
# Handle nested arrays within objects
|
||||
for key, value in data.items():
|
||||
if isinstance(value, list):
|
||||
for item in value:
|
||||
yield from process_item(item, context=key)
|
||||
elif isinstance(value, dict):
|
||||
yield from process_item(value, context=key)
|
||||
elif isinstance(value, str) and len(value) > 20:
|
||||
yield (f"{key}: {value}", {**base_metadata})
|
||||
|
||||
|
||||
def find_json_files() -> list[Path]:
|
||||
"""Find all JSON files in the state directory."""
|
||||
files = []
|
||||
for pattern in ["*.json", "**/*.json"]:
|
||||
files.extend(STATE_DIR.glob(pattern))
|
||||
return sorted(set(files))
|
||||
|
||||
|
||||
def index_personal(quiet: bool = False, force: bool = False) -> dict:
|
||||
"""
|
||||
Index all personal state files.
|
||||
|
||||
Args:
|
||||
quiet: Suppress progress output
|
||||
force: Force reindex even if already exists
|
||||
|
||||
Returns:
|
||||
Summary statistics
|
||||
"""
|
||||
if not quiet:
|
||||
print(f"Indexing personal state from {STATE_DIR}")
|
||||
|
||||
# Initialize model and client
|
||||
model = SentenceTransformer(MODEL_NAME)
|
||||
CHROMA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
client = chromadb.PersistentClient(path=str(CHROMA_DIR))
|
||||
|
||||
# Delete and recreate collection for clean reindex
|
||||
try:
|
||||
client.delete_collection(COLLECTION_NAME)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
collection = client.create_collection(
|
||||
name=COLLECTION_NAME,
|
||||
metadata={"description": "Personal state files from ~/.claude/state"}
|
||||
)
|
||||
|
||||
# Find and process files
|
||||
files = find_json_files()
|
||||
if not quiet:
|
||||
print(f"Found {len(files)} JSON files")
|
||||
|
||||
total_chunks = 0
|
||||
chunks = []
|
||||
metadatas = []
|
||||
ids = []
|
||||
|
||||
for file_path in files:
|
||||
if not quiet:
|
||||
print(f" Processing: {file_path.relative_to(STATE_DIR)}")
|
||||
|
||||
for chunk_text, metadata in chunk_json_file(file_path):
|
||||
# Skip empty or very short chunks
|
||||
if not chunk_text or len(chunk_text.strip()) < 10:
|
||||
continue
|
||||
|
||||
chunk_id = f"personal_{total_chunks}"
|
||||
chunks.append(chunk_text)
|
||||
metadatas.append(metadata)
|
||||
ids.append(chunk_id)
|
||||
total_chunks += 1
|
||||
|
||||
# Batch embed and add to collection
|
||||
if chunks:
|
||||
if not quiet:
|
||||
print(f"Embedding {len(chunks)} chunks...")
|
||||
|
||||
embeddings = model.encode(chunks, show_progress_bar=not quiet).tolist()
|
||||
|
||||
# Add in batches (ChromaDB has limits)
|
||||
batch_size = 100
|
||||
for i in range(0, len(chunks), batch_size):
|
||||
end_idx = min(i + batch_size, len(chunks))
|
||||
collection.add(
|
||||
documents=chunks[i:end_idx],
|
||||
embeddings=embeddings[i:end_idx],
|
||||
metadatas=metadatas[i:end_idx],
|
||||
ids=ids[i:end_idx]
|
||||
)
|
||||
|
||||
stats = {
|
||||
"collection": COLLECTION_NAME,
|
||||
"files_processed": len(files),
|
||||
"chunks_indexed": total_chunks,
|
||||
"indexed_at": datetime.now().isoformat()
|
||||
}
|
||||
|
||||
if not quiet:
|
||||
print(f"\nIndexed {total_chunks} chunks from {len(files)} files")
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Index personal state files for RAG search"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--quiet", "-q",
|
||||
action="store_true",
|
||||
help="Suppress progress output"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--force", "-f",
|
||||
action="store_true",
|
||||
help="Force reindex even if already indexed"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--stats",
|
||||
action="store_true",
|
||||
help="Output stats as JSON"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
stats = index_personal(quiet=args.quiet, force=args.force)
|
||||
|
||||
if args.stats:
|
||||
print(json.dumps(stats, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+184
@@ -0,0 +1,184 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
RAG Search - Main search entry point
|
||||
|
||||
Searches personal and/or docs indexes for semantically similar content.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
# Add venv site-packages to path
|
||||
VENV_PATH = Path(__file__).parent.parent / "venv" / "lib" / "python3.13" / "site-packages"
|
||||
if str(VENV_PATH) not in sys.path:
|
||||
sys.path.insert(0, str(VENV_PATH))
|
||||
|
||||
import chromadb
|
||||
from sentence_transformers import SentenceTransformer
|
||||
|
||||
# Constants
|
||||
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||
CHROMA_DIR = DATA_DIR / "chroma"
|
||||
MODEL_NAME = "all-MiniLM-L6-v2"
|
||||
DEFAULT_TOP_K = 5
|
||||
|
||||
# Lazy-loaded globals
|
||||
_model: Optional[SentenceTransformer] = None
|
||||
_client: Optional[chromadb.PersistentClient] = None
|
||||
|
||||
|
||||
def get_model() -> SentenceTransformer:
|
||||
"""Lazy-load the embedding model."""
|
||||
global _model
|
||||
if _model is None:
|
||||
_model = SentenceTransformer(MODEL_NAME)
|
||||
return _model
|
||||
|
||||
|
||||
def get_client() -> chromadb.PersistentClient:
|
||||
"""Lazy-load the ChromaDB client."""
|
||||
global _client
|
||||
if _client is None:
|
||||
CHROMA_DIR.mkdir(parents=True, exist_ok=True)
|
||||
_client = chromadb.PersistentClient(path=str(CHROMA_DIR))
|
||||
return _client
|
||||
|
||||
|
||||
def search(
|
||||
query: str,
|
||||
index: Optional[str] = None,
|
||||
top_k: int = DEFAULT_TOP_K,
|
||||
) -> dict:
|
||||
"""
|
||||
Search for semantically similar content.
|
||||
|
||||
Args:
|
||||
query: The search query
|
||||
index: Which index to search ("personal", "docs", or None for both)
|
||||
top_k: Number of results to return per collection
|
||||
|
||||
Returns:
|
||||
dict with query, results, and metadata
|
||||
"""
|
||||
client = get_client()
|
||||
model = get_model()
|
||||
|
||||
# Embed the query
|
||||
query_embedding = model.encode(query).tolist()
|
||||
|
||||
# Determine which collections to search
|
||||
collections_to_search = []
|
||||
if index is None or index == "personal":
|
||||
try:
|
||||
collections_to_search.append(("personal", client.get_collection("personal")))
|
||||
except Exception:
|
||||
pass # Collection doesn't exist
|
||||
if index is None or index == "docs":
|
||||
try:
|
||||
collections_to_search.append(("docs", client.get_collection("docs")))
|
||||
except Exception:
|
||||
pass # Collection doesn't exist
|
||||
|
||||
if not collections_to_search:
|
||||
return {
|
||||
"query": query,
|
||||
"results": [],
|
||||
"searched_collections": [],
|
||||
"total_chunks_searched": 0,
|
||||
"error": f"No collections found for index: {index or 'any'}"
|
||||
}
|
||||
|
||||
# Search each collection
|
||||
all_results = []
|
||||
total_chunks = 0
|
||||
searched_collections = []
|
||||
|
||||
for coll_name, collection in collections_to_search:
|
||||
searched_collections.append(coll_name)
|
||||
count = collection.count()
|
||||
total_chunks += count
|
||||
|
||||
if count == 0:
|
||||
continue
|
||||
|
||||
results = collection.query(
|
||||
query_embeddings=[query_embedding],
|
||||
n_results=min(top_k, count),
|
||||
include=["documents", "metadatas", "distances"]
|
||||
)
|
||||
|
||||
# Process results
|
||||
if results["documents"] and results["documents"][0]:
|
||||
for i, (doc, metadata, distance) in enumerate(zip(
|
||||
results["documents"][0],
|
||||
results["metadatas"][0],
|
||||
results["distances"][0]
|
||||
)):
|
||||
# Convert distance to similarity score (cosine distance to similarity)
|
||||
score = 1 - (distance / 2) # Normalized for cosine distance
|
||||
all_results.append({
|
||||
"source": coll_name,
|
||||
"file": metadata.get("file", "unknown"),
|
||||
"chunk": doc,
|
||||
"score": round(score, 3),
|
||||
"metadata": {k: v for k, v in metadata.items() if k != "file"}
|
||||
})
|
||||
|
||||
# Sort by score and add ranks
|
||||
all_results.sort(key=lambda x: x["score"], reverse=True)
|
||||
for i, result in enumerate(all_results[:top_k]):
|
||||
result["rank"] = i + 1
|
||||
|
||||
return {
|
||||
"query": query,
|
||||
"results": all_results[:top_k],
|
||||
"searched_collections": searched_collections,
|
||||
"total_chunks_searched": total_chunks
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Search the RAG index for relevant content",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s "how did I configure ArgoCD sync?"
|
||||
%(prog)s --index personal "past decisions about caching"
|
||||
%(prog)s --index docs "k0s node maintenance"
|
||||
%(prog)s --top-k 10 "prometheus alerting rules"
|
||||
"""
|
||||
)
|
||||
parser.add_argument("query", help="Search query")
|
||||
parser.add_argument(
|
||||
"--index", "-i",
|
||||
choices=["personal", "docs"],
|
||||
help="Search only this index (default: both)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--top-k", "-k",
|
||||
type=int,
|
||||
default=DEFAULT_TOP_K,
|
||||
help=f"Number of results to return (default: {DEFAULT_TOP_K})"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--raw",
|
||||
action="store_true",
|
||||
help="Output raw JSON (default: formatted)"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
results = search(args.query, args.index, args.top_k)
|
||||
|
||||
if args.raw:
|
||||
print(json.dumps(results))
|
||||
else:
|
||||
print(json.dumps(results, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+230
@@ -0,0 +1,230 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
RAG Search - Test Suite
|
||||
|
||||
Tests all components of the RAG search skill.
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Constants
|
||||
SKILL_DIR = Path(__file__).parent.parent
|
||||
SCRIPTS_DIR = SKILL_DIR / "scripts"
|
||||
VENV_PYTHON = SKILL_DIR / "venv" / "bin" / "python"
|
||||
DATA_DIR = Path.home() / ".claude" / "data" / "rag-search"
|
||||
|
||||
|
||||
def run_script(script_name: str, args: list[str] = None) -> tuple[int, str, str]:
|
||||
"""Run a script and return (returncode, stdout, stderr)."""
|
||||
cmd = [str(VENV_PYTHON), str(SCRIPTS_DIR / script_name)]
|
||||
if args:
|
||||
cmd.extend(args)
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
return result.returncode, result.stdout, result.stderr
|
||||
|
||||
|
||||
def test_chromadb_embeddings():
|
||||
"""Test 1: ChromaDB + embeddings working."""
|
||||
print("Test 1: ChromaDB + embeddings...")
|
||||
|
||||
# Add venv to path and test imports
|
||||
venv_path = SKILL_DIR / "venv" / "lib" / "python3.13" / "site-packages"
|
||||
sys.path.insert(0, str(venv_path))
|
||||
|
||||
try:
|
||||
import chromadb
|
||||
from sentence_transformers import SentenceTransformer
|
||||
|
||||
# Test ChromaDB
|
||||
client = chromadb.PersistentClient(path=str(DATA_DIR / "chroma"))
|
||||
assert client is not None, "Failed to create ChromaDB client"
|
||||
|
||||
# Test embedding model
|
||||
model = SentenceTransformer("all-MiniLM-L6-v2")
|
||||
embedding = model.encode("test query")
|
||||
assert len(embedding) == 384, f"Expected 384 dimensions, got {len(embedding)}"
|
||||
|
||||
print(" PASS: ChromaDB and embeddings working")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f" FAIL: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def test_personal_index():
|
||||
"""Test 2: Personal index populated from ~/.claude/state."""
|
||||
print("Test 2: Personal index populated...")
|
||||
|
||||
# Check if collection exists and has data
|
||||
venv_path = SKILL_DIR / "venv" / "lib" / "python3.13" / "site-packages"
|
||||
if str(venv_path) not in sys.path:
|
||||
sys.path.insert(0, str(venv_path))
|
||||
|
||||
try:
|
||||
import chromadb
|
||||
|
||||
client = chromadb.PersistentClient(path=str(DATA_DIR / "chroma"))
|
||||
collection = client.get_collection("personal")
|
||||
count = collection.count()
|
||||
|
||||
assert count > 0, f"Personal collection is empty (count={count})"
|
||||
print(f" PASS: Personal index has {count} chunks")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f" FAIL: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def test_docs_index():
|
||||
"""Test 3: At least one external doc source indexed."""
|
||||
print("Test 3: External docs indexed...")
|
||||
|
||||
# Check if collection exists and has data
|
||||
venv_path = SKILL_DIR / "venv" / "lib" / "python3.13" / "site-packages"
|
||||
if str(venv_path) not in sys.path:
|
||||
sys.path.insert(0, str(venv_path))
|
||||
|
||||
try:
|
||||
import chromadb
|
||||
|
||||
client = chromadb.PersistentClient(path=str(DATA_DIR / "chroma"))
|
||||
collection = client.get_collection("docs")
|
||||
count = collection.count()
|
||||
|
||||
assert count > 0, f"Docs collection is empty (count={count})"
|
||||
|
||||
# Also verify sources.json has at least one source
|
||||
sources_file = SKILL_DIR / "references" / "sources.json"
|
||||
with open(sources_file) as f:
|
||||
sources = json.load(f)
|
||||
assert len(sources.get("sources", [])) > 0, "No sources configured"
|
||||
|
||||
print(f" PASS: Docs index has {count} chunks from {len(sources['sources'])} source(s)")
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f" FAIL: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def test_search_returns_results():
|
||||
"""Test 4: search.py returns relevant results."""
|
||||
print("Test 4: Search returns relevant results...")
|
||||
|
||||
# Test personal search
|
||||
returncode, stdout, stderr = run_script("search.py", ["--index", "personal", "decisions"])
|
||||
if returncode != 0:
|
||||
print(f" FAIL: Personal search failed: {stderr}")
|
||||
return False
|
||||
|
||||
try:
|
||||
result = json.loads(stdout)
|
||||
personal_results = result.get("results", [])
|
||||
if not personal_results:
|
||||
print(" WARN: No personal results found (may be expected if state is minimal)")
|
||||
except json.JSONDecodeError:
|
||||
print(f" FAIL: Invalid JSON output: {stdout}")
|
||||
return False
|
||||
|
||||
# Test docs search
|
||||
returncode, stdout, stderr = run_script("search.py", ["--index", "docs", "kubernetes"])
|
||||
if returncode != 0:
|
||||
print(f" FAIL: Docs search failed: {stderr}")
|
||||
return False
|
||||
|
||||
try:
|
||||
result = json.loads(stdout)
|
||||
docs_results = result.get("results", [])
|
||||
if not docs_results:
|
||||
print(" FAIL: No docs results found for 'kubernetes'")
|
||||
return False
|
||||
except json.JSONDecodeError:
|
||||
print(f" FAIL: Invalid JSON output: {stdout}")
|
||||
return False
|
||||
|
||||
# Test combined search
|
||||
returncode, stdout, stderr = run_script("search.py", ["configuration"])
|
||||
if returncode != 0:
|
||||
print(f" FAIL: Combined search failed: {stderr}")
|
||||
return False
|
||||
|
||||
try:
|
||||
result = json.loads(stdout)
|
||||
assert "query" in result, "Missing 'query' in output"
|
||||
assert "results" in result, "Missing 'results' in output"
|
||||
assert "searched_collections" in result, "Missing 'searched_collections'"
|
||||
assert len(result["searched_collections"]) == 2, "Should search both collections"
|
||||
except json.JSONDecodeError:
|
||||
print(f" FAIL: Invalid JSON output: {stdout}")
|
||||
return False
|
||||
|
||||
print(f" PASS: Search returns properly formatted results")
|
||||
return True
|
||||
|
||||
|
||||
def test_skill_structure():
|
||||
"""Test 5: All required files exist."""
|
||||
print("Test 5: Skill structure complete...")
|
||||
|
||||
required_files = [
|
||||
SKILL_DIR / "SKILL.md",
|
||||
SCRIPTS_DIR / "search.py",
|
||||
SCRIPTS_DIR / "index_personal.py",
|
||||
SCRIPTS_DIR / "index_docs.py",
|
||||
SCRIPTS_DIR / "add_doc_source.py",
|
||||
SKILL_DIR / "references" / "sources.json",
|
||||
]
|
||||
|
||||
missing = []
|
||||
for f in required_files:
|
||||
if not f.exists():
|
||||
missing.append(str(f.relative_to(SKILL_DIR)))
|
||||
|
||||
if missing:
|
||||
print(f" FAIL: Missing files: {', '.join(missing)}")
|
||||
return False
|
||||
|
||||
print(" PASS: All required files exist")
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
print("=" * 60)
|
||||
print("RAG Search Test Suite")
|
||||
print("=" * 60)
|
||||
print()
|
||||
|
||||
tests = [
|
||||
test_chromadb_embeddings,
|
||||
test_personal_index,
|
||||
test_docs_index,
|
||||
test_search_returns_results,
|
||||
test_skill_structure,
|
||||
]
|
||||
|
||||
results = []
|
||||
for test in tests:
|
||||
results.append(test())
|
||||
print()
|
||||
|
||||
print("=" * 60)
|
||||
print("Summary")
|
||||
print("=" * 60)
|
||||
|
||||
passed = sum(results)
|
||||
total = len(results)
|
||||
print(f"Passed: {passed}/{total}")
|
||||
|
||||
if passed == total:
|
||||
print("\nAll tests passed!")
|
||||
return 0
|
||||
else:
|
||||
print(f"\n{total - passed} test(s) failed")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+313
-153
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"version": "1.1",
|
||||
"generated": "2026-01-01T10:30:00.000000-08:00",
|
||||
"version": "1.0",
|
||||
"generated": "2026-01-04T14:29:44.138959-08:00",
|
||||
"description": "Component registry for PA session awareness. Read at session start for routing.",
|
||||
"skills": {
|
||||
"sysadmin-health": {
|
||||
@@ -78,6 +78,18 @@
|
||||
"history"
|
||||
]
|
||||
},
|
||||
"morning-report": {
|
||||
"description": "Generate daily morning dashboard with email, calendar, stocks, weather, tasks, infra, and news",
|
||||
"script": "~/.claude/skills/morning-report/scripts/generate.py",
|
||||
"triggers": [
|
||||
"morning report",
|
||||
"morning",
|
||||
"daily report",
|
||||
"dashboard",
|
||||
"briefing",
|
||||
"daily briefing"
|
||||
]
|
||||
},
|
||||
"stock-lookup": {
|
||||
"description": "Look up stock prices and quotes",
|
||||
"script": "~/.claude/skills/stock-lookup/scripts/quote.py",
|
||||
@@ -92,23 +104,32 @@
|
||||
"performance"
|
||||
]
|
||||
},
|
||||
"morning-report": {
|
||||
"description": "Generate daily morning dashboard with email, calendar, stocks, weather, tasks, infra, and news",
|
||||
"script": "~/.claude/skills/morning-report/scripts/generate.py",
|
||||
"rag-search": {
|
||||
"description": "Semantic search across personal state files and external documentation (k0s, etc.)",
|
||||
"script": "~/.claude/skills/rag-search/scripts/search.py",
|
||||
"triggers": [
|
||||
"morning report",
|
||||
"morning",
|
||||
"daily report",
|
||||
"dashboard",
|
||||
"briefing",
|
||||
"daily briefing"
|
||||
"search",
|
||||
"find",
|
||||
"lookup",
|
||||
"what did",
|
||||
"how did",
|
||||
"when did",
|
||||
"past decisions",
|
||||
"previous",
|
||||
"documentation",
|
||||
"docs",
|
||||
"remember",
|
||||
"history"
|
||||
]
|
||||
}
|
||||
},
|
||||
"commands": {
|
||||
"/pa": {
|
||||
"description": "Personal assistant entrypoint",
|
||||
"aliases": ["/assistant", "/ask"],
|
||||
"aliases": [
|
||||
"/assistant",
|
||||
"/ask"
|
||||
],
|
||||
"invokes": "agent:personal-assistant"
|
||||
},
|
||||
"/programmer": {
|
||||
@@ -118,24 +139,160 @@
|
||||
},
|
||||
"/gcal": {
|
||||
"description": "Google Calendar access",
|
||||
"aliases": ["/calendar", "/cal"],
|
||||
"aliases": [
|
||||
"/calendar",
|
||||
"/cal"
|
||||
],
|
||||
"invokes": "skill:gcal"
|
||||
},
|
||||
"/stock": {
|
||||
"description": "Stock price lookup",
|
||||
"aliases": ["/quote", "/ticker"],
|
||||
"invokes": "skill:stock-lookup"
|
||||
},
|
||||
"/morning": {
|
||||
"description": "Generate morning report dashboard",
|
||||
"aliases": ["/briefing", "/daily"],
|
||||
"invokes": "skill:morning-report"
|
||||
},
|
||||
"/usage": {
|
||||
"description": "View usage statistics",
|
||||
"aliases": ["/stats"],
|
||||
"aliases": [
|
||||
"/stats"
|
||||
],
|
||||
"invokes": "skill:usage"
|
||||
},
|
||||
"/README": {
|
||||
"description": "TODO",
|
||||
"aliases": [],
|
||||
"invokes": ""
|
||||
},
|
||||
"/agent-info": {
|
||||
"description": "Show agent information",
|
||||
"aliases": [
|
||||
"/agent",
|
||||
"/agents"
|
||||
],
|
||||
"invokes": "command:agent-info"
|
||||
},
|
||||
"/config": {
|
||||
"description": "View and manage configuration settings",
|
||||
"aliases": [
|
||||
"/settings",
|
||||
"/prefs"
|
||||
],
|
||||
"invokes": "command:config"
|
||||
},
|
||||
"/debug": {
|
||||
"description": "Debug and troubleshoot configuration",
|
||||
"aliases": [
|
||||
"/diag",
|
||||
"/diagnose"
|
||||
],
|
||||
"invokes": "command:debug"
|
||||
},
|
||||
"/diff": {
|
||||
"description": "Compare config with backup",
|
||||
"aliases": [
|
||||
"/config-diff",
|
||||
"/compare"
|
||||
],
|
||||
"invokes": "command:diff"
|
||||
},
|
||||
"/export": {
|
||||
"description": "Export session data for sharing",
|
||||
"aliases": [
|
||||
"/session-export",
|
||||
"/share"
|
||||
],
|
||||
"invokes": "command:export"
|
||||
},
|
||||
"/help": {
|
||||
"description": "Show available commands and skills",
|
||||
"aliases": [
|
||||
"/commands",
|
||||
"/skills"
|
||||
],
|
||||
"invokes": "command:help"
|
||||
},
|
||||
"/log": {
|
||||
"description": "View and analyze logs",
|
||||
"aliases": [
|
||||
"/logs",
|
||||
"/logview"
|
||||
],
|
||||
"invokes": "command:log"
|
||||
},
|
||||
"/maintain": {
|
||||
"description": "Configuration maintenance (backup, validate, etc.)",
|
||||
"aliases": [
|
||||
"/maintenance",
|
||||
"/admin"
|
||||
],
|
||||
"invokes": "command:maintain"
|
||||
},
|
||||
"/mcp-status": {
|
||||
"description": "Check MCP integration status",
|
||||
"aliases": [
|
||||
"/mcp",
|
||||
"/integrations"
|
||||
],
|
||||
"invokes": "command:mcp-status"
|
||||
},
|
||||
"/remember": {
|
||||
"description": "Quick shortcut to save something to memory",
|
||||
"aliases": [
|
||||
"/save",
|
||||
"/note"
|
||||
],
|
||||
"invokes": "command:remember"
|
||||
},
|
||||
"/search": {
|
||||
"description": "Search memory, history, and configuration",
|
||||
"aliases": [
|
||||
"/find",
|
||||
"/lookup"
|
||||
],
|
||||
"invokes": "command:search"
|
||||
},
|
||||
"/rag": {
|
||||
"description": "Semantic search across state files and documentation",
|
||||
"aliases": [
|
||||
"/rag-search",
|
||||
"/semantic-search"
|
||||
],
|
||||
"invokes": "skill:rag-search"
|
||||
},
|
||||
"/skill-info": {
|
||||
"description": "Show skill information",
|
||||
"aliases": [
|
||||
"/skill",
|
||||
"/skills-info"
|
||||
],
|
||||
"invokes": "command:skill-info"
|
||||
},
|
||||
"/status": {
|
||||
"description": "Quick status overview across all domains",
|
||||
"aliases": [
|
||||
"/overview",
|
||||
"/dashboard"
|
||||
],
|
||||
"invokes": "command:status"
|
||||
},
|
||||
"/summarize": {
|
||||
"description": "Summarize and save session to memory",
|
||||
"aliases": [
|
||||
"/save-session",
|
||||
"/session-summary"
|
||||
],
|
||||
"invokes": "command:summarize"
|
||||
},
|
||||
"/template": {
|
||||
"description": "Manage session templates",
|
||||
"aliases": [
|
||||
"/templates",
|
||||
"/session-template"
|
||||
],
|
||||
"invokes": "command:template"
|
||||
},
|
||||
"/workflow": {
|
||||
"description": "List and describe workflows",
|
||||
"aliases": [
|
||||
"/workflows",
|
||||
"/wf"
|
||||
],
|
||||
"invokes": "command:workflow"
|
||||
},
|
||||
"/sysadmin:health": {
|
||||
"description": "System health check",
|
||||
"aliases": [],
|
||||
@@ -166,137 +323,125 @@
|
||||
"aliases": [],
|
||||
"invokes": "agent:k8s-diagnostician"
|
||||
},
|
||||
"/help": {
|
||||
"description": "Show available commands and skills",
|
||||
"aliases": ["/commands", "/skills"],
|
||||
"invokes": "command:help"
|
||||
"/stock": {
|
||||
"description": "Stock price lookup",
|
||||
"aliases": [
|
||||
"/quote",
|
||||
"/ticker"
|
||||
],
|
||||
"invokes": "skill:stock-lookup",
|
||||
"status": "removed"
|
||||
},
|
||||
"/status": {
|
||||
"description": "Quick status overview across all domains",
|
||||
"aliases": ["/overview", "/dashboard"],
|
||||
"invokes": "command:status"
|
||||
},
|
||||
"/summarize": {
|
||||
"description": "Summarize and save session to memory",
|
||||
"aliases": ["/save-session", "/session-summary"],
|
||||
"invokes": "command:summarize"
|
||||
},
|
||||
"/maintain": {
|
||||
"description": "Configuration maintenance (backup, validate, etc.)",
|
||||
"aliases": ["/maintenance", "/admin"],
|
||||
"invokes": "command:maintain"
|
||||
},
|
||||
"/remember": {
|
||||
"description": "Quick shortcut to save something to memory",
|
||||
"aliases": ["/save", "/note"],
|
||||
"invokes": "command:remember"
|
||||
},
|
||||
"/config": {
|
||||
"description": "View and manage configuration settings",
|
||||
"aliases": ["/settings", "/prefs"],
|
||||
"invokes": "command:config"
|
||||
},
|
||||
"/search": {
|
||||
"description": "Search memory, history, and configuration",
|
||||
"aliases": ["/find", "/lookup"],
|
||||
"invokes": "command:search"
|
||||
},
|
||||
"/log": {
|
||||
"description": "View and analyze logs",
|
||||
"aliases": ["/logs", "/logview"],
|
||||
"invokes": "command:log"
|
||||
},
|
||||
"/debug": {
|
||||
"description": "Debug and troubleshoot configuration",
|
||||
"aliases": ["/diag", "/diagnose"],
|
||||
"invokes": "command:debug"
|
||||
},
|
||||
"/export": {
|
||||
"description": "Export session data for sharing",
|
||||
"aliases": ["/session-export", "/share"],
|
||||
"invokes": "command:export"
|
||||
},
|
||||
"/mcp-status": {
|
||||
"description": "Check MCP integration status",
|
||||
"aliases": ["/mcp", "/integrations"],
|
||||
"invokes": "command:mcp-status"
|
||||
},
|
||||
"/workflow": {
|
||||
"description": "List and describe workflows",
|
||||
"aliases": ["/workflows", "/wf"],
|
||||
"invokes": "command:workflow"
|
||||
},
|
||||
"/skill-info": {
|
||||
"description": "Show skill information",
|
||||
"aliases": ["/skill", "/skills-info"],
|
||||
"invokes": "command:skill-info"
|
||||
},
|
||||
"/agent-info": {
|
||||
"description": "Show agent information",
|
||||
"aliases": ["/agent", "/agents"],
|
||||
"invokes": "command:agent-info"
|
||||
},
|
||||
"/diff": {
|
||||
"description": "Compare config with backup",
|
||||
"aliases": ["/config-diff", "/compare"],
|
||||
"invokes": "command:diff"
|
||||
},
|
||||
"/template": {
|
||||
"description": "Manage session templates",
|
||||
"aliases": ["/templates", "/session-template"],
|
||||
"invokes": "command:template"
|
||||
"/morning": {
|
||||
"description": "Generate morning report dashboard",
|
||||
"aliases": [
|
||||
"/briefing",
|
||||
"/daily"
|
||||
],
|
||||
"invokes": "skill:morning-report",
|
||||
"status": "removed"
|
||||
}
|
||||
},
|
||||
"agents": {
|
||||
"linux-sysadmin": {
|
||||
"description": "Workstation management",
|
||||
"model": "sonnet",
|
||||
"triggers": ["system", "linux", "package", "service", "disk", "process"]
|
||||
"triggers": [
|
||||
"system",
|
||||
"linux",
|
||||
"package",
|
||||
"service",
|
||||
"disk",
|
||||
"process"
|
||||
]
|
||||
},
|
||||
"k8s-orchestrator": {
|
||||
"description": "Kubernetes cluster management",
|
||||
"model": "opus",
|
||||
"triggers": ["kubernetes", "k8s", "cluster", "deploy"]
|
||||
"triggers": [
|
||||
"kubernetes",
|
||||
"k8s",
|
||||
"cluster",
|
||||
"deploy"
|
||||
]
|
||||
},
|
||||
"k8s-diagnostician": {
|
||||
"description": "Kubernetes troubleshooting",
|
||||
"model": "sonnet",
|
||||
"triggers": ["pod issue", "crashloop", "k8s error", "deployment failed"]
|
||||
"triggers": [
|
||||
"pod issue",
|
||||
"crashloop",
|
||||
"k8s error",
|
||||
"deployment failed"
|
||||
]
|
||||
},
|
||||
"argocd-operator": {
|
||||
"description": "ArgoCD GitOps operations",
|
||||
"model": "sonnet",
|
||||
"triggers": ["argocd", "gitops", "sync", "app sync"]
|
||||
"triggers": [
|
||||
"argocd",
|
||||
"gitops",
|
||||
"sync",
|
||||
"app sync"
|
||||
]
|
||||
},
|
||||
"prometheus-analyst": {
|
||||
"description": "Metrics and alerting analysis",
|
||||
"model": "sonnet",
|
||||
"triggers": ["metrics", "prometheus", "alert", "grafana"]
|
||||
"triggers": [
|
||||
"metrics",
|
||||
"prometheus",
|
||||
"alert",
|
||||
"grafana"
|
||||
]
|
||||
},
|
||||
"git-operator": {
|
||||
"description": "Git repository operations",
|
||||
"model": "sonnet",
|
||||
"triggers": ["git", "commit", "branch", "merge", "repo"]
|
||||
"triggers": [
|
||||
"git",
|
||||
"commit",
|
||||
"branch",
|
||||
"merge",
|
||||
"repo"
|
||||
]
|
||||
},
|
||||
"programmer-orchestrator": {
|
||||
"description": "Code development coordination",
|
||||
"model": "opus",
|
||||
"triggers": ["code", "develop", "implement", "program"]
|
||||
"triggers": [
|
||||
"code",
|
||||
"develop",
|
||||
"implement",
|
||||
"program"
|
||||
]
|
||||
},
|
||||
"code-planner": {
|
||||
"description": "Code planning and design",
|
||||
"model": "sonnet",
|
||||
"triggers": ["plan code", "design", "architecture"]
|
||||
"triggers": [
|
||||
"plan code",
|
||||
"design",
|
||||
"architecture"
|
||||
]
|
||||
},
|
||||
"code-implementer": {
|
||||
"description": "Code implementation",
|
||||
"model": "sonnet",
|
||||
"triggers": ["write code", "implement", "build"]
|
||||
"triggers": [
|
||||
"write code",
|
||||
"implement",
|
||||
"build"
|
||||
]
|
||||
},
|
||||
"code-reviewer": {
|
||||
"description": "Code review",
|
||||
"model": "sonnet",
|
||||
"triggers": ["review", "code review", "check code"]
|
||||
"triggers": [
|
||||
"review",
|
||||
"code review",
|
||||
"check code"
|
||||
]
|
||||
},
|
||||
"master-orchestrator": {
|
||||
"description": "Coordinate and enforce policies",
|
||||
@@ -306,49 +451,94 @@
|
||||
"personal-assistant": {
|
||||
"description": "User interface, ultimate oversight",
|
||||
"model": "opus",
|
||||
"triggers": ["help", "assist", "question"]
|
||||
"triggers": [
|
||||
"help",
|
||||
"assist",
|
||||
"question"
|
||||
]
|
||||
},
|
||||
"README": {
|
||||
"description": "TODO",
|
||||
"model": "sonnet",
|
||||
"triggers": [
|
||||
"TODO"
|
||||
]
|
||||
}
|
||||
},
|
||||
"workflows": {
|
||||
"validate-agent-format": {
|
||||
"description": "Validate agent file format",
|
||||
"triggers": ["validate agent", "check agent format"]
|
||||
"triggers": [
|
||||
"validate agent",
|
||||
"check agent format"
|
||||
]
|
||||
},
|
||||
"health/cluster-health-check": {
|
||||
"description": "Kubernetes cluster health check",
|
||||
"triggers": ["cluster health", "k8s health"]
|
||||
"triggers": [
|
||||
"cluster health",
|
||||
"k8s health"
|
||||
]
|
||||
},
|
||||
"health/cluster-daily-summary": {
|
||||
"description": "Daily cluster health summary",
|
||||
"triggers": ["daily summary", "cluster summary"]
|
||||
"triggers": [
|
||||
"daily summary",
|
||||
"cluster summary"
|
||||
]
|
||||
},
|
||||
"deploy/deploy-app": {
|
||||
"description": "Deploy application to Kubernetes",
|
||||
"triggers": ["deploy app", "deploy to k8s"]
|
||||
"triggers": [
|
||||
"deploy app",
|
||||
"deploy to k8s"
|
||||
]
|
||||
},
|
||||
"incidents/pod-crashloop": {
|
||||
"description": "Handle pod crashloop",
|
||||
"triggers": ["crashloop", "pod crashing", "restart loop"]
|
||||
"triggers": [
|
||||
"crashloop",
|
||||
"pod crashing",
|
||||
"restart loop"
|
||||
]
|
||||
},
|
||||
"incidents/node-issue-response": {
|
||||
"description": "Respond to node issues",
|
||||
"triggers": ["node issue", "node down", "node problem"]
|
||||
"triggers": [
|
||||
"node issue",
|
||||
"node down",
|
||||
"node problem"
|
||||
]
|
||||
},
|
||||
"incidents/resource-pressure-response": {
|
||||
"description": "Handle resource pressure",
|
||||
"triggers": ["resource pressure", "out of memory", "disk full"]
|
||||
"triggers": [
|
||||
"resource pressure",
|
||||
"out of memory",
|
||||
"disk full"
|
||||
]
|
||||
},
|
||||
"incidents/argocd-sync-failure": {
|
||||
"description": "Handle ArgoCD sync failures",
|
||||
"triggers": ["sync failed", "argocd error"]
|
||||
"triggers": [
|
||||
"sync failed",
|
||||
"argocd error"
|
||||
]
|
||||
},
|
||||
"sysadmin/health-check": {
|
||||
"description": "System health check workflow",
|
||||
"triggers": ["system check", "health check"]
|
||||
"triggers": [
|
||||
"system check",
|
||||
"health check"
|
||||
]
|
||||
},
|
||||
"sysadmin/system-update": {
|
||||
"description": "System update workflow",
|
||||
"triggers": ["system update", "update packages", "upgrade"]
|
||||
"triggers": [
|
||||
"system update",
|
||||
"update packages",
|
||||
"upgrade"
|
||||
]
|
||||
}
|
||||
},
|
||||
"delegation_helpers": {
|
||||
@@ -360,35 +550,5 @@
|
||||
"description": "Calendar API with tiered delegation",
|
||||
"location": "~/.claude/mcp/delegation/gcal_delegate.py"
|
||||
}
|
||||
},
|
||||
"automation": {
|
||||
"scripts": {
|
||||
"validate-setup": "~/.claude/automation/validate-setup.sh",
|
||||
"quick-status": "~/.claude/automation/quick-status.sh",
|
||||
"backup": "~/.claude/automation/backup.sh",
|
||||
"restore": "~/.claude/automation/restore.sh",
|
||||
"clean": "~/.claude/automation/clean.sh",
|
||||
"install": "~/.claude/automation/install.sh",
|
||||
"test": "~/.claude/automation/test-scripts.sh",
|
||||
"memory-add": "~/.claude/automation/memory-add.py",
|
||||
"memory-list": "~/.claude/automation/memory-list.py",
|
||||
"search": "~/.claude/automation/search.py",
|
||||
"history-browser": "~/.claude/automation/history-browser.py",
|
||||
"log-viewer": "~/.claude/automation/log-viewer.py",
|
||||
"debug": "~/.claude/automation/debug.sh",
|
||||
"daily-maintenance": "~/.claude/automation/daily-maintenance.sh",
|
||||
"session-export": "~/.claude/automation/session-export.py",
|
||||
"mcp-status": "~/.claude/automation/mcp-status.sh",
|
||||
"upgrade": "~/.claude/automation/upgrade.sh",
|
||||
"workflow-info": "~/.claude/automation/workflow-info.py",
|
||||
"skill-info": "~/.claude/automation/skill-info.py",
|
||||
"agent-info": "~/.claude/automation/agent-info.py",
|
||||
"config-diff": "~/.claude/automation/config-diff.py",
|
||||
"session-template": "~/.claude/automation/session-template.py"
|
||||
},
|
||||
"completions": {
|
||||
"bash": "~/.claude/automation/completions.bash",
|
||||
"zsh": "~/.claude/automation/completions.zsh"
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -189,6 +189,41 @@
|
||||
"ended": null,
|
||||
"summarized": false,
|
||||
"topics": []
|
||||
},
|
||||
{
|
||||
"id": "2026-01-04_13-30-26",
|
||||
"started": "2026-01-04T13:30:26-08:00",
|
||||
"ended": null,
|
||||
"summarized": false,
|
||||
"topics": []
|
||||
},
|
||||
{
|
||||
"id": "2026-01-04_14-20-18",
|
||||
"started": "2026-01-04T14:20:18-08:00",
|
||||
"ended": null,
|
||||
"summarized": false,
|
||||
"topics": []
|
||||
},
|
||||
{
|
||||
"id": "2026-01-04_23-21-33",
|
||||
"started": "2026-01-04T23:21:33-08:00",
|
||||
"ended": null,
|
||||
"summarized": false,
|
||||
"topics": []
|
||||
},
|
||||
{
|
||||
"id": "2026-01-04_23-22-43",
|
||||
"started": "2026-01-04T23:22:43-08:00",
|
||||
"ended": null,
|
||||
"summarized": false,
|
||||
"topics": []
|
||||
},
|
||||
{
|
||||
"id": "2026-01-04_23-22-43",
|
||||
"started": "2026-01-04T23:22:43-08:00",
|
||||
"ended": null,
|
||||
"summarized": false,
|
||||
"topics": []
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user