---
title: n8n Implementation Handoff
area: automation
tags: [automation, n8n, handoff, atlas]
created: 2026-05-13
updated: 2026-05-13
status: active
related: [[Infrastructure/Automation/n8n Workflows]], [[Infrastructure/Automation/n8n IMAP Triage Pipeline]], [[Infrastructure/Automation/n8n Morning Brief]], [[Infrastructure/Automation/n8n Evening Digest]], [[Infrastructure/Services/Docker Services]]
---

# n8n Implementation Handoff

## Current state

n8n is running on `n8n-agent` at `http://127.0.0.1:18808`.

Last verified on 2026-05-13 (evening):

- Container: `n8n-agent` running and healthy.
- Health endpoint: `GET /healthz` returns `{"status":"ok"}`.
- Workflow export: `n8n export:workflow --all` succeeds.
- Active workflows: 13.
- Inactive workflows: 1 (Nightly Obsidian Vault Sync replaced by Evening Digest).
- Archived workflows: 2 unrecoverable duplicate IMAP workflows archived after SQLite recovery.
- Docker health endpoint: `GET :18809/health` returns container state for 7 services.
- Systemd user service `docker-health-endpoint.service` active and enabled.
- Obsidian reindex endpoint: `POST :18810/reindex` triggers incremental vault reindex.
- Systemd user service `obsidian-reindex-endpoint.service` active and enabled.

## Implemented and active

### Swarm Health Watchdog

- Workflow ID: `lDKocSFXBQWQrDd3`
- Status: active
- Trigger: every 15 minutes
- Checks:
  - Brave MCP `18802`
  - SearXNG `18803`
  - LiteLLM `18804`
  - Kokoro `18805`
  - llama.cpp `18806`
  - Ollama embeddings `18807`
  - n8n `18808`
  - Whisper `18811`
- Current behavior:
  - Runs from inside the n8n container.
  - Uses Docker bridge gateway `172.19.0.1` for host-published services.
  - Uses `127.0.0.1:5678` for n8n self-health.
  - Alerts Telegram and Discord after two consecutive failures.
  - Sends recovery messages when a previously-alerted service becomes healthy again.
- Docker health enrichment: implemented 2026-05-13.
  - Host-side Python HTTP endpoint at `~/lab/swarm/scripts/docker-health-server.py` on port `18809`.
  - Systemd user service `docker-health-endpoint.service`.
  - Watchdog now calls `http://172.19.0.1:18809/health` and includes container status/health/restarts in alerts.
- Remaining improvement:
  - Tune alert deduplication thresholds after observing real failure patterns.

### n8n Failure Digest

- Workflow ID: `G9ylNbHbnJ6fWX2C`
- Status: active
- Trigger: hourly plus manual trigger
- Current behavior:
  - Queries recent failed n8n executions.
  - Groups failures by workflow/node/error.
  - Sends a concise Telegram digest.
- Discord delivery: implemented 2026-05-13.
  - Now delivers to both Telegram and Discord #ops-alerts in parallel.
- Remaining improvement:
  - Tune grouping/windowing after observing real failure volume.
  - Add first-seen/last-seen persistence if the current digest is too repetitive.

### Local AI Inbox Triage / IMAP

- Primary workflow ID: `9sFwRyUDz51csAp7`
- Status: active
- Current behavior:
  - Polls IMAP every 15 minutes.
  - Uses local LLM classification.
  - Sends signal digests to Telegram.
  - Writes per-email notes to Obsidian.
- Archived duplicates:
  - `xjUoQf97TkBrawc8` — `IMAP Inbox Triage + Obsidian Notes (squareffect)`
  - `kHDK9QdUSiAJ8rCM` — `IMAP Inbox Triage + Obsidian Notes (wills-portal)`
- Container URLs fixed 2026-05-13: stale `192.168.153.130` replaced with Docker bridge `172.19.0.1`.
- Remaining improvement:
  - Rebuild any needed second-mailbox triage flows from scratch instead of unarchiving the corrupted duplicates.
  - Add draft-reply generation for reply-needed messages.
  - Add clearer classes: `needs_reply`, `FYI`, `receipt`, `newsletter`, `noise`, `action_required`.
  - Add explicit Obsidian destinations per class.

### Gmail Inbox Monitor + Obsidian Notes

- Workflow ID: `whtdorf7yJMVYeHm`
- Status: active, OAuth trigger replaced with IMAP fallback on 2026-05-13
- Current behavior:
  - Original Gmail OAuth trigger failed because the stored Google OAuth client secret is invalid; refresh returns `invalid_client`.
  - The workflow now uses n8n `Email Trigger (IMAP)` against Gmail with the Hermes Gmail app-password credential `Gmail IMAP (Hermes App Password)`.
  - Fetches unseen INBOX mail, marks processed mail read, and tracks last message ID.
  - Local LLM and Obsidian REST calls use Docker bridge gateway `172.19.0.1` from inside the n8n container.
  - Workflow export succeeds and no new `Gmail Trigger` warning appeared immediately after activation.
- Remaining implementation:
  - Observe the next real Gmail ingestion event and confirm Telegram/Obsidian output shape.
  - Re-authorize Google OAuth only if Gmail API-specific features are needed later; otherwise keep IMAP fallback.
  - Add the same expanded triage classes and draft-reply behavior as the IMAP flow.

### Calendar to Obsidian Notes

- Workflow ID: `QRCCdHNXZUHc2Oz4`
- Status: active
- Current behavior:
  - Fetches upcoming calendar events.
  - Writes or updates notes in the shared vault.
- Remaining improvement:
  - Add meeting-prep note template.
  - Add after-meeting follow-up path that collects event + notes + transcript and produces action items.

### Evening Digest (replaces Nightly Obsidian Vault Sync)

- Old workflow ID: `75JCevkdgkyCr2qH` (deactivated)
- New workflow ID: `PlZywwqL8MRNEAN6`
- Status: active
- Trigger: daily at 21:00 America/Los_Angeles
- Current behavior:
  - Queries n8n executions API for today's successes and failures.
  - Checks Docker container health via `172.19.0.1:18809`.
  - Lists new Obsidian notes created today.
  - Synthesizes with local gemma-4-26B LLM.
  - Delivers to Telegram, Discord #ops-alerts, and Obsidian.
- Remaining improvement:
  - Add systemd timer health once a host-side endpoint exists.
  - Add reminders and task queue summary.

### Morning Brief

- Workflow ID: `g3IdGZCK1EtTsv9T`
- Status: active
- Trigger: daily at 06:30 America/Los_Angeles
- Current behavior:
  - Collects: weather (wttr.in/Seattle), swarm Docker health (`172.19.0.1:18809`), n8n self-health (`127.0.0.1:5678`), LiteLLM health (`172.19.0.1:18804`), email highlights via n8n API (recent IMAP Triage executions), Google Calendar (graceful skip if OAuth expired).
  - Synthesizes with local gemma-4-26B LLM (`172.19.0.1:18806`).
  - Delivers to Telegram (chat 8367012007, HTML format) and Obsidian (`Notes/YYYY-MM-DD Morning Brief.md` with YAML frontmatter).
  - All 6 data-collection nodes have `continueOnFail`; one failure does not block the brief.
- Remaining improvement:
  - Re-enable calendar section once Google OAuth is re-authorized.
  - Add relevant news via SearXNG or Brave Search.
  - Add tasks/reminders section from Obsidian notes tagged `action-required`.

### Daily OpenClaw Session Digest

- Workflow ID: `qqYwAD05AvRHrHPc`
- Status: active
- Trigger: nightly
- Current behavior:
  - Summarizes recent OpenClaw sessions into Obsidian.
- Remaining improvement:
  - Decide whether to rename/reframe this as a broader Atlas/Hermes daily agent digest.
  - Pull in Hermes cron/gateway/Kanban activity once useful.

### Voice Memo Capture

- Workflow ID: `El1BHJZ56JlzhrRZ`
- Status: active
- Type: webhook
- Current behavior:
  - Accepts three ingress modes: `audio_url`, `telegram_file_id`, or `discord_audio_url`.
  - Host-side processor on port `18813` (`voice-memo-processor.py`) handles download, Whisper transcription, and local LLM summarization.
  - Optional Kokoro TTS read-back of summary (`include_tts: true`).
  - Writes transcript/summary to Obsidian with YAML frontmatter including `source_type`.
  - Sends Telegram notification with source type and optional TTS audio link.
- Host-side service: `~/lab/swarm/scripts/voice-memo-processor.py` on port `18813`.
- Systemd user service: `voice-memo-processor.service` (enabled).
- Remaining improvement:
  - Add durable action-item routing to notes/task queue.
  - Test end-to-end with real Telegram voice messages.

### Web-to-Notes Capture

- Workflow ID: `GSmzuA5dgGgyRg5v`
- Status: active
- Type: webhook
- Current behavior:
  - Accepts a URL.
  - Host-side content extractor on port `18812` (`url-content-extractor.py`) classifies and extracts content.
  - Supports YouTube (transcript via `youtube-transcript-api`), PDF (text via `pymupdf`), and web (readable text via `readability-lxml`).
  - Summarizes with local llama.cpp.
  - Writes markdown to Obsidian with YAML frontmatter including `content_type`, `source_url`, `title`, `date`, and tags.
- Host-side service: `~/lab/swarm/scripts/url-content-extractor.py` on port `18812`.
- Systemd user service: `url-content-extractor.service` (enabled).
- Remaining improvement:
  - Add claim extraction and source metadata enrichment.
  - Add optional Atlas/Hermes higher-quality synthesis for important captures.

### OpenClaw Action Bus / Reminder Webhook

- Action Bus ID: `Jwi54VWMdlLqYnRo`
- Reminder Webhook ID: `RUR1CGn0ikkxbPin`
- Status: active
- Current behavior:
  - Provides stable webhook ingress for approved automation and reminders.
- Remaining improvement:
  - Define a clean Atlas-facing webhook catalog with paths, auth, payload schema, and response shape.
  - Add handlers for:
    - `process_url`
    - `summarize_pdf`
    - `add_reminder`
    - `sync_vault`
    - `run_health_check`
    - `process_voice_memo`

### Obsidian Vault Reindex

- Workflow ID: `85ntyyphDJ4Ms2b4`
- Status: active
- Trigger: every 6 hours
- Current behavior:
  - n8n schedule trigger calls `POST http://172.19.0.1:18810/reindex`.
  - Host-side `obsidian-reindex-server.py` on port `18810` runs the incremental Obsidian vault indexer.
  - Systemd user service `obsidian-reindex-endpoint.service`.

## Obsidian Semantic Index

Implemented 2026-05-13.

### Architecture

- **Vector store**: Hermes rag-search ChromaDB embedded at `~/.hermes/data/rag-search/chroma/`; live Obsidian semantic endpoint uses collection `obsidian_bge_npu`.
- **Embeddings**: OpenVINO Intel NPU service on `18817` using `bge-base-en-v1.5-int8-ov` (768-dim vectors). Legacy Ollama `nomic-embed-text` on `18807` remains available as rollback/comparison data.
- **Indexer**: `~/.hermes/skills/note-taking/rag-search/scripts/reindex_obsidian.py`
- **Chunking**: Markdown files are split by heading sections; long sections get sliding-window chunks (max 2000 chars, 200 char overlap). YAML frontmatter is extracted and stored as metadata.
- **Search**: `~/.hermes/skills/note-taking/rag-search/scripts/search.py --index obsidian "query"` or Hermes native `rag_search` tool.
- **Cross-collection search**: `search.py "query"` searches `personal`, `docs`, and `obsidian` using the appropriate embedding backend per collection.

### Live BGE/NPU state (2026-06-03)

- Collection: `obsidian_bge_npu`
- Notes indexed: 194
- Vector count: 466
- Embedding backend: `http://127.0.0.1:18817`
- Embedding model: `bge-base-en-v1.5-int8-ov`
- OpenVINO device: Intel NPU via `openvino-embeddings.service`
- Semantic health: `curl -fsS http://127.0.0.1:18810/semantic-health`
- Embedding health: `curl -fsS http://127.0.0.1:18817/health`

### Legacy index stats (2026-05-13)

- 36 markdown files indexed
- 231 chunks
- Embedding model: `nomic-embed-text` via Ollama
- Full index time: ~5 minutes (Ollama CPU inference at ~1.2s/text, batch=10)
- Incremental reindex (no changes): ~1.4 seconds

### Incremental updates

- File content SHA-256 hashes are tracked in the collection-specific state file, e.g. `~/.hermes/data/rag-search/obsidian_bge_npu_index_state.json` for the live BGE/NPU collection.
- Only changed files are re-indexed on subsequent runs.
- Deleted files have their chunks removed from ChromaDB.

### Automated reindex

- n8n workflow `Obsidian Vault Reindex` (`85ntyyphDJ4Ms2b4`) triggers every 6 hours.
- Calls `POST http://172.19.0.1:18810/reindex` (host-side endpoint).
- Host endpoint: `~/lab/swarm/scripts/obsidian-reindex-server.py` on port `18810`.
- Systemd service: `obsidian-reindex-endpoint.service` (enabled).
- Manual trigger: `curl -X POST http://127.0.0.1:18810/reindex`

### Verification commands

```bash
# Check index state
curl -fsS http://127.0.0.1:18810/reindex/status | python3 -m json.tool

# Trigger manual reindex
curl -X POST http://127.0.0.1:18810/reindex | python3 -m json.tool

# Search the Obsidian index
~/.hermes/skills/note-taking/rag-search/venv/bin/python \
  ~/.hermes/skills/note-taking/rag-search/scripts/search.py --index obsidian "health monitoring"

# Check live semantic and NPU embedding health
curl -fsS http://127.0.0.1:18810/semantic-health | python3 -m json.tool
curl -fsS http://127.0.0.1:18817/health | python3 -m json.tool

# Check ChromaDB data
du -sh ~/.hermes/data/rag-search/chroma/

# Check systemd service
systemctl --user status obsidian-reindex-endpoint.service

# Verify from inside n8n container
docker exec n8n-agent wget -qO- http://172.19.0.1:18810/healthz
```

## Not yet implemented

### Weekly review

Desired scope:

- Obsidian note changes.
- Calendar summary.
- GitHub/Linear activity.
- Inbox triage rollup.
- Open tasks and decisions.

Recommended implementation:

1. Use n8n weekly trigger.
2. Collect structured inputs from calendar, Obsidian, GitHub/Linear, email summaries.
3. Use Atlas/Hermes or cloud model for final synthesis.
4. Write `Notes/YYYY-MM-DD Weekly Review.md`.

### Personal data routing

Desired scope:

- Gmail/IMAP items to Obsidian.
- Telegram/Discord messages to notes or task queues.
- Calendar events to prep docs.
- YouTube/RSS/blog links to summaries.

Current status:

- Email and calendar pieces exist.
- URL capture exists.
- Telegram/Discord message-to-note routing is not fully built.
- RSS/blog monitoring is not fully built in n8n.

Recommended implementation:

1. Define routing taxonomy and destination folders/tags.
2. Implement one source at a time.
3. Avoid dumping noisy raw messages into Obsidian; use classification gates.

## Recommended next order of work

1. ~~Fix Gmail Trigger warning.~~ Done 2026-05-13: replaced with IMAP fallback.
2. ~~Add Docker health-state enrichment to Swarm Health Watchdog.~~ Done 2026-05-13: host-side endpoint on 18809, systemd service.
3. ~~Build Morning Brief v1.~~ Done 2026-05-13: workflow `g3IdGZCK1EtTsv9T`, daily 06:30 PT.
4. ~~Extend Evening Digest.~~ Done 2026-05-13: workflow `PlZywwqL8MRNEAN6`, daily 21:00 PT.
5. ~~Add Discord delivery to n8n Failure Digest.~~ Done 2026-05-13.
6. ~~Fix stale container URLs in IMAP workflow.~~ Done 2026-05-13.
7. ~~Implement Obsidian Semantic Index.~~ Done 2026-05-13: ChromaDB `obsidian` collection, Ollama nomic-embed-text, automated reindex every 6h.

8. ~~Upgrade Web-to-Notes Capture.~~ Done 2026-05-13: host-side content extractor on :18812, supports YouTube/PDF/web, workflow updated.
   - Remaining: claim extraction, Atlas/Hermes synthesis.

9. ~~Upgrade Voice Memo Pipeline.~~ Done 2026-05-13: host-side processor on :18813, Telegram/Discord voice ingress, Kokoro TTS read-back.
   - Remaining: test with real Telegram voice, action-item routing.

10. ~~Define webhook action bus catalog.~~ Done 2026-05-13: catalog at `Infrastructure/Automation/Webhook Action Bus.md`.
    - Remaining: implement `summarize_pdf`, `sync_vault`, `run_health_check` webhook wrappers.

## Verification commands

Run from host:

```bash
cd /home/will/lab/swarm
make status
curl -fsS --max-time 3 http://127.0.0.1:18808/healthz

docker inspect --format '{{.Name}} {{.State.Status}} {{if .State.Health}}{{.State.Health.Status}}{{else}}no-health{{end}} restarts={{.RestartCount}}' n8n-agent

docker exec n8n-agent n8n export:workflow --all --output=/tmp/workflows-verify.json

# Docker health endpoint (host-side systemd service)
curl -fsS --max-time 3 http://127.0.0.1:18809/health | python3 -m json.tool

# Obsidian reindex endpoint
curl -fsS http://127.0.0.1:18810/healthz
curl -fsS http://127.0.0.1:18810/reindex/status | python3 -m json.tool

# URL content extractor (Web-to-Notes)
curl -fsS http://127.0.0.1:18812/healthz

# Voice memo processor
curl -fsS http://127.0.0.1:18813/healthz

# Verify from inside n8n container
docker exec n8n-agent wget -qO- http://172.19.0.1:18809/health
docker exec n8n-agent wget -qO- http://172.19.0.1:18810/healthz
```

### n8n Public API access

The `N8N_API_KEY` in `~/lab/swarm/.env` is stale (returns 401). Get the working API key from the n8n credential store:

```bash
docker exec n8n-agent sh -lc 'n8n export:credentials --id=UPAHgUJVRqZQceL4 --decrypted --output=/tmp/n8n-api-cred.json >/tmp/n8n-api-cred.log 2>&1'
docker cp n8n-agent:/tmp/n8n-api-cred.json /tmp/n8n-api-cred.json
API_KEY=$(python3 -c "import json; c=json.load(open('/tmp/n8n-api-cred.json')); c=c[0] if isinstance(c,list) else c; print(c['data']['value'])")
rm -f /tmp/n8n-api-cred.json
docker exec n8n-agent rm -f /tmp/n8n-api-cred.json /tmp/n8n-api-cred.log

# Example: list workflows
curl -sS -H "X-N8N-API-KEY: $API_KEY" http://127.0.0.1:18808/api/v1/workflows?limit=100
```

Do NOT print the API key in logs or output.

Inspect workflow inventory from SQLite copy:

```bash
tmp=/tmp/n8n-check-$(date +%s)
mkdir -p "$tmp"
docker cp n8n-agent:/home/node/.n8n/database.sqlite "$tmp/database.sqlite"
sqlite3 "$tmp/database.sqlite" "select name, active, isArchived from workflow_entity order by name;"
```

Check logs without printing secrets:

```bash
docker logs n8n-agent --tail 120
```

## Pitfalls

- Do not assume `/healthz` means all workflows are valid. Verify with `n8n export:workflow --all`.
- If SQLite corruption returns, copy DB files first, then recover offline. Do not make blind in-place edits.
- Validate `workflow_entity` JSON-ish fields after SQLite recovery: `nodes`, `connections`, `settings`, `staticData`, `pinData`, `meta`.
- Empty strings in these fields can crash n8n even when SQLite integrity is `ok`.
- From inside n8n, host-published swarm services use Docker bridge gateway `172.19.0.1:<port>`.
- From inside n8n, n8n self-calls should use `127.0.0.1:5678`, not host port `18808`.
- `N8N_API_KEY` in `~/lab/swarm/.env` is stale and returns 401. Get the working key from n8n credential `UPAHgUJVRqZQceL4` (see Verification commands).
- Do not commit DB backups, workflow execution history, secrets, or runtime state.
- The Google OAuth credential (`wpcf2epDDCT57Y5x`) cannot refresh (`invalid_client`). Gmail workflows use IMAP fallback instead.
- The Docker health endpoint (`18809`) and reindex endpoint (`18810`) must bind to `0.0.0.0` (not `127.0.0.1`) so the n8n container can reach them via the Docker bridge gateway.
- The `obsidian` ChromaDB collection uses Ollama `nomic-embed-text` embeddings, while `personal` and `docs` use Sentence Transformers `all-MiniLM-L6-v2`. They cannot be compared directly by score across backends.
- Ollama embedding on CPU is ~1.2s per text. Full vault reindex takes ~5 minutes for 231 chunks. Incremental (no changes) takes ~1.4 seconds.
