diff --git a/.gitignore b/.gitignore index f548e48..b5f1380 100644 --- a/.gitignore +++ b/.gitignore @@ -22,3 +22,6 @@ openclaw/cron/runs/ # Runtime logs *.log + +# Local n8n SQLite recovery backups +.n8n-db-backups/ diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Workflows.md b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Workflows.md index 3d5effc..3a2c4c5 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Workflows.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Workflows.md @@ -3,7 +3,7 @@ title: n8n Workflows area: automation tags: [automation, integrations, n8n] created: 2026-03-18 -updated: 2026-03-27 +updated: 2026-05-12 status: active related: [[Infrastructure/Architecture]], [[Infrastructure/Automation/Cron Jobs]], [[Infrastructure/Services/Docker Services]], [[Infrastructure/Automation/n8n IMAP Triage Pipeline]], [[Infrastructure/Automation/n8n Nightly Vault Sync]] --- @@ -14,9 +14,11 @@ Running on `n8n-agent` at port `18808`. See [[Infrastructure/Services/Docker Ser ## Division of labor -- **n8n:** tasks where n8n has a native node or is acting as the controlled integration/scheduling layer -- **OpenClaw / OS cron:** shell-heavy local tasks, reminders, and agent-driven jobs that fit better outside n8n -- When in doubt: if n8n has the right native node or a clean HTTP/API path, prefer n8n. If it needs host shell/script glue, prefer OpenClaw cron or OS cron. +- **n8n:** schedules, webhooks, polling, retries, credentials, native integrations, and lightweight HTTP/API glue. +- **Atlas / Hermes:** reasoning-heavy decisions, debugging, code/config changes, workflow repair, and higher-quality synthesis. +- **Swarm local services:** cheap local LLM (`llama.cpp` on `18806`), embeddings (`Ollama` on `18807`), transcription (`Whisper` on `18811`), TTS (`Kokoro` on `18805`), and search (`SearXNG` on `18803`, Brave MCP on `18802`). +- **Obsidian:** durable human-readable notes and summaries. +- **Telegram / Discord:** notifications and quick approval loops. ## Core workflows @@ -25,13 +27,13 @@ Running on `n8n-agent` at port `18808`. See [[Infrastructure/Services/Docker Ser - **Status:** ✅ Active - **Trigger:** Schedule polling every 15 minutes - **Account:** `wills_portal` IMAP -- **Local LLM:** `gemma-3-12b-it-q4_0.gguf` +- **Local LLM:** `gemma-4-26B-A4B-it-UD-IQ2_M.gguf` via `llama.cpp` on `18806` - **Purpose:** reads unseen IMAP mail, drops obvious noise, judges ambiguous mail with the local LLM, sends signal digests to Telegram, and writes per-email notes to the shared Obsidian vault - **Details:** see [[Infrastructure/Automation/n8n IMAP Triage Pipeline]] ### Gmail Inbox Monitor + Obsidian Notes - **ID:** `whtdorf7yJMVYeHm` -- **Status:** ✅ Active +- **Status:** ⚠️ Active, but startup logs still report a Gmail Trigger warning (`undefined`) - **Trigger:** Gmail-triggered monitor flow - **Purpose:** watches Gmail, applies local triage, writes signal notes to Obsidian, and supports notification routing @@ -39,14 +41,14 @@ Running on `n8n-agent` at port `18808`. See [[Infrastructure/Services/Docker Ser - **ID:** `QRCCdHNXZUHc2Oz4` - **Status:** ✅ Active - **Trigger:** scheduled sync -- **Purpose:** fetches upcoming calendar events and writes/update notes in the shared vault +- **Purpose:** fetches upcoming calendar events and writes/updates notes in the shared vault - **Credential:** `Google Calendar account` ### Nightly Obsidian Vault Sync - **ID:** `75JCevkdgkyCr2qH` - **Status:** ✅ Active - **Trigger:** nightly cron (`45 23 * * *`) -- **Local LLM:** `gemma-3-12b-it-q4_0.gguf` +- **Local LLM:** `gemma-4-26B-A4B-it-UD-IQ2_M.gguf` via `llama.cpp` on `18806` - **Purpose:** fetches key operational notes plus live n8n/LLM health, uses the local LLM to generate a concise nightly vault sync note, and writes it to `Notes/YYYY-MM-DD Nightly Vault Sync.md` - **Details:** see [[Infrastructure/Automation/n8n Nightly Vault Sync]] @@ -62,30 +64,57 @@ Running on `n8n-agent` at port `18808`. See [[Infrastructure/Services/Docker Ser - **Type:** webhook - **Purpose:** reminder ingress path for simple reminder-style automation -## Reminder / utility workflows +## Near-term personal ops workflows -### Chase Sapphire Payment Reminder -- **ID:** `3OY2At6jP2WMALTp` +### Swarm Health Watchdog +- **ID:** `lDKocSFXBQWQrDd3` - **Status:** ✅ Active -- **Purpose:** scheduled credit-card payment reminder +- **Trigger:** every 15 minutes +- **Purpose:** checks the core swarm endpoints from inside the n8n container and sends Telegram + Discord alerts after two consecutive failures, plus recovery messages. +- **Checks:** Brave MCP `18802`, SearXNG `18803`, LiteLLM `18804`, Kokoro `18805`, llama.cpp `18806`, Ollama embeddings `18807`, n8n `18808`, Whisper `18811`. +- **Implementation note:** uses the Docker bridge gateway `172.19.0.1` for host-published services and `127.0.0.1:5678` for n8n self-health. +- **Limitation:** Docker health state is not yet available inside n8n because the Docker socket is not mounted; alerts include endpoint status and suggested host-side checks instead. -### Amex Payment Reminder -- **ID:** `rJeUsRbMSmI1PRHf` +### n8n Failure Digest +- **ID:** `G9ylNbHbnJ6fWX2C` - **Status:** ✅ Active -- **Purpose:** scheduled credit-card payment reminder +- **Trigger:** hourly plus manual trigger +- **Purpose:** queries recent failed n8n executions, groups failures by workflow/node/error, and sends a concise Telegram digest. -### Sink Leak Fix Reminder -- **ID:** `M0U2Ag0XRUec5ASJ` +### Voice Memo Capture (Audio URL + Local Whisper) +- **ID:** `El1BHJZ56JlzhrRZ` - **Status:** ✅ Active -- **Purpose:** reminder workflow for the sink leak follow-up +- **Type:** webhook +- **Purpose:** accepts an audio URL, downloads the audio, transcribes with local Whisper on `18811`, summarizes with local llama.cpp, writes transcript/summary/action items to Obsidian, and sends a Telegram notification. + +### Web-to-Notes Capture (Local LLM + Obsidian) +- **ID:** `GSmzuA5dgGgyRg5v` +- **Status:** ✅ Active +- **Type:** webhook +- **Purpose:** accepts a URL, fetches the page, extracts readable text, summarizes with local llama.cpp, and writes a clean markdown note to Obsidian. + +### Daily OpenClaw Session Digest +- **ID:** `qqYwAD05AvRHrHPc` +- **Status:** ✅ Active +- **Trigger:** nightly cron (`15 23 * * *`, 11:15 PM Pacific) +- **Local LLM:** local llama.cpp path +- **Purpose:** pulls recent OpenClaw sessions via gateway RPC, curates transcripts with the local LLM into a structured daily note (Summary, Decisions, Tasks, Follow-ups, Durable Facts), and writes it to `Notes/YYYY-MM-DD OpenClaw Session Digest.md` in the shared vault. +- **Fallback:** if the local LLM fails, writes a metadata-only note instead of silently dropping. + +## Archived / recovery notes ### IMAP Inbox Triage + Obsidian Notes (squareffect) +- **ID:** `xjUoQf97TkBrawc8` +- **Status:** 🗄️ Archived on 2026-05-12 after SQLite recovery because workflow JSON was empty/unrecoverable. + +### IMAP Inbox Triage + Obsidian Notes (wills-portal duplicate) - **ID:** `kHDK9QdUSiAJ8rCM` -- **Status:** ✅ Active -- **Purpose:** second IMAP triage flow for the squareffect mailbox +- **Status:** 🗄️ Archived on 2026-05-12 after SQLite recovery because workflow JSON was empty/unrecoverable. ## Operational notes -- Local n8n API and workflow management were re-verified live on 2026-03-27. -- Local LLM inference path was re-verified live on 2026-03-27 via `http://192.168.153.113:18806/v1`. -- Obsidian REST writes were re-verified on 2026-03-27 after fixing shared-vault cross-user permissions. +- n8n DB was recovered on 2026-05-12 from a `SQLITE_CORRUPT: database disk image is malformed` crash loop using SQLite `.recover`, then repaired until `n8n export:workflow --all` succeeded. +- Host-side DB backups from the repair were stored under `/home/will/lab/swarm/.n8n-db-backups/` and should remain untracked. +- `n8n export:workflow --all` was re-verified on 2026-05-12 and exported 13 workflows successfully. +- n8n health after recovery: `http://127.0.0.1:18808/healthz` returns `{"status":"ok"}` and the container is healthy. +- Startup still logs the known Gmail Trigger warning (`undefined`) for `Gmail Inbox Monitor + Obsidian Notes`; this is the next cleanup target.