From 2e751d6ff5f467713dff00dac08c6223c51ca3b4 Mon Sep 17 00:00:00 2001 From: William Valentin Date: Wed, 27 May 2026 14:31:39 -0700 Subject: [PATCH] chore(vault): capture generated Obsidian updates --- .../2026-05-23 - Hermes AI Brief.md | 26 +++++++ .../2026-05-24 - Hermes AI Brief.md | 27 +++++++ .../2026-05-25 - Hermes AI Brief.md | 24 +++++++ .../2026-05-26 - Hermes AI Brief.md | 28 ++++++++ ...-25-codex-hermes-proxy-experiment-audit.md | 56 +++++++++++++++ ...2026-05-27 Obsidian Data Accuracy Audit.md | 70 +++++++++++++++++++ .../Daily/Reviews/2026-05-23 Daily Review.md | 34 +++++++++ .../Daily/Reviews/2026-05-24 Daily Review.md | 34 +++++++++ .../Daily/Reviews/2026-05-25 Daily Review.md | 34 +++++++++ .../Daily/Reviews/2026-05-26 Daily Review.md | 34 +++++++++ .../Daily/Reviews/2026-05-27 Daily Review.md | 34 +++++++++ .../Decisions/Runbook Suggestions.md | 4 +- .../Inbox/Triage/2026-05-24.md | 30 ++++++++ .../Inbox/Triage/2026-05-25.md | 30 ++++++++ .../Inbox/Triage/2026-05-26.md | 30 ++++++++ .../Inbox/Triage/2026-05-27.md | 30 ++++++++ .../Infrastructure/Architecture.md | 49 ++++++------- .../Infrastructure/Automation/Cron Jobs.md | 28 ++++---- .../Automation/n8n Nightly Vault Sync.md | 10 +-- .../Services/Docker Services.md | 20 +++--- .../Resources/Obsidian Automation Health.md | 6 +- .../Resources/Service Catalog.md | 17 +++-- 22 files changed, 584 insertions(+), 71 deletions(-) create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-23 - Hermes AI Brief.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-24 - Hermes AI Brief.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-25 - Hermes AI Brief.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-26 - Hermes AI Brief.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-25-codex-hermes-proxy-experiment-audit.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-27 Obsidian Data Accuracy Audit.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-23 Daily Review.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-24 Daily Review.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-25 Daily Review.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-26 Daily Review.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-27 Daily Review.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-24.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-25.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-26.md create mode 100644 swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-27.md diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-23 - Hermes AI Brief.md b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-23 - Hermes AI Brief.md new file mode 100644 index 0000000..a369cfd --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-23 - Hermes AI Brief.md @@ -0,0 +1,26 @@ +# Daily Hermes + AI Research Brief — 2026-05-23 + +## Important updates + +- **Hermes v0.14.0 is the big local-agent upgrade to test next.** The latest GitHub release (May 16) says Hermes added SuperGrok OAuth with 1M-token Grok 4.3 context, an OpenAI-compatible local proxy for OAuth-authed providers, first-class `x_search`, Microsoft Teams plumbing, lighter/lazy installs, PyPI install support, ~19s faster cold start, 180x faster browser CDP calls, LINE/SimpleX gateways, `/handoff`, LSP diagnostics on writes, improved `computer_use`, Windows beta, OpenRouter Pareto Code router, and trusted Hugging Face skills taps. This directly affects Atlas provider routing, gateway reach, browser automation, and local coding-agent ergonomics. Source: [Hermes releases](https://github.com/NousResearch/hermes-agent/releases). + +- **Hermes v0.13.0’s durable Kanban board is still the most relevant architectural feature for Atlas/swarm work.** The release notes describe Kanban as a durable multi-agent board with heartbeat, reclaim, zombie detection, auto-block, dependency DAGs, dispatch daemon, and worker-gated tools. That maps closely to Will’s local swarm / Pi agent harness needs: durable task state beats fragile background chats. Source: [Hermes v0.13 release](https://github.com/NousResearch/hermes-agent/blob/main/RELEASE_v0.13.0.md). + +- **OpenAI Codex is moving toward long-running, remote, cross-device agents.** The May 21 Codex changelog says Goal mode is now available across the app, IDE extension, and CLI for objectives that can run “hours or even days”; Codex also added Appshots, remote computer use after a Mac locks, plugin marketplace sharing with skills/MCP/hooks bundles, and browser-use reliability/structured extraction improvements. Useful pattern for Hermes/Atlas: explicit goal mode + remote host continuity + shareable tool bundles. Source: [Codex changelog](https://developers.openai.com/codex/changelog). + +- **Codex now supports trusted non-interactive automation tokens.** OpenAI’s May 5 changelog entry says Enterprise admins can allow members to create Codex access tokens for scripts, schedulers, and private CI runners. Even if Will does not use Codex Enterprise, this is a strong signal: scheduled/CI agent auth is becoming a first-class surface, and Hermes cron/gateway auth should keep separating human OAuth, job identity, and least-privilege automation tokens. Source: [Codex changelog](https://developers.openai.com/codex/changelog). + +- **Agent traffic gateways are converging around LLM + MCP + A2A + Kubernetes in one data plane.** Agentgateway positions itself as a Linux Foundation project for “LLM, MCP, A2A, and HTTP in one data plane,” with integrations for Claude Code, Codex, OpenCode, Hermes, MCP servers, OpenTelemetry, Kubernetes, and major providers. For CoreWeave-style GPU/k8s work, watch this class of gateway as the possible control plane between local agents, hosted inference, MCP tools, and observability. Source: [agentgateway](https://agentgateway.dev/). + +## Actionable ideas for us + +- **[quick] Upgrade-check Hermes locally** with `hermes update` / `hermes doctor`, then specifically smoke-test v0.14 features that matter: PyPI/lazy deps, browser CDP speed, LSP diagnostics, `/handoff`, and the OpenRouter Pareto Code router. +- **[experiment] Prototype a small Hermes Kanban board** for Atlas jobs: one dispatcher task, two worker profiles, heartbeat/reclaim enabled, and Obsidian note output as the completion artifact. +- **[experiment] Test Hermes’ OpenAI-compatible local proxy** as a unifying endpoint for Codex/Aider/Cline/Continue against existing OAuth-backed providers; compare latency, context caching, and failure modes. +- **[watch] Track agentgateway’s MCP/A2A routing model** for whether it can front local MCP servers and GPU-hosted inference without overcomplicating Atlas’ current setup. + +## Worth ignoring + +- Generic “Claude Code vs Codex” comparison posts unless they include reproducible benchmarks, config details, or concrete failure cases. +- Broad “top LLMOps tools” listicles; most are vendor SEO unless they show real routing, eval, cost, or Kubernetes deployment examples. +- Consumer-only agent demos that do not expose tool permissions, auth, logs, or recoverability. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-24 - Hermes AI Brief.md b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-24 - Hermes AI Brief.md new file mode 100644 index 0000000..3b9cc34 --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-24 - Hermes AI Brief.md @@ -0,0 +1,27 @@ +# Daily Hermes + AI Research Brief — 2026-05-24 + +## Important updates + +- **Hermes Agent v0.14.0 shipped May 16 as the “Foundation Release.”** Highlights: PyPI install path, ~19s faster cold start, 180x faster Browser CDP calls, native Windows beta, OpenAI-compatible local proxy for OAuth-authed providers, xAI/SuperGrok OAuth with 1M-context Grok, LINE/SimpleX/Teams platform work, `/handoff`, LSP diagnostics on writes, and new optional skills. This matters for Atlas because it reduces install friction and opens a useful “Hermes as local provider proxy” pattern for Codex/Aider/Cline/Continue experiments. Source: [GitHub releases](https://github.com/NousResearch/hermes-agent/releases). + +- **Hermes v0.13.0’s durable Kanban/multi-agent board remains the most relevant recent Hermes architecture change.** The release notes describe durable heartbeats, reclaim/zombie detection, auto-blocking, task lifecycle, and worker/orchestrator patterns. This is directly applicable to Will’s Atlas/local-swarm work: use the board as the durable coordination boundary instead of ad-hoc background sessions. Source: [Hermes releases](https://github.com/NousResearch/hermes-agent/releases). + +- **LiteLLM opened an alpha Managed Agents Platform on May 8.** It is a self-hosted platform for running multiple agents in production, focused on per-team/context sandboxes and session persistence across pod restarts/upgrades. Worth watching as a comparison point for Hermes Kanban + profiles + gateway on Kubernetes. Source: [LiteLLM blog](https://docs.litellm.ai/blog/agent-platform-alpha). + +- **vLLM’s latest developer docs now document experimental disaggregated prefilling.** The page describes splitting prefill and decode across separate vLLM instances and transferring KV cache via connectors such as NIXL, OffloadingConnector, and FlexKV; it was updated May 6. This is practical for CoreWeave-style GPU/k8s serving experiments where prefill/decode bottlenecks need separate scaling. Source: [vLLM disaggregated prefilling docs](https://docs.vllm.ai/en/latest/features/disagg_prefill/). + +- **MCP governance/security capacity is expanding.** The MCP project added Clare Liguori as Core Maintainer and Den Delimarsky as Lead Maintainer; the post emphasizes production agent runtimes, triggers/events, authorization, RFC 8707 resource indicators, and security work. This matters because MCP is becoming long-lived infra, not just a tool-call fad; Hermes MCP integration should track auth/security spec changes closely. Source: [MCP blog](https://blog.modelcontextprotocol.io/posts/2026-04-08-expanding-maintainer-team/). + +## Actionable ideas for us + +- **[quick]** Test `pip install hermes-agent` in a clean disposable venv/container and note any Atlas bootstrap simplifications vs the current install path. +- **[experiment]** Prototype the new Hermes OpenAI-compatible local proxy with one external coding client (Aider/Cline/Codex) and one OAuth provider; record latency, auth refresh behavior, and whether prompt caching survives the proxy path. +- **[experiment]** Map Hermes Kanban concepts to the Pi/Atlas swarm: task table = durable queue, profile = worker role, heartbeat/reclaim = supervisor; avoid inventing a parallel scheduler until this is evaluated. +- **[watch]** Track vLLM disaggregated prefill + LiteLLM componentized/agent platform work for a future GPU/k8s inference control plane design. + +## Worth ignoring + +- Generic “agent framework star race” and listicle posts unless they include reproducible architecture details or benchmarks. +- Consumer-only “AI assistant” announcements without tool protocol, deployment, privacy, or local-first implications. + +Saved to Obsidian: Atlas/Daily Research/2026-05-24 - Hermes AI Brief.md diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-25 - Hermes AI Brief.md b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-25 - Hermes AI Brief.md new file mode 100644 index 0000000..56602cf --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-25 - Hermes AI Brief.md @@ -0,0 +1,24 @@ +# Daily Hermes + AI Research Brief — 2026-05-25 + +## Important updates + +- **Hermes v0.14.0 is the big local-agent upgrade to study.** The May 16 release claims PyPI install support, ~19s faster cold start, 180× faster browser CDP calls, native Windows beta, LINE/SimpleX support, LSP diagnostics on write, prompt caching, `/handoff`, `computer_use`, and a local OpenAI-compatible proxy for OAuth-authed providers. This directly maps to Atlas goals: faster cron/gateway loops, cross-tool provider routing, and better spawned-agent ergonomics. [Source: GitHub releases](https://github.com/NousResearch/hermes-agent/releases) +- **Hermes provider/tooling surface is expanding beyond “chat agent” into router/proxy glue.** The release notes highlight xAI SuperGrok OAuth, Grok 4.3 1M context, `x_search`, Microsoft Teams plumbing, Zed ACP registry integration via `uvx`, OpenRouter Pareto Code router, and `huggingface/skills` as a trusted tap. For Will, this is a reason to treat Hermes/Atlas as a control plane for heterogeneous model/tool backends rather than a single CLI. [Source: Hermes v0.14.0 release](https://github.com/NousResearch/hermes-agent/releases) +- **OpenAI is pushing Codex toward durable, cross-app agent work.** Codex now has background computer use, an in-app browser, image generation, 90+ plugins including MCP servers, multiple terminals, SSH devbox support, reusable automation threads, scheduling, and preview memory. This is highly relevant because it validates the same design space Atlas is using: cron + memory + tools + multi-agent task carryover. [Source: OpenAI](https://openai.com/index/codex-for-almost-everything/) +- **Agent governance is becoming an infra layer, not an afterthought.** Databricks’ Unity AI Gateway release adds MCP governance, on-behalf-of permissions, LLM/MCP observability, cost attribution, model fallback/rate-limit/guardrail policies, and consistent controls across OpenAI/Anthropic/Google/open-source models. Practical takeaway: Atlas/Hermes should log model+tool calls as first-class audit events, especially for Pi swarm and GPU/k8s agents. [Source: Databricks](https://www.databricks.com/blog/ai-gateway-governance-layer-agentic-ai) +- **MCP is continuing to standardize enterprise tool use.** Red Hat’s MCP guidance frames MCP as the layer for tool discovery, policy, and auditable actions, moving beyond one-off RAG/tool-call glue. This supports investing in MCP-compatible wrappers for local services instead of bespoke integrations. [Source: Red Hat Developer](https://developers.redhat.com/articles/2026/01/08/building-effective-ai-agents-mcp) +- **vLLM v0.21.0 has inference changes worth tracking for local/GPU backends.** The release includes KV offload + Hybrid Memory Allocator work, speculative decoding that respects reasoning/thinking budgets, new Blackwell attention backend work for DeepSeek/Kimi-style models, more model/tool-parser support, and a breaking C++20 build requirement. This matters for any CoreWeave-style or home GPU inference path. [Source: vLLM releases](https://github.com/vllm-project/vllm/releases) + +## Actionable ideas for us + +- **[quick]** Check whether the running Hermes install is already on v0.14.0; if not, schedule a manual upgrade window and verify cron/gateway after restart. +- **[quick]** Add a short “agent audit event” note/template for Atlas: model, provider, tool/MCP server, data touched, outcome, and cost/latency where available. +- **[experiment]** Prototype Hermes’ local OpenAI-compatible proxy with one external coding tool (Codex/Aider/Cline/Continue) to see if Atlas can centralize provider auth and routing. +- **[watch]** Track vLLM v0.21.x adoption notes before upgrading any production-ish inference service because C++20/Transformers v5 changes may break builds. + +## Worth ignoring + +- Generic “2026 agent landscape” listicles unless they include reproducible implementation details or protocol/tooling changes. +- Consumer-only Codex/Claude comparisons without concrete workflow, eval, or infra takeaways. + +Saved to Obsidian: Atlas/Daily Research/2026-05-25 - Hermes AI Brief.md diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-26 - Hermes AI Brief.md b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-26 - Hermes AI Brief.md new file mode 100644 index 0000000..75cddaa --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Daily Research/2026-05-26 - Hermes AI Brief.md @@ -0,0 +1,28 @@ +# Daily Hermes + AI Research Brief — 2026-05-26 + +## Important updates + +- **Hermes Agent v0.14.0 is the big local-agent update to study.** The May 16 release notes say PyPI install now works, cold start is ~19s faster, Browser CDP calls are 180× faster, native Windows is in beta, LINE/SimpleX were added, `x_search` landed, an OpenAI-compatible local proxy can expose OAuth-backed providers to Codex/Aider/Cline/Continue, and `/handoff`, prompt caching, LSP diagnostics, video generation, and provider-agnostic computer-use improved. This is directly relevant to Atlas as a multi-channel, local-first, tool-heavy agent stack. Source: [NousResearch/hermes-agent releases](https://github.com/NousResearch/hermes-agent/releases). + +- **Hermes' May 7 “Tenacity” release introduced durable multi-agent coordination primitives.** The release search result reports Kanban as a durable multi-agent board with heartbeat, reclaim, zombie detection, and auto-block behavior. That maps closely to Will's swarm/worker orchestration needs: durable task state beats pure chat-thread delegation for long-running local agents. Source: [Hermes Agent v2026.5.7 release](https://github.com/NousResearch/hermes-agent/releases) / [NewReleases mirror](https://newreleases.io/project/github/NousResearch/hermes-agent/release/v2026.5.7). + +- **GitHub Copilot is pushing hard on cloud-agent ergonomics and routing.** The Copilot changelog lists May updates for auto model selection in VS Code, semantic issue search, applying code-review feedback with Copilot cloud agent, one-click fixes for failing Actions, cheaper/simple-task models, REST API auditing of cloud-agent config, and remote control for Copilot CLI sessions. Useful pattern for Atlas: expose agent state, config, and fix loops as first-class APIs/UI actions. Source: [GitHub Copilot changelog](https://github.blog/changelog/label/copilot/). + +- **OpenAI's Ramp case study is a concrete benchmark for agent usefulness: code review + on-call assistant.** Ramp says Codex with GPT-5.5 gives substantive PR feedback “in minutes instead of hours” and is being used to build an internal on-call assistant for complex incident workflows. Practical takeaway: measure Atlas/Hermes value around latency-to-first-review, incident context retention, and whether agents reduce interruption cost—not just task completion demos. Source: [OpenAI: Ramp engineers accelerate code review with Codex](https://openai.com/index/ramp/). + +- **Microsoft Agent Framework is converging on the same building blocks Hermes already emphasizes.** Microsoft documents agents that call tools and MCP servers, workflows with type-safe routing/checkpointing/human-in-loop, model clients, agent sessions, memory providers, middleware, and MCP clients; supported backends include Foundry, Anthropic, Azure OpenAI, OpenAI, Ollama, and more. This is worth watching for interoperability ideas and enterprise patterns around middleware/checkpointing. Source: [Microsoft Agent Framework overview](https://learn.microsoft.com/en-us/agent-framework/overview/). + +- **Anthropic's agent platform direction continues to validate MCP + generated client surfaces.** Anthropic's May 18 announcement says it acquired Stainless; the search snippet frames the move around MCP and agent connectivity. Even if product details are thin from the public page, the signal is clear: typed API/client generation plus MCP-style tool connectivity is becoming core agent infrastructure. Source: [Anthropic acquires Stainless](https://www.anthropic.com/news/anthropic-acquires-stainless). + +## Actionable ideas for us + +- **[quick] Upgrade-check Hermes and read v0.14 notes against Atlas pain points.** Prioritize PyPI install, CDP speedups, `/handoff`, local proxy, prompt caching, LSP diagnostics, and `x_search`. +- **[experiment] Prototype a Hermes Kanban-backed worker lane** for one recurring Atlas task: create → claim → heartbeat → complete/block. Compare with current cron/session-only flow. +- **[experiment] Test the OpenAI-compatible local proxy** as a unifying endpoint for Aider/Cline/Continue/Codex-style tools using existing Hermes auth/provider routing. +- **[watch] Track Copilot's cloud-agent API/config surface** for ideas: REST-auditable agent config, remote CLI control, semantic issue search, and auto model routing are all patterns Atlas could mirror locally. + +## Worth ignoring + +- Generic “best AI coding agents in 2026” listicles unless they include reproducible benchmarks or specific workflow design ideas. +- Funding/enterprise-adoption announcements without concrete APIs, open-source code, or operational lessons. +- Consumer AI stories unless they improve local/private agent workflows or tool orchestration. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-25-codex-hermes-proxy-experiment-audit.md b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-25-codex-hermes-proxy-experiment-audit.md new file mode 100644 index 0000000..995e99e --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-25-codex-hermes-proxy-experiment-audit.md @@ -0,0 +1,56 @@ +--- +type: agent-audit-event +project: "Atlas Capability Upgrade Program" +status: accepted +owner: "Atlas" +created: 2026-05-25 +updated: 2026-05-25 +source: "Telegram request: prototype Hermes local OpenAI-compatible proxy with one external coding tool" +tags: [atlas, audit, agent-event, hermes, proxy, codex] +--- + +# Agent Audit Event — Codex + Hermes local proxy experiment + +## Summary + +- **When:** 2026-05-25 +- **Actor:** Atlas / Hermes default Telegram session +- **Trigger:** User request from daily Hermes + AI research brief follow-up +- **Outcome:** partial +- **One-line result:** Codex can be pointed at a local OpenAI-compatible base URL, but the current Hermes proxy is not ready for a complete Codex-through-Hermes run without a logged-in proxy upstream and/or endpoint compatibility work. + +## Runtime + +- **Model:** gpt-5.5 +- **Provider:** OpenAI Codex backend for Hermes session +- **Profile/session:** default Hermes Telegram session +- **Tools used:** terminal, file/search/read/write, process +- **MCP servers / external services:** Hermes local proxy CLI; Codex CLI 0.129.0; local loopback capture server + +## Data touched + +- **Local paths:** `/home/will/.hermes/hermes_cli/proxy/*`, `/home/will/.codex/config.toml`, temporary files under `/tmp` +- **Remote systems:** none for the prototype request path; no real upstream completion was performed +- **Secrets/PII exposure:** Codex used its own local auth while probing a local capture endpoint; do not use raw request capture servers for future authenticated probes unless auth is disabled with a temporary `CODEX_HOME`. +- **Writes/side effects:** created and removed/terminated a temporary local HTTP capture process; wrote this audit note + +## Cost and latency + +- **Approx. cost:** no successful model completion through the proxy; cost unknown / likely none for the failed local capture run +- **Latency / runtime:** quick local smoke, under a few minutes +- **Notable performance notes:** Codex retried `/v1/responses` several times after websocket upgrade failures before falling back to POST. + +## Evidence + +- **Hermes proxy status:** `hermes proxy status` showed `nous` and `xai` adapters available but not logged in on this profile. +- **Hermes proxy allowed paths:** + - Nous: `/chat/completions`, `/completions`, `/embeddings`, `/models` + - xAI: `/chat/completions`, `/completions`, `/embeddings`, `/models`, `/responses` +- **Codex endpoint behavior:** with `openai_base_url="http://127.0.0.1:8765/v1"`, Codex 0.129.0 probed `GET /v1/models?client_version=0.129.0`, then attempted websocket `GET /v1/responses`, then `POST /v1/responses`. +- **Compatibility finding:** + - Codex is not a good first full-success target for the Nous proxy path because Codex uses Responses API, while the Nous proxy adapter only forwards chat/completions-style paths. + - Codex may be plausible through the xAI proxy adapter because xAI allows `/responses`, but the adapter was not logged in and websocket behavior still needs verification. + - Aider is likely the better first full-success target for Nous because it uses OpenAI-compatible chat completions, but it is not installed on this machine. +- **Follow-up:** For a clean full prototype, either: + 1. authenticate `hermes proxy --provider xai` and retry Codex with a temporary `CODEX_HOME`, or + 2. install/use Aider against `hermes proxy --provider nous` once Nous Portal proxy auth is available. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-27 Obsidian Data Accuracy Audit.md b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-27 Obsidian Data Accuracy Audit.md new file mode 100644 index 0000000..4e5f4f2 --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Atlas/Reports/2026-05-27 Obsidian Data Accuracy Audit.md @@ -0,0 +1,70 @@ +--- +title: Obsidian Data Accuracy Audit +area: atlas-reports +tags: [obsidian, data-quality, infrastructure, audit] +created: 2026-05-27 +status: completed +--- + +# Obsidian Data Accuracy Audit + +## Scope + +Pass over the shared Obsidian vault for stale infrastructure/model claims, prompted by the wrong local LLM model in [[Infrastructure/Architecture]]. + +Vault path checked: + +`/home/will/lab/swarm/swarm-common/obsidian-vault/will/will-shared-zap` + +## Live evidence checked on 2026-05-27 + +- Local llama.cpp service: `llama-server.service` active on `18806`. +- `/v1/models` on `http://127.0.0.1:18806` returned current model id: + - `gemma-4-26B-A4B-it-UD-IQ2_M.gguf` +- Ollama on `18807` includes `nomic-embed-text:latest` for embeddings and older chat models; it is not the direct local LLM endpoint used by llama.cpp. +- Hermes default model config: `openai-codex` / `gpt-5.5`. +- Current host IPs include `192.168.153.130`, `192.168.153.141`, and Tailscale `100.123.88.127`. +- Vault path is on host btrfs (`/home`), not currently a VM `virtiofs` mount. +- n8n on `18808` returned HTTP 200. +- MinIO health at `192.168.153.253:9000` returned HTTP 200. +- Hermes cron job `atlas-minio-self-backup` is enabled, scheduled daily at 02:50, and last status was `ok`. + +## Updated during this pass + +- [[Infrastructure/Architecture]] + - Corrected local LLM from Gemma 3 to `gemma-4-26B-A4B-it-UD-IQ2_M.gguf`. + - Corrected Hermes model from `gpt-5.4` to `gpt-5.5` via `openai-codex`. + - Marked OpenClaw VM / `virtiofs` details as historical. + - Corrected current host IPs, MinIO bucket purpose, Gitea repo references, and local model verification command. +- [[Infrastructure/Automation/Cron Jobs]] + - Marked the old OpenClaw cron inventory as stale/historical. + - Added current Hermes cron/MinIO backup details from live `cronjob list`. +- [[Resources/Service Catalog]] + - Refreshed high-risk local service rows: SearXNG, LiteLLM, Brave MCP, whisper containers, direct local LLM model id. + +## Previously blocked edits completed + +Will repaired the permissions on the previously blocked files. On the follow-up pass, ownership was verified as `will:will` and the stale active-note claims were corrected in: + +- [[Infrastructure/Services/Docker Services]] + - Updated old LAN host references from `192.168.153.113` / `.117` to current local/LAN endpoints. + - Updated llama.cpp model from `gemma-3-12b-it-q4_0.gguf` to `gemma-4-26B-A4B-it-UD-IQ2_M.gguf`. + - Updated MinIO backup wording from current OpenClaw `zap` backups to current Atlas/Hermes `atlas` backups with legacy `zap` noted. +- [[Infrastructure/Automation/n8n IMAP Triage Pipeline]] + - Updated model and endpoint to the current Gemma 4 / `18806` endpoint. + - Marked old `claw`/`openclaw` VM ownership checks as historical. +- [[Infrastructure/Automation/n8n Nightly Vault Sync]] + - Updated local LLM and health URLs to current local endpoints. + +## Historical snapshot notes + +Many `Notes/YYYY-MM-DD Nightly Vault Sync.md` files contain the old Gemma 3 model id and old VM/virtiofs/IP state. I did **not** rewrite those because they are dated snapshots. They should be read as historical breadcrumbs, not canonical current-state docs. + +## Search patterns used + +Searched current vault markdown for local-model and infrastructure drift terms including: + +- `gemma-3-12b`, `gemma 3`, `gpt-5.4` +- `192.168.153.113`, `192.168.153.117` +- `~/.openclaw`, `openclaw.json`, `bucket zap` +- local service ports `18801`–`18814`, `27123` diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-23 Daily Review.md b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-23 Daily Review.md new file mode 100644 index 0000000..94ca34b --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-23 Daily Review.md @@ -0,0 +1,34 @@ +--- +type: daily-review +date: 2026-05-23 +tags: [type/daily-review, automation/n8n] +--- + +# Daily Review 2026-05-23 + +## Top priorities + +- [ ] +- [ ] +- [ ] + +## Inbox sweep + +- [ ] Review [[Inbox]] +- [ ] Promote useful captures into [[Projects Home]], [[Resources Home]], [[Decisions Home]], or [[Runbooks Home]] + +## Open loops + +- [ ] Check [[Projects Home]] +- [ ] Check [[Meetings Home]] action items +- [ ] Check [[Runbooks Home]] for procedures that need updates + +## Notes / log + +- + +## End-of-day reflection + +- What moved forward? +- What is blocked? +- What should start tomorrow? diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-24 Daily Review.md b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-24 Daily Review.md new file mode 100644 index 0000000..e104062 --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-24 Daily Review.md @@ -0,0 +1,34 @@ +--- +type: daily-review +date: 2026-05-24 +tags: [type/daily-review, automation/n8n] +--- + +# Daily Review 2026-05-24 + +## Top priorities + +- [ ] +- [ ] +- [ ] + +## Inbox sweep + +- [ ] Review [[Inbox]] +- [ ] Promote useful captures into [[Projects Home]], [[Resources Home]], [[Decisions Home]], or [[Runbooks Home]] + +## Open loops + +- [ ] Check [[Projects Home]] +- [ ] Check [[Meetings Home]] action items +- [ ] Check [[Runbooks Home]] for procedures that need updates + +## Notes / log + +- + +## End-of-day reflection + +- What moved forward? +- What is blocked? +- What should start tomorrow? diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-25 Daily Review.md b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-25 Daily Review.md new file mode 100644 index 0000000..42ee900 --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-25 Daily Review.md @@ -0,0 +1,34 @@ +--- +type: daily-review +date: 2026-05-25 +tags: [type/daily-review, automation/n8n] +--- + +# Daily Review 2026-05-25 + +## Top priorities + +- [ ] +- [ ] +- [ ] + +## Inbox sweep + +- [ ] Review [[Inbox]] +- [ ] Promote useful captures into [[Projects Home]], [[Resources Home]], [[Decisions Home]], or [[Runbooks Home]] + +## Open loops + +- [ ] Check [[Projects Home]] +- [ ] Check [[Meetings Home]] action items +- [ ] Check [[Runbooks Home]] for procedures that need updates + +## Notes / log + +- + +## End-of-day reflection + +- What moved forward? +- What is blocked? +- What should start tomorrow? diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-26 Daily Review.md b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-26 Daily Review.md new file mode 100644 index 0000000..42606b7 --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-26 Daily Review.md @@ -0,0 +1,34 @@ +--- +type: daily-review +date: 2026-05-26 +tags: [type/daily-review, automation/n8n] +--- + +# Daily Review 2026-05-26 + +## Top priorities + +- [ ] +- [ ] +- [ ] + +## Inbox sweep + +- [ ] Review [[Inbox]] +- [ ] Promote useful captures into [[Projects Home]], [[Resources Home]], [[Decisions Home]], or [[Runbooks Home]] + +## Open loops + +- [ ] Check [[Projects Home]] +- [ ] Check [[Meetings Home]] action items +- [ ] Check [[Runbooks Home]] for procedures that need updates + +## Notes / log + +- + +## End-of-day reflection + +- What moved forward? +- What is blocked? +- What should start tomorrow? diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-27 Daily Review.md b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-27 Daily Review.md new file mode 100644 index 0000000..073b191 --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Daily/Reviews/2026-05-27 Daily Review.md @@ -0,0 +1,34 @@ +--- +type: daily-review +date: 2026-05-27 +tags: [type/daily-review, automation/n8n] +--- + +# Daily Review 2026-05-27 + +## Top priorities + +- [ ] +- [ ] +- [ ] + +## Inbox sweep + +- [ ] Review [[Inbox]] +- [ ] Promote useful captures into [[Projects Home]], [[Resources Home]], [[Decisions Home]], or [[Runbooks Home]] + +## Open loops + +- [ ] Check [[Projects Home]] +- [ ] Check [[Meetings Home]] action items +- [ ] Check [[Runbooks Home]] for procedures that need updates + +## Notes / log + +- + +## End-of-day reflection + +- What moved forward? +- What is blocked? +- What should start tomorrow? diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Decisions/Runbook Suggestions.md b/swarm-common/obsidian-vault/will/will-shared-zap/Decisions/Runbook Suggestions.md index 7ac5934..f7a4471 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Decisions/Runbook Suggestions.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Decisions/Runbook Suggestions.md @@ -1,6 +1,6 @@ -# Decision / Runbook Suggestions 2026-05-18 +# Decision / Runbook Suggestions 2026-05-25 -Generated: 2026-05-18T15:29:06.051Z +Generated: 2026-05-25T15:00:25.654Z Review candidates and promote useful items into durable Decision or Runbook notes. This note is overwritten weekly. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-24.md b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-24.md new file mode 100644 index 0000000..cfd3a5b --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-24.md @@ -0,0 +1,30 @@ +# Inbox Triage 2026-05-24 + +Generated: 2026-05-24T01:00:00.048Z + +## Inbox items + +- [ ] [[Chat Summaries]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox Home]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Weekly Review]] — classify as Project / Resource / Decision / Runbook / Archive + +## Promote to Projects + +- [ ] + +## Promote to Resources + +- [ ] + +## Promote to Decisions + +- [ ] + +## Promote to Runbooks + +- [ ] + +## Archive / Delete / Defer + +- [ ] diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-25.md b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-25.md new file mode 100644 index 0000000..6447c8e --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-25.md @@ -0,0 +1,30 @@ +# Inbox Triage 2026-05-25 + +Generated: 2026-05-25T01:00:12.199Z + +## Inbox items + +- [ ] [[Chat Summaries]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox Home]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Weekly Review]] — classify as Project / Resource / Decision / Runbook / Archive + +## Promote to Projects + +- [ ] + +## Promote to Resources + +- [ ] + +## Promote to Decisions + +- [ ] + +## Promote to Runbooks + +- [ ] + +## Archive / Delete / Defer + +- [ ] diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-26.md b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-26.md new file mode 100644 index 0000000..96776cb --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-26.md @@ -0,0 +1,30 @@ +# Inbox Triage 2026-05-26 + +Generated: 2026-05-26T01:00:00.134Z + +## Inbox items + +- [ ] [[Chat Summaries]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox Home]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Weekly Review]] — classify as Project / Resource / Decision / Runbook / Archive + +## Promote to Projects + +- [ ] + +## Promote to Resources + +- [ ] + +## Promote to Decisions + +- [ ] + +## Promote to Runbooks + +- [ ] + +## Archive / Delete / Defer + +- [ ] diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-27.md b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-27.md new file mode 100644 index 0000000..eaa7c37 --- /dev/null +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Inbox/Triage/2026-05-27.md @@ -0,0 +1,30 @@ +# Inbox Triage 2026-05-27 + +Generated: 2026-05-27T01:00:00.703Z + +## Inbox items + +- [ ] [[Chat Summaries]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox Home]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Inbox]] — classify as Project / Resource / Decision / Runbook / Archive +- [ ] [[Weekly Review]] — classify as Project / Resource / Decision / Runbook / Archive + +## Promote to Projects + +- [ ] + +## Promote to Resources + +- [ ] + +## Promote to Decisions + +- [ ] + +## Promote to Runbooks + +- [ ] + +## Archive / Delete / Defer + +- [ ] diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Architecture.md b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Architecture.md index fb05b2a..16a005c 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Architecture.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Architecture.md @@ -3,7 +3,7 @@ title: Architecture area: infrastructure tags: [infrastructure, homelab, assistant, integrations, automation, diagrams] created: 2026-03-18 -updated: 2026-05-19 +updated: 2026-05-27 status: active related: [[Infrastructure/Services/Docker Services]], [[Infrastructure/Automation/Cron Jobs]], [[Infrastructure/Automation/n8n Workflows]] --- @@ -12,7 +12,7 @@ related: [[Infrastructure/Services/Docker Services]], [[Infrastructure/Automatio High-level map of Will's homelab + assistant stack. For operational details (IPs, credential paths, commands), see TOOLS.md in the assistant workspace. -Operational note: the shared Obsidian vault lives on a `virtiofs` mount and is touched by both the host Obsidian user (`claw`) and the VM assistant user (`openclaw`); collaborative note folders need write permissions that work for both sides. +Operational note: current vault path is `/home/will/lab/swarm/swarm-common/obsidian-vault/will/will-shared-zap` on the host btrfs filesystem. Older notes about a VM `virtiofs` mount and `claw`/`openclaw` ownership are historical unless the VM is re-enabled. See sub-notes for details: - [[Infrastructure/Services/Docker Services]] @@ -21,16 +21,15 @@ See sub-notes for details: ## Current deployment -- The assistant currently runs in a VM on Will's laptop. -- Planned later move: main host. +- Atlas/Hermes currently runs from Will's host-side `~/.hermes` checkout on the laptop; older OpenClaw VM notes are historical. +- OpenClaw VM infrastructure is intentionally dormant unless Will re-enables it. - Shared note workspace lives in `will/will-shared-zap/` inside the Obsidian vault tree. ## Networking -- **Primary host LAN IP:** `192.168.153.130` -- **Secondary host LAN IP:** `192.168.153.140` (stale/unreachable from VM as of 2026-04-15) -- **Previous host LAN IPs:** `192.168.153.113`, `192.168.153.117` +- **Primary host LAN IPs:** `192.168.153.130`, `192.168.153.141` - **Tailscale IP:** `100.123.88.127` +- **Previous host LAN IPs:** `192.168.153.113`, `192.168.153.117` - **Separate MinIO endpoint:** `192.168.153.253:9000` ## Diagram set @@ -62,9 +61,9 @@ Why Canvas: it gives much better layout control in Obsidian than Mermaid for thi - **Ollama embeddings** on `18807` for memory search ### Automation / orchestration -- **OpenClaw runtime:** main assistant in a VM -- **Current main interactive session model:** `gpt-5.4` -- **Local LLM runtime:** llama.cpp on `18806` serving `gemma-3-12b-it-q4_0.gguf` +- **Atlas/Hermes runtime:** default Hermes profile, host-side `~/.hermes` checkout; Telegram/Discord gateway active. +- **Current main interactive session model:** `gpt-5.5` via `openai-codex`. +- **Local LLM runtime:** llama.cpp on `18806` serving `gemma-4-26B-A4B-it-UD-IQ2_M.gguf` (Gemma 4 26B GGUF, ctx 32768). - **n8n-agent:** `18808` for scheduled/API-native workflows ### Speech / media @@ -74,14 +73,14 @@ Why Canvas: it gives much better layout control in Obsidian than Mermaid for thi ## Storage / repos ### MinIO -- Bucket: `zap` -- Purpose: full `~/.openclaw` backups every 6 hours +- Bucket: `atlas` for current Atlas/Hermes laptop backups; legacy `zap` bucket still exists for older OpenClaw-era artifacts. +- Purpose: daily `~/.hermes`/Atlas backup snapshots via Hermes cron script `atlas-backup-to-minio-cron.sh` - Accessed at `192.168.153.253:9000` ### Gitea -- Repo host: `gitea-http.taildb3494.ts.net` -- Repo: `will/swarm-zap.git` -- Purpose: workspace backup / review / sync point +- Repo host: `gitea-gitea-ssh.taildb3494.ts.net` +- Swarm repo: `will/swarm-master.git` +- Atlas/Hermes repo: Gitea `will/atlas-code` (authoritative) with upstream GitHub mirror/backup as documented in Hermes memory. ## Kubernetes @@ -93,8 +92,8 @@ Why Canvas: it gives much better layout control in Obsidian than Mermaid for thi - **Telegram:** primary interrupt / reminder / mobile channel - **Discord:** deeper technical work and scoped threads -- **Obsidian:** shared vault `will/will-shared-zap/` under `/mnt/swarm-common/obsidian-vault/` -- **Google Workspace:** Gmail / Calendar / Drive / Tasks via `gog` +- **Obsidian:** shared vault `will/will-shared-zap/` at `/home/will/lab/swarm/swarm-common/obsidian-vault/will/will-shared-zap`; currently on host btrfs, not a VM `virtiofs` mount. +- **Google Workspace:** Gmail / Calendar / Drive via Hermes/Google Workspace tooling; old `gog`/OpenClaw paths in historical notes may be stale. ## Operating decisions worth keeping @@ -102,12 +101,10 @@ Why Canvas: it gives much better layout control in Obsidian than Mermaid for thi - n8n vs cron: use n8n when it has the right native node or API path, keep shell-heavy local tasks in OpenClaw or OS cron - Shared vault is human-readable layer; workspace memory remains source of truth -## OpenClaw local model provider migration -- Date: 2026-04-15 -- Local llama.cpp Gemma model is now exposed as `local/gemma-3-12b-local` -- Previous namespace was `litellm/gemma-3-12b-local` -- Migration required two config changes in `~/.openclaw/openclaw.json`: - - move the model from `models.providers.litellm.models` to a dedicated `models.providers.local` pointing at `http://192.168.153.130:18806/v1` - - update stale explicit model ref in `agents.defaults.models` from `litellm/gemma-3-12b-local` to `local/gemma-3-12b-local` -- Lesson: changing provider blocks alone may not change the live model catalog or Control UI if old `provider/model` refs remain in defaults/fallback maps -- Verify with: `openclaw models list | grep -i -E "gemma|local/"` +## Local model provider notes +- Date checked: 2026-05-27 +- Current direct local model endpoint: `http://127.0.0.1:18806/v1` / `http://192.168.153.130:18806/v1`. +- Current served model id from `/v1/models`: `gemma-4-26B-A4B-it-UD-IQ2_M.gguf`. +- Ollama on `18807` is for embeddings (`nomic-embed-text`) and also has older chat models installed; do not describe the direct local LLM as Gemma 3. +- The 2026-04-15 OpenClaw `local/gemma-3-12b-local` migration was historical and should not be used as current Atlas/Hermes config guidance. +- Verify with: `curl -s http://127.0.0.1:18806/v1/models`. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/Cron Jobs.md b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/Cron Jobs.md index f11fe5d..83a601d 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/Cron Jobs.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/Cron Jobs.md @@ -3,16 +3,16 @@ title: Cron Jobs area: automation tags: [automation, health, assistant, cron] created: 2026-03-18 -updated: 2026-04-28 +updated: 2026-05-27 status: active related: [[Infrastructure/Architecture]], [[Infrastructure/Automation/n8n Workflows]], [[Plans/Daily Schedule]] --- -# OpenClaw Cron Jobs +# Hermes Cron Jobs -This note reflects the **live OpenClaw cron inventory** as checked on 2026-04-13. +This note reflects a partial **live Hermes cron inventory** checked on 2026-05-27. Older OpenClaw job names below are historical/stale unless they also appear in the current Hermes `cronjob list`. -All jobs run via the OpenClaw cron scheduler. +All current jobs run via the Hermes cron scheduler. ## Enabled jobs @@ -87,18 +87,14 @@ All jobs run via the OpenClaw cron scheduler. ## OS cron (VM) -### MinIO backup -- **Schedule:** every 6 hours at minute `17` -- **Purpose:** full `~/.openclaw` backup to MinIO bucket `zap` -- **Behavior note:** backup script now tolerates benign hot-backup `tar` race warnings like `file changed as we read it`, but fails hard on non-benign archive creation errors so broken archives are less likely to be silently uploaded -- **Failure alerting:** cron should call `scripts/backup-to-minio-wrapper.sh`, which appends output to `logs/minio-backup.log` and sends a Discord alert to `#general` if the backup exits non-zero +### Atlas/Hermes MinIO backup +- **Schedule:** daily at `02:50` America/Los_Angeles via Hermes job `atlas-minio-self-backup` (`c15ee395a38d`) +- **Purpose:** Atlas/Hermes backup to MinIO bucket `atlas` using `atlas-backup-to-minio-cron.sh` +- **Last live check:** job enabled and last status `ok` on 2026-05-27 -### Session transcript cleanup -- **Schedule:** daily at `03:17` (America/Los_Angeles) -- **Script:** `scripts/cleanup-session-transcripts.sh` -- **Purpose:** delete session `.jsonl` files older than 90 days from `~/.openclaw/agents/main/sessions/` -- **Log:** `logs/cleanup-sessions.log` -- **Added:** 2026-03-31 +### System / local automation watchdogs +- **Examples live on 2026-05-27:** `system threshold watchdog` every 10m, `blocked kanban escalation` every 30m, `local-ai-automation-watchdog` daily 03:20, `agent-ops-watchdog` daily 03:40, `hermes-live-checkout-kanban-guard` every 5m. +- **Delivery:** mostly Discord `#ops-alerts` or origin, depending on job. ## Operational notes @@ -110,4 +106,4 @@ All jobs run via the OpenClaw cron scheduler. ## Note -Earlier versions of this note listed several jobs that are not present in the current live cron inventory anymore. This file was refreshed from the actual runtime state on 2026-03-27. +Earlier versions of this note listed several OpenClaw jobs that are not present in the current Hermes cron inventory anymore. Treat this file as refreshed for high-risk/current infra items on 2026-05-27, not a complete replacement for `hermes cron list` / the Hermes `cronjob` tool. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Nightly Vault Sync.md b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Nightly Vault Sync.md index e107a9a..01dbcd2 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Nightly Vault Sync.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Automation/n8n Nightly Vault Sync.md @@ -3,7 +3,7 @@ title: n8n Nightly Vault Sync area: infrastructure tags: [n8n, obsidian, llm, automation, nightly] created: 2026-03-27 -updated: 2026-03-27 23:39 UTC +updated: 2026-05-27 status: active related: [[Infrastructure/Automation/n8n Workflows]], [[Infrastructure/Architecture]], [[Infrastructure/Automation/Cron Jobs]], [[Infrastructure/Services/Docker Services]] --- @@ -16,7 +16,7 @@ related: [[Infrastructure/Automation/n8n Workflows]], [[Infrastructure/Architect - **Workflow ID:** `75JCevkdgkyCr2qH` - **Status:** active - **Trigger:** nightly cron expression `45 23 * * *` -- **Local LLM:** `gemma-3-12b-it-q4_0.gguf` +- **Local LLM:** `gemma-4-26B-A4B-it-UD-IQ2_M.gguf` ## Purpose @@ -31,9 +31,9 @@ Each run: - `Infrastructure/Automation/Cron Jobs.md` - `Infrastructure/Services/Docker Services.md` 2. checks live health from: - - `http://192.168.153.113:18808/healthz` - - `http://192.168.153.113:18806/v1/models` -3. sends that context to the local Gemma model + - `http://127.0.0.1:18808/healthz` + - `http://127.0.0.1:18806/v1/models` +3. sends that context to the local Gemma 4 model 4. writes a nightly note to: - `Notes/YYYY-MM-DD Nightly Vault Sync.md` diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Services/Docker Services.md b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Services/Docker Services.md index fa4454e..c4f2bd5 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Services/Docker Services.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Infrastructure/Services/Docker Services.md @@ -3,14 +3,14 @@ title: Docker Services area: infrastructure tags: [infrastructure, homelab, docker] created: 2026-03-18 -updated: 2026-03-27 +updated: 2026-05-27 status: active related: [[Infrastructure/Architecture]], [[Infrastructure/Automation/n8n Workflows]] --- # Docker Services -Most service containers run on the LAN host at `192.168.153.113`. Some adjacent AI/runtime services run as host-level user services instead of Docker; those are noted separately below. +Most service containers run on Will's laptop/host network and publish local/LAN ports. Current host LAN IPs include `192.168.153.130` and `192.168.153.141`; older `192.168.153.113` / `.117` URLs should be treated as historical unless re-verified. ## Dockerized services @@ -19,7 +19,7 @@ Most service containers run on the LAN host at `192.168.153.113`. Some adjacent - **Image:** `searxng/searxng:latest` - **Port:** `18803 -> 8080` - **Role:** preferred local-first web search backend -- **URLs:** `http://192.168.153.113:18803`, `http://192.168.153.117:18803` +- **URLs:** `http://127.0.0.1:18803`, `http://192.168.153.130:18803` ### brave-search (MCP) - **Container:** `brave-search` @@ -33,8 +33,8 @@ Most service containers run on the LAN host at `192.168.153.113`. Some adjacent - **Image:** `docker.n8n.io/n8nio/n8n:latest` - **Port:** `18808 -> 5678` - **Role:** automation workflows and integration runtime -- **Primary URL:** `http://192.168.153.113:18808` -- **Health:** `/healthz` returned `{"status":"ok"}` during live check on 2026-03-27 +- **Primary URL:** `http://127.0.0.1:18808` / `http://192.168.153.130:18808` +- **Health:** `/healthz` returned HTTP 200 during live check on 2026-05-27 - **Workflows:** see [[Infrastructure/Automation/n8n Workflows]] ### whisper-server @@ -58,9 +58,9 @@ Most service containers run on the LAN host at `192.168.153.113`. Some adjacent ### llama.cpp - **Port:** `18806` -- **Model:** `gemma-3-12b-it-q4_0.gguf` +- **Model:** `gemma-4-26B-A4B-it-UD-IQ2_M.gguf` - **Role:** local OpenAI-compatible LLM endpoint for private/offline reasoning and workflow inference -- **Live check:** model endpoint and chat-completion path re-verified on 2026-03-27 +- **Live check:** `/v1/models` returned Gemma 4 26B GGUF on 2026-05-27 ### Ollama - **Port:** `18807` @@ -71,9 +71,9 @@ Most service containers run on the LAN host at `192.168.153.113`. Some adjacent ### MinIO - **Endpoint:** `192.168.153.253:9000` -- **Bucket:** `zap` -- **Role:** object storage for OpenClaw backups +- **Bucket:** `atlas` for current Atlas/Hermes backups; legacy `zap` bucket still exists for old OpenClaw-era artifacts +- **Role:** object storage for Atlas/Hermes backups ## Operational note -The shared Obsidian vault is not a Docker service, but it is a critical part of this stack. Its Local REST API path was re-verified on 2026-03-27 after fixing cross-user write permissions on the shared `virtiofs` mount. +The shared Obsidian vault is not a Docker service, but it is a critical part of this stack. Current path is `/home/will/lab/swarm/swarm-common/obsidian-vault/will/will-shared-zap`; `findmnt` showed it on host btrfs on 2026-05-27. Older VM/`virtiofs` permission notes are historical. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Obsidian Automation Health.md b/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Obsidian Automation Health.md index 7535fbb..257cb8e 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Obsidian Automation Health.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Obsidian Automation Health.md @@ -1,12 +1,12 @@ # Obsidian Automation Health -Updated: 2026-05-23T01:00:49.446Z +Updated: 2026-05-27T21:00:30.749Z ## Status - Overall: Needs attention - Obsidian REST: OK -- Reindex trigger: {"status":"ok","mode":"incremental","vault_path":"/home/will/lab/swarm/swarm-common/obsidian-vault/will/will-shared-zap","note_count":162,"added":0,"changed":0,"removed":0,"chunks_indexed":0,"vector_count":321,"state_file":"/home/will/.hermes/data/rag-search/obsidian_index_state.json","chroma_path":"/home/will/.hermes/data/rag-search/chroma"} -- RAG/embedding check: {"ok":false,"status":"failed","exitCode":127,"output":"RAG health script missing: /home/will/.hermes/scripts/rag_embedding_health.py","durationMs":0} +- Reindex trigger: {"error":{"message":"500 - \"{\\n \\\"error\\\": \\\"reindex failed\\\",\\n \\\"exit_code\\\": 3,\\n \\\"stderr\\\": \\\"\\\"\\n}\"","name":"AxiosError","stack":"AxiosError: Request failed with status code 500\n at settle (/usr/local/lib/node_modules/n8n/node_modules/.pnpm/axios@1.13.5/node_modules/axios/lib/core/settle.js:19:12)\n at RedirectableRequest.handleResponse (/usr/local/lib/node_modules/n8n/node_modules/.pnpm/axios@1.13.5/node_modules/axios/lib/adapters/http.js:746:9)\n at +- RAG/embedding check: {"ok":false,"status":"failed","exitCode":1,"output":"{\n \"checks\": {\n \"obsidian_semantic\": {\n \"error\": \"HTTPError: HTTP Error 503: Service Unavailable\",\n \"ok\": false\n },\n \"ollama\": {\n \"embedding_dim\": 768,\n \"error\": null,\n \"model\": \"nomic-embed-text\",\n \"ok\": true\n }\n },\n \"durationMs\": 3135,\n \"failed\": [\n \"obsidian_semantic\"\n ],\n \"ok\": false,\n \"status\": \"failed\"\n}","durationMs":3187} This note is automatically overwritten by n8n. diff --git a/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Service Catalog.md b/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Service Catalog.md index ff322a6..01a18d8 100644 --- a/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Service Catalog.md +++ b/swarm-common/obsidian-vault/will/will-shared-zap/Resources/Service Catalog.md @@ -1,7 +1,7 @@ --- type: service-catalog created: 2026-05-14T14:50:46-07:00 -updated: 2026-05-14T14:50:46-07:00 +updated: 2026-05-27T12:12:06-07:00 tags: - service-catalog - swarm @@ -13,7 +13,7 @@ tags: Canonical index of local services, automation tools, Hermes capabilities, and where to find their operational docs. -> Generated/updated by Atlas from live system inventory on `2026-05-14T14:50:46-07:00`. Secrets are intentionally omitted. +> Generated by Atlas from live system inventory on `2026-05-14T14:50:46-07:00`; high-risk local AI/service rows refreshed on `2026-05-27T12:12:06-07:00`. Secrets are intentionally omitted. ## Quick links @@ -41,9 +41,9 @@ Canonical index of local services, automation tools, Hermes capabilities, and wh | Service | Port | Status | Purpose | Health / base URL | | --- | --- | --- | --- | --- | -| Brave Search MCP | 18802 | HTTPError | Brave Search MCP server for Hermes MCP tools | `http://127.0.0.1:18802/mcp` | -| SearXNG | 18803 | TimeoutError | SearXNG metasearch | `http://127.0.0.1:18803/search?q=test&format=json` | -| LiteLLM | 18804 | TimeoutError | LiteLLM OpenAI-compatible model proxy | `http://127.0.0.1:18804/health/liveliness` | +| Brave Search MCP | 18802 | HTTP 406 on plain GET `/mcp` | Brave Search MCP server for Hermes MCP tools | `http://127.0.0.1:18802/mcp` | +| SearXNG | 18803 | OK 200 | SearXNG metasearch | `http://127.0.0.1:18803/search?q=test&format=json` | +| LiteLLM | 18804 | no listener / HTTP 000 on 2026-05-27 | LiteLLM OpenAI-compatible model proxy | `http://127.0.0.1:18804/health/liveliness` | | Kokoro TTS | 18805 | OK 200 | Kokoro local TTS | `http://127.0.0.1:18805/health` | | llama.cpp | 18806 | OK 200 | llama.cpp local LLM | `http://127.0.0.1:18806/v1/models` | | Ollama embeddings | 18807 | OK 200 | Ollama embeddings API | `http://127.0.0.1:18807/api/version` | @@ -61,9 +61,8 @@ Canonical index of local services, automation tools, Hermes capabilities, and wh | Container | Status | Ports | | --- | --- | --- | | n8n-agent | Up 21 hours (healthy) | 0.0.0.0:18808->5678/tcp, [::]:18808->5678/tcp | -| whisper-server | Up 25 hours (healthy) | 0.0.0.0:18811->8080/tcp, [::]:18811->8080/tcp | -| litellm | Up 4 hours (healthy) | 0.0.0.0:18804->4000/tcp, [::]:18804->4000/tcp | -| litellm-db | Up 25 hours (healthy) | 5432/tcp | +| whisper-server-gpu | Up 27 hours (healthy) | 0.0.0.0:18801->8080/tcp, [::]:18801->8080/tcp | +| whisper-server | Up 27 hours (healthy) | 0.0.0.0:18811->8080/tcp, [::]:18811->8080/tcp | | kokoro-tts | Up 25 hours | 0.0.0.0:18805->8880/tcp, [::]:18805->8880/tcp | | brave-search | Up 25 hours | 0.0.0.0:18802->8000/tcp, [::]:18802->8000/tcp | | searxng | Up 25 hours | 0.0.0.0:18803->8080/tcp, [::]:18803->8080/tcp | @@ -246,7 +245,7 @@ Profile Model Gateway Alias Distribu | --- | --- | --- | | Web search | SearXNG `18803` or Brave MCP `18802` | Hermes web search and MCP Brave Search are both available | | Model proxy | LiteLLM `18804` | Use for OpenAI-compatible routed models | -| Direct local LLM | llama.cpp `18806` | Current model: Gemma 4 26B GGUF; useful for n8n/local automation | +| Direct local LLM | llama.cpp `18806` | Current model id: `gemma-4-26B-A4B-it-UD-IQ2_M.gguf`; useful for n8n/local automation | | Embeddings | Ollama `18807` | Use raw Ollama API root, not `/v1`, for `/api/embed` | | Text-to-speech | Kokoro `18805` / Hermes TTS tool | Local speech generation | | Speech-to-text | Whisper `18811` and wrappers | Local transcription fallback |