swarm-zap/memory/2026-03-03-litellm-test.md

Session: 2026-03-03 18:00:45 UTC

• Session Key: agent:main:main
• Session ID: fe1b7054-41fb-462f-8177-f4e8998f988b
• Source: gateway:sessions.reset

Conversation Summary

assistant: reply_to_current Done — updated.

What’s now in litellm-config.yaml

• Gemini was already present:
  • gemini-1.5-pro
  • gemini-1.5-flash
• z.ai added:
  • zai-glm-4.7
  • zai-glm-4.7-flash
  • both using:
    • api_base: https://api.z.ai/api/coding/paas/v4
    • api_key: os.environ/ZAI_API_KEY

If you want, I can also add newer Gemini aliases (e.g. 2.x) and/or switch Gemini key var from GOOGLE_API_KEY to GEMINI_API_KEY if that’s what your env uses. user: [Tue 2026-03-03 09:37 UTC] Update the file accordingly based on the currently configured providers assistant: reply_to_current Done — I updated litellm-config.yaml to match the providers currently configured on this host.

Kept (configured)

• OpenAI-compatible local endpoint (OPENAI_BASE_URL + OPENAI_API_KEY)
• z.ai (ZAI_API_KEY, base https://api.z.ai/api/coding/paas/v4)
  • glm-5
  • glm-4.7
  • glm-4.7-flash
  • glm-4.7-flashx

Removed (not currently configured)

• Anthropic
• OpenRouter
• Opencode
• Gemini

If you want Gemini included too, I can add it back with GEMINI_API_KEY entries as soon as you confirm that key/env is set. user: [Tue 2026-03-03 09:42 UTC] We have litellm fixed. Can you test it? assistant: reply_to_current I tested from here — current status:

• http://192.168.153.113:18805/v1/models responds, but returns TTS models (kokoro, tts-1, etc.), not your LiteLLM model list.
• http://192.168.153.113:18805/v1/chat/completions returns 404 Not Found.
• No local LiteLLM listener appears to be running on this host (127.0.0.1:4000/8000 not reachable).

So LiteLLM itself is likely on a different port/host (or not started yet).

If you share the exact LiteLLM base URL (e.g. http://<host>:<port>/v1), I’ll run a real end-to-end test immediately. user: [Tue 2026-03-03 09:43 UTC] NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS brave-search mcp/brave-search:latest "node dist/index.js" brave-search 2 hours ago Up 2 hours 0.0.0.0:18802->8000/tcp, [::]:18802->8000/tcp kokoro-tts ghcr.io/remsky/kokoro-fastapi-cpu:latest "./entrypoint.sh" kokoro-tts 2 hours ago Up 2 hours 0.0.0.0:18805->8880/tcp, [::]:18805->8880/tcp litellm litellm/litellm:latest "docker/prod_entrypo…" litellm 2 hours ago Up 3 minutes (unhealthy) 0.0.0.0:18804->4000/tcp, [::]:18804->4000/tcp searxng searxng/searxng:latest "/usr/local/searxng/…" searxng 2 hours ago Up 2 hours 0.0.0.0:18803->8080/tcp, [::]:18803->8080/tcp whisper-server ghcr.io/ggml-org/whisper.cpp:main "whisper-server --mo…" whisper-server 2 hours ago Up 2 hours (healthy) 0.0.0.0:18801->8080/tcp, [::]:18801->8080/tcp assistant: reply_to_current Yep — tested. Good news/bad news:

✅ LiteLLM is reachable at :18804 and serving model aliases:

• gpt-4o
• gpt-4-turbo
• zai-glm-5
• zai-glm-4.7
• zai-glm-4.7-flash
• zai-glm-4.7-flashx

❌ Actual completions fail (auth upstream):

• zai-glm-4.7-flash returns Authentication parameter not received in Header
• Health endpoint shows all configured backends unhealthy for missing upstream API keys.

So your LiteLLM config is loaded, but container env is missing provider keys (ZAI_API_KEY, and for OpenAI route, OPENAI_API_KEY if used upstream directly).

If you want, I can give you a minimal docker-compose env block to make it green in one restart. user: [Tue 2026-03-03 10:09 UTC] Can you try to hit the litellm API? Here is a virtual key: sk-IlDTV8FaW0vTUL_a4VDhMw assistant: reply_to_current Tested with your virtual key.

• GET /v1/models ✅ works (returns your configured aliases)
• POST /v1/chat/completions ❌ still fails:
  • zai-glm-4.7-flash: upstream auth missing
  • gpt-4o: upstream OpenAI API key missing

So your virtual key is valid for LiteLLM, but LiteLLM itself still lacks provider creds in its container env.

You need to set (at least):

• ZAI_API_KEY (for z.ai models)
• OPENAI_API_KEY (for gpt-4o / gpt-4-turbo)

Then restart litellm container and re-test. user: [Tue 2026-03-03 10:41 UTC] ok test now assistant: reply_to_current Yep — much better now. I re-tested and it’s working.

Results

• /v1/models ✅ returns models (now includes Claude/Gemini/z.ai aliases too).
• gpt-4o chat completion ✅ returned OK.
• zai-glm-4.7-flash chat completion ✅ works too (returned OK when given enough tokens).
• /health now shows partial recovery: 7 healthy / 4 unhealthy.

So LiteLLM is up and functional with your virtual key.
If you want, I can run one quick probe to list exactly which 4 models are still unhealthy. user: [Tue 2026-03-03 11:07 UTC] Test litellm with gpt-5-mini assistant: reply_to_current Tested.

gpt-5-mini currently fails on your LiteLLM with:

no healthy deployments for this model (model=gpt-5-mini)

So it isn’t mapped/configured (or healthy) in your current LiteLLM setup yet. user: [Tue 2026-03-03 11:08 UTC] Test litellm with openai/gpt-5-mini assistant: reply_to_current Tested openai/gpt-5-mini.

Result: ❌ fails with:

OpenAIException - Unknown parameter: 'credential_name'

So this route is being attempted, but there’s a provider-side/request-shape mismatch in the current LiteLLM/OpenAI config for that model.