Local models

Lumoswitch can register the OpenAI-compatible endpoints exposed by Ollama and LM Studio as upstreams. Synchronous inference originates from the Lumoswitch inference service, so both the control plane and Lumoswitch Runtime networks must reach the model server: the control plane performs connection tests and model discovery, while Lumoswitch Runtime makes real model calls.

Ollama

1. Pull and start a model in Ollama.
2. Confirm its compatible API is available.
3. Add an Ollama upstream in Lumoswitch.
4. Use a base URL reachable by both the control plane and Lumoswitch Runtime, such as http://ollama:11434/v1.
5. Run connection, discovery, and chat tests.

Use a model that exists in ollama list; llama3.2 is only the preset example.

LM Studio

Load a model, start Local Server, and add an LM Studio upstream. http://127.0.0.1:1234/v1 works only when Lumoswitch Runtime and LM Studio share a host; use a routable container or network address otherwise. Then test connection, model discovery, and chat.

Understand localhost

localhost separately refers to the Lumoswitch control plane or Lumoswitch Runtime process that is making the request, not necessarily your computer. The same base URL must be reachable from both environments. Across containers, use a service name or container-network address. A remote runtime normally cannot reach a model on your laptop's loopback interface.

Do not expose a local model port directly to the public internet. Prefer a controlled private network, VPN, or secure tunnel.

Some Lumoswitch instances block private addresses to reduce SSRF risk. If a private upstream is rejected, ask the instance administrator whether local upstream access is permitted.

Troubleshooting

• Empty model list: make sure the model is downloaded and loaded.
• Connection failure: test from both the Lumoswitch control-plane and Lumoswitch Runtime networks, not only from your browser.
• 127.0.0.1 fails in Docker: use the model service name or a host-reachable address.
• Slow responses: check inference hardware, context size, and concurrency.