Integration
Fact-checking & guardrails for local models
Local models serve different use cases than cloud models – specialized domains, latency constraints, or specific inference hardware. They still hallucinate, follow injected instructions, and emit malformed JSON just as readily as hosted models. The proxy guards local models with the identical pipeline it runs on cloud traffic, screening and validating regardless of where the model runs.
Point the proxy at any OpenAI-compatible local server – LM Studio, or any other – and the identical pipeline that guards hosted models guards your local inference. Route sensitive workloads to local models; route the rest to cloud providers; the proxy enforces the same checks on all traffic regardless of where the model runs.
1. Serve the model
Run LM Studio (or any OpenAI-compatible server) with your local models – the proxy treats it as an upstream like any other.
2. Point a key at it
Configure a proxy key with the local server’s URL as its upstream. Different keys can front different upstreams – local for sensitive workloads, hosted for the rest.
3. Keep the same client code
Apps call the proxy the same way regardless of which upstream serves the request – switching a workload to local is a key configuration, not a code change.
4. Watch quality per model
Grounding and safety findings per request show you exactly how your local models compare to hosted ones on your own traffic.
Why do local models need guardrails at all?
Residency controls where data goes, not what the model does. A local model still alters figures, invents citations, obeys injected instructions, and breaks JSON – the failure modes guardrails exist for are model behaviors, not hosting properties.
Can the whole stack run offline?
The proxy, dashboard, and database ship as Docker Compose and can run beside your model server; core checks (PII, numeric grounding, format, patterns) are local. Optional external grounding – web search, hosted judge models – requires the corresponding connectivity and skips cleanly when absent.
Which local servers work?
Anything exposing an OpenAI-compatible chat-completions API. LM Studio is the documented path; the adapter treats compatible servers interchangeably.
Do agent guardrails work with local models?
Yes, when the local model emits OpenAI-style tool calls – the tool policy, tiers, grounding, and budget stages read the same structures regardless of where the model runs.
Guard your local models too
Front LM Studio through the proxy and run the same guardrails on models that never leave your machine. We are running a limited demo - sign up and we will get you in as soon as we can.