The inference gateway for your entire stack.
JahKnow is an OpenAI-compatible API gateway that sits in front of local Ollama inference, with JWT-enforced plans, per-route scope control, rate limiting, and full request telemetry.
Production-hardened. Every route scope-controlled.
JahKnow is the same gateway Gem Coral runs its own products on, hardened through recurring internal security review and real production use, not built for a brochure.
Every endpoint requires a valid JWT. Scopes are enforced per route, not per service: a token scoped for chat:write cannot reach admin or telemetry routes regardless of the caller.
Rate limiting is applied at the gateway layer before any inference request reaches Ollama. Burst and sustained limits are configurable per API plan tier, ensuring that no single consumer can saturate the GPU.
The gateway is OpenAI-API-compatible. Any service that talks to OpenAI can be pointed at JahKnow with a base URL change: no SDK changes, no prompt rewrites.
Everything you need to operate inference at scale.
Any SDK or service targeting the OpenAI API works with JahKnow unchanged: set base_url to the gateway, swap your API key, and inference routes to local models.
Inference requests enter a lightweight queue before reaching Ollama. This prevents GPU saturation under concurrent load and provides backpressure signals to upstream callers.
Each route can be mapped to a different local model in configuration. Reasoning tasks route to the primary orchestration model; utility tasks route to a faster, lighter model, with no code changes needed to update assignments.
Chat completions support server-sent events streaming. Responses are proxied token-by-token from Ollama to the client, compatible with all OpenAI streaming clients.
Every request emits a structured JSON log line with request ID, model, latency, token count, plan tier, and outcome. Ingest into any log aggregator with no parsing required.
All inference runs through the local Ollama instance. No OpenAI tokens, no Anthropic billing, no Azure endpoints. Cost per query: the electricity to run the GPU.
Your inference layer. Your hardware. Your data.
JahKnow is part of Gem Coral's infrastructure portfolio. Reach out if you'd like to discuss what it demonstrates, or how a gateway like this could apply to your organization.