OpenAI Rolls Out Decisions API as Decision‑Model Race Intensifies

OpenAI announced the broad release of its Decisions API this week, stepping into a market segment that prizes ultra‑fast, low‑cost inference for routing, classification, and confidence‑scoring workloads.

The category was popularized by TypeSafe’s Jev model, which acts as a super‑classifier: given a set of options it returns a choice and a confidence score in a fraction of a second, often replacing full‑scale LLMs in production pipelines.

Rather than training a brand‑new decision model, OpenAI reconfigured its Luna model to cut latency from 1–3 seconds down to 100–180 milliseconds, and added voice and vision modalities. Early adopters such as Box, Datadog, Ramp, and Speak report throughput gains of an order of magnitude.

Why it matters for GPU / AI infrastructure

Decision‑model workloads demand consistently low latency across geographies, pushing cloud providers to place inference capacity nearer to users and to optimize GPU scheduling for many short‑lived requests. For a GPU cloud like AiGpu, this trend translates into higher demand for geographically distributed, low‑latency GPU instances and for tooling that can autoscale thousands of tiny inference jobs per second.