Ask a large language model the same question twice and it may not give the same answer. That might feel quirky when you’re debating pizza toppings with ChatGPT, but in regulated industries it’s a governance nightmare. Oversight depends on reproducibility. Auditors and regulators don’t care if your model is “creative”; they care if it is testable.
Determinism, in theory, is straightforward: same input, same output, every time. In practice, it’s trickier. Atil et al. (2024) demonstrated that even at “temperature 0,” outputs varied significantly across runs. Song et al. (2024) went further, warning that single-shot evaluation masks this variance entirely, creating benchmarks that look solid but rest on shaky ground.
So much for our neat definition. But things get more interesting once you leave the algorithm and peek into the machinery running it.
I’ll admit something. More than once this year, I’ve told clients: “LLMs are non-deterministic by nature.” It sounded convincing. After all, they’re probabilistic systems, built to approximate language rather than calculate it like a spreadsheet. My point was that such systems need special governance treatment—still true. But the first part, that they are inherently non-deterministic, turns out to be wrong, or at least incomplete.
Recent work shows the real story isn’t just in the model. Much of the variability comes from the way inference is executed on modern hardware. When workloads are split across multiple GPU kernels, tiny floating-point differences, concurrency quirks, and batching strategies can yield divergent outputs. The Thinking Machines team (2025) demonstrated that even with sampling randomness dialed down to zero, these low-level effects were enough to produce different answers.
That changes the narrative. Non-determinism is not simply a property of “how LLMs think.” It’s a property of how we run them. Governance, in other words, must account not just for the probabilistic model but for the infrastructure stack that breathes life into it.
For enterprises, this isn’t technical trivia. It’s a governance lever.
• Root cause analysis: if your system is deterministic, you can tell whether a deviation came from the model itself or from your infrastructure. Without that, you’re stuck in a blame-passing loop.
• Regulatory defense: the EU AI Act doesn’t outlaw non-determinism, but it does demand explainability and reproducibility. A model that behaves differently each time makes it nearly impossible to show a regulator how a decision was reached.
• Vendor contracts: when suppliers quote performance numbers, stability across runs matters as much as headline accuracy. Without determinism, you’re negotiating on averages, not guarantees.
• Monitoring at scale: regression tests and golden datasets only work if results are stable. Otherwise, every alert risks being noise.
Determinism won’t make models smarter. A deterministic mistake is still a mistake. But it gives you a fighting chance to govern responsibly. The new understanding—that infrastructure itself introduces non-determinism—changes the equation. Risk management isn’t just about which model you buy, but how you run it.
For organizations building AI into high-stakes processes, that insight should ring alarm bells. If you can’t reproduce results, you can’t defend them. And if you can’t defend them, you’re one regulatory letter away from trouble.
Predictability may not be glamorous, but it’s the quiet foundation on which trust, and compliance, rests.