Probabilistic vs. Deterministic
This concept is the fundamental distinction you need to understand when developing software with AI.
I start every new task by asking myself this question first:
Does this task require a probabilistic solution, a deterministic solution, or a combination of both?
For the purposes of this conversation, I'm defining the terms like this:
- Probabilistic: the direct output of a large language model (LLM); often standardized via a repeatable
SKILL.mdfile - Deterministic: a function or script written in a traditional programming language
I'm playing a bit fast and loose with the terms here. For example, a function named GetToday() that returns the current date is not technically deterministic, since its output tomorrow will be different than its output today. But for our purposes here, I would treat it as "deterministic" in order to distinguish it from its "probabilistic" counterpart, which would be to prompt an LLM for the current date.
A direct LLM response will give you a date. But it won't necessarily be today's date.
Just like the United States would eventually fight a war to end slavery–just not the Revolutionary War:
Probabilism Leads to Hallucinations
The probabilistic nature of LLMs is what causes it to hallucinate.
A pure LLM is little more than a highly-trained probability machine. Given a set of words, an LLM will predict what the next set of words will be based on its extensive training. So, when you ask it, "What is today's date?" it will search for connections from it's voluminous corpus of training data for the most common words likely to be found as an answer to that question. It might choose "January" because it has more days than other months and then "1" because it's the most common digit to follow the word "January" in its training data.
Determinism is the Solution
Careful readers will note that I referred to "pure" LLMs in my explanation above.
If you prompt an AI today with, "What is today's date?" it will likely return an accurate answer, despite what I wrote above. That's not because newer LLMs have solved the hallucination issue. It's because modern LLMs combine their probabilistic "creativity" with deterministic reliability via tool calls.
Large Language Models have a training data cutoff date. It would be impossible for the language model itself to tell you today's date based only on its training data. Instead, modern language models are built to call tools on an as-needed basis.
So now when you ask ChatGPT for the current date, it simply uses some "tool" at its disposal (e.g., a Windows API call, a Powershell cmdlet, etc.) to query the authoritative "current date."
This combination of probabilistic responses and deterministic tool calls is the source of many of the perceived improvements in AI models over the past 6 to 12 months.
Probabilistic or Deterministic?
Back to that first question I ask myself:
Does this task require a probabilistic solution, a deterministic solution, or a combination of both?
When answering that question, I consider the relative strengths of each approach:
Deterministic Strengths
- Reproducible
- Auditable
- Reliable
- Cheap (requires no AI inference [aka, tokens])
- Fast
Probabilistic Strengths
- Flexible
- Resilient
- Adaptable
- Faster to write
Rules of Thumb
- Choose Deterministic for tasks that are well-defined, repeatable processes
- Choose Probabilistic for one-off tasks that need to handle arbitrary inputs
- Use a Combination for repeatable tasks with ill-defined inputs
For recurring developer tasks, I often start with a probabilistic solution (because they are quick to develop and flexible enough to handle new situations). If the process recurs frequently or would benefit from more reliable outcomes, then I will often use the first few probabilistic runs to inform the building of a deterministic solution.
This is very similar to my "Copying and Pasting Code with Purpose" approach:
Mike Wolfe
