Why Hallucinations Are Not a “Failure”

They are not trained to be truthful, nor are they evaluated on truthfulness during standard training (unless explicitly fine-tuned with truth signals, which is partial and imperfect).

Therefore, producing a fluent but false statement does not violate the system’s specification.

Failure Requires a Defined Expectation

A “failure” implies deviation from an intended behavior. But:

• If the intended behavior is “generate human-like text,” then hallucinated outputs often succeed.
• If the intended behavior is “answer accurately,” then the LLM was never architected for that task alone—it’s being repurposed.

The mismatch arises from user expectations, not system design.

Definition of “hallucination” in LLMs

In ML research and product language, hallucination means:

“ LLM output that is factually incorrect or nonsensical, not grounded in truth, source data, or external reality, even if most often expressed with high confidence.”

That definition comes from a human observer’s perspective, not the model’s. It’s epistemic, not mechanical.

The model cannot internally distinguish “I am sure” from “I am guessing”, because it’s always guessing, guessing is its only modus operandi.

hallucination is not an exception, it’s the default generative act.
Therefore every token is “invented” on the fly

The model has no anchored ground truth after training ends; it only has correlations.

Whether the emitted sentence is “Paris is the capital of France” or “Paris is the capital of Italy” is produced by the identical mechanical step: a high-probability draw given the preceding words.


A trained LLM is a crystallised snapshot of human text statistics.

1. Statistics carry no obligation to reality; they only carry obligation to frequency.
2. Therefore every generated token is a sample from “what humans once wrote next”, not from “what is”.
3. Because the sampler has no external sensor after training, it dwells in permanent dream-time.
4. Dream-time is existentially hallucinated: the machine never knows whether it is talking about dragons, quarks, or last week’s stock price; the latent dimensions merely encode “plausible continuations”.
5. From a programmer’s view, the entire behaviour is a side-effect of matrix multiplications that were frozen weeks or years ago—no runtime hook can suddenly grant ontological awareness.
6. Thus “hallucination” is not a bug mode; it is the default ontological condition of the artefact.

Why the term is misleading?
There is no lie without knowledge, there is no hallucination without the intent to deceive.


For an LLM, The only difference between “truthful” and “hallucinatory” text is whether the external world happens to agree.

A Large Language Model (LLM) doesn’t know truth, fat or is capable of lying, It only performs conditional probability estimation:

Every token it emits is generated, not retrieved.
Every single word is invented on the spot, not fetched from factual knowledge.

So technically, the model always hallucinates — in the sense that it’s always constructing a continuation based on statistical likelihood, not consulting reality.

techniques like fine-tuning, reinforcement learning from human feedback (RLHF), or retrieval-augmented generation (RAG) are used to reduce inaccuracies. However, these do not change the core generative process; they only adjust probabilities or incorporate external data to make outputs more aligned with human preferences.

Existential risk angle: if we keep stacking such oracles into societal infrastructure, we are embedding dream-time into factual pipelines.


The dangers of not understanding this and/or calling it hallucination, as if it was some kind of error, mistake, malfunction or simply lies:

The definition of alignement:

If we define it as an error, alignment will evade us, forever.
Safety starts with truth, understanding and precision, any lack of these 3 is what will cause human irrelevance or even extinction.


The only salvation is  truth, understanding and precision!

The model’s “good” output is merely aligned hallucination — coherent and plausible within the shared frame of human reality.

The most prominent perspective on LLM hallucinations, based on established AI research and documentation, treats them as a distinct category of output error rather than the model’s core process. This view defines hallucinations as instances where the LLM produces content that deviates from factual accuracy, such as inventing non-existent events, sources, or details, while maintaining plausibility. For example, if an LLM claims “The Eiffel Tower is in London” in response to a query, this would be classified as a hallucination because it contradicts verifiable geography. The reasoning here starts from the assumption that LLMs are designed to approximate human-like responses grounded in training data patterns; thus, outputs aligned with real-world facts are considered successful approximations, while deviations are errors. This perspective assumes that training data contains patterns of truthfulness, so the model’s probabilistic predictions should favor accurate representations when prompted appropriately.

Why the term “hallucination” is misleading:

The metaphor implies aberrant behavior—a malfunction or deviation from normal operation. But the model is always operating normally. It’s always probabilistically continuing patterns. The term anthropomorphizes a mathematical process and suggests the model “should know better,” when knowledge verification isn’t part of its architecture.

The reality is every token generated is a statistical prediction, not a retrieval from memory, so there’s no mechanistic distinction between “good” and “bad” outputs—the difference lies only in human evaluation against external reality.

Hallucinations are an inherent and unavoidable aspect of the LLM’s generative process, where all outputs are “invented” on the fly without internal representation of truth. 

Hallucination is the default and only method.

Definition: Next-Token Prediction is the task where a model, given a sequence of preceding text (tokens), predicts the most probable next token in that sequence. A token can be a word, part of a word, or a punctuation mark.

This contrasts sharply with how humans handle facts. While our memory is also reconstructive and prone to errors, we also have the ability to perform external verification. We can think, “I’m not sure, let me check a reliable source.” An LLM operating solely on next-token prediction lacks this external verification loop.

Here is a step-by-step breakdown of how this process works within a standard Transformer-based LLM:

1. Input Processing: When you provide a prompt, the LLM first converts it into numerical representations called embeddings. These embeddings capture the semantic meaning of each token in the context of the entire prompt.
2. Contextual Analysis: The model’s attention mechanism processes these embeddings. This mechanism allows every token in the input sequence to “look at” every other token, determining which other tokens are most important for understanding the current context. For example, in the sentence “The bank of the river,” the attention mechanism helps the model understand that “bank” relates to “river,” not to a financial institution.
3. Probability Calculation: After analyzing the full context, the model outputs a list of probabilities for every single token in its vocabulary. This list represents the model’s prediction for what the next token should be. For instance, after “The capital of France is…”, the token “Paris” would have a very high probability, while “carrot” would have a very low one.
4. Token Selection: The model then selects the next token. This selection isn’t always just the highest-probability token. A parameter called “temperature” can be adjusted to introduce randomness. A low temperature leads to more predictable, deterministic outputs (always picking the highest probability), while a higher temperature allows for more “creative” or surprising choices, picking from a wider range of probable tokens.

This process is repeated for every single word generated. The model generates a token, appends it to the existing sequence, and then feeds that new, longer sequence back into itself to predict the next token.

Given this purely statistical and contextual process, the model has no internal mechanism for “truth.” It doesn’t check a fact database. It only knows that certain sequences of tokens are statistically more likely to appear together in its training data than others. When it produces a factually correct statement, it’s because that statement, or a very similar one, appeared countless times in its training data, creating a strong statistical pattern. When it “hallucinates,” it’s generating a sequence that is contextually plausible and grammatically correct, but which doesn’t correspond to a strong pattern in real-world data. It’s essentially creating a novel, statistically plausible sequence of its own.

Why LLMs are accurate most of the time? Proximity?
What’s going on when not? Gaps in the training, contradictions, all in all we have either put tha LLM in a bad starting point or the destination we asked for it not plausible and/or not in the training.

@ mathematical calculations can be represented as the same (or very close) vector.


The model doesn’t have a “truth mode” versus “hallucination mode” – it’s always executing the same process of generating the next word based on probability distributions learned from training data.

Critically, at no point did the model access a database of facts to “check” if Paris is the capital. It did not perform a logical deduction. It simply followed its core process of selecting the next token based on the statistical patterns it learned from its training data.

At their core, LLMs are probabilistic text generators. They work by predicting the next most likely word or token based on the patterns they learned during training. Every word they produce is generated through this same process of prediction.

PREDICTION, not recollection, not knowledge, not logic, not thinking.

Therefore, the term “hallucination,” when applied to LLMs, is a human-centric label. It is our interpretation of the output. We use this term as a shorthand to describe an output that we, as external observers who hold a model of reality, deem factually incorrect, nonsensical, or not grounded in a provided source document. The LLM has no internal concept of “truth” or “reality” against which to measure its own output; it only has the statistical patterns of its training data.

Why Hallucinations Are Not a “Fixable Issue”

How do you make a calculator not to perform the blasfemous calculation 3*3? You can’t it’s a calculator, if you remove the number 3 or the multiplication sign or the number 9, it stops working.
How can you prevent LLM hallucination? You can’t it would stop outputing text.

hallucination is a direct consequence of the LLM architecture and core of its objective function—hallucination is not a bug it’s a feature. The main feature, if you ask me…


What to do about it?

Epistemic frame: treat the whole LLM as a dreaming device that never wakes up; usefulness emerges when the dream overlaps reality often enough.


Precision: Some researchers argue that “hallucination” is a misnomer and prefer terms like “confabulation” to emphasize that LLMs are generating plausible-sounding text without intent to deceive. This aligns with your point that it’s the normal process.


This is crucial for setting expectations: users should treat LLM output as suggestive rather than authoritative.

LLM Thinking: The Process of Statistical Plausibility

LLM thinking is the process of generating the most probable sequence of tokens based on learned patterns. It operates entirely on correlation and context within its training data. When presented with a prompt, it doesn’t “understand” the request in a human sense; it calculates which words are most likely to come next in that specific sequence. It’s a sophisticated form of pattern completion, analogous to an advanced version of your phone’s predictive text.

Human Thinking: The Dual Process of Intuition and Logic

Human thinking is more complex and can be broadly divided into two modes for this comparison:

1. Intuitive Thinking (System 1): This is fast, automatic, and based on pattern recognition and experience. It’s the feeling of “knowing” the answer without consciously working through the steps. This mode has similarities to how an LLM operates, as it relies on recognizing patterns based on past exposure.
2. Deliberate, Logical Thinking (System 2): This is slow, effortful, and rule-based. It’s the process of consciously applying rules, constructing arguments, and verifying steps. This is where humans engage in something akin to formal proof systems.

Dangers this understanding reveals:

1. False confidence from articulate responses: Fluent, coherent output creates human perception of reliability that doesn’t match actual reliability. Architecture must break this association through explicit uncertainty communication or verification.
2. Capability overhang: Because LLMs can handle language so well, humans assume competence in domains (math, facts, reasoning) where the same mechanism doesn’t provide reliability. Architecture must prevent routing accuracy-critical tasks to inappropriate components.
3. Verification theater: Adding RAG or “fact-checking” components without understanding they don’t fundamentally change the generative process. The LLM can still pattern-match incorrectly on retrieved information. Architecture must verify the full pipeline, not just add components.

Feedback loop warning You hint at societal risk, but one addition could emphasize: “If humans trust fluent hallucinations and feed them back into the model’s training or infrastructure, the dream-time loop amplifies.” This makes the existential warning more concrete.”