When large language models generate step-by-step reasoning paths, their visible textual tokens do not necessarily reveal the full mechanics of their computation. Researchers at South Korea's KAIST and Naver AI Lab set out to test whether these reasoning steps correspond to distinguishable internal patterns across numerical states rather than simply reflecting surface-level word choices.
Tracing Logic in Hidden Layers
To measure these representations, the KAIST and Naver AI Lab team isolated eight recurring reasoning operations, including extraction, decomposition, formula recall, deduction, and computation. They tasked Qwen2.5-7B, Qwen-8B, and Gemma-31B with solving math problems, breaking down the resulting solution paths into distinct segments.
The findings confirmed that internal representations reliably reflect the active operation, with separation peaking in the middle layers of the models. A classifier analyzing internal numerical representations outperformed a classifier that evaluated only the tokens used, proving that hidden states encode functional logic beyond vocabulary.
The same word gets a different internal representation depending on which reasoning step it belongs to.
Even standard function words such as "a," "is," or "the" split into distinct numerical representations in deeper layers based on whether the model is executing a deduction or running a calculation. When researchers blocked attention to the preceding 30 tokens using targeted interventions, the operational signal weakened, showing that reasoning steps build directly upon their surrounding context.
Identifiable Steps and Unfaithful Outputs
Internal functional signatures persisted even across failed reasoning paths. When models missed the correct solution, the specific operation remained identifiable; a flawed computation step continued to exhibit the internal characteristics of calculation despite returning an incorrect numerical result. The separability of operations was further replicated in Llama-3-8B, while classifiers trained on Qwen-8B transferred successfully to the GPQA-Diamond and MATH-500 benchmarks.
For enterprise engineering, this shifts the paradigm from treating AI as an inscrutable black box to monitoring intermediate generation steps. By verifying the computational validity of hidden activations rather than relying on surface-level text, technical leads gain a concrete mechanism to audit reasoning fidelity. However, current empirical evidence remains bounded by mathematical tasks across a narrow set of architectures, and whether internal state classifiers can actively intercept generation errors in real time remains an open research question.