Banya AI 6: Metacognition from Self-Output Distribution , Routing Certain, Vague, and Unknown States

Metacognition, the ability to know whether one knows, can be extracted without any dedicated circuit by having the model read its own output distribution, the per-next-word probabilities. This paper measures whether three states — certain (what the model knows), vague (what is ambiguous), and unknown (what it does not know at all) — separate using only statistics of the output distribution. The statistics are the max probability (pmax, the probability of the most likely word), the normalized entropy (a measure of how spread the distribution is: 0 means concentrated at one point, 1 means uniformly spread), and the peak count (the number of words whose probability exceeds a threshold). Measurement shows that certain concentrates at one point with max probability 1.00 and entropy 0.00, separating clearly from the rest. Vague (max probability 0.32, entropy 0.35) and unknown (0.35, 0.37) both separate strongly from certain but overlap each other in the statistics. One cause is that the words designed for the unknown bundle (airplane, computer, dinosaur) in fact appear many times in the training corpus, so the item design was not truly unknown. What can be read directly from the output distribution without any extra training or labels is the split between certain and non-certain: the model knows whether it is certain merely by monitoring its own distribution. This signal can be used for routing — deciding which answers to emit as-is, which to refine further, and which to mark as unknown. Moreover, the signal also works as a trigger. When the vaguest hidden states extracted from the real corpus (max probability 0.11 to 0.13) are jointly reinterpreted with the immediately preceding context placed in front, they cross the certainty threshold of 0.55 in a single pass and switch to the certain state, whereas the context-free solo-reinterpretation control barely rises over the same number of passes. This record contains the English edition (_en.pdf) and the Korean original (_kr.pdf). Part 6 of the six-part Banya AI research series. Part 1: https://doi.org/10.5281/zenodo.21385447. Part 2: https://doi.org/10.5281/zenodo.21385516. Part 3: https://doi.org/10.5281/zenodo.21385676. Part 4: https://doi.org/10.5281/zenodo.21385699. Part 5: https://doi.org/10.5281/zenodo.21385717. All measured figures are reproducible with the public code package: https://ubmscoin.github.io/banya/banya_ai_paper_code/ (code Apache-2.0). Frozen model checkpoints: https://doi.org/10.5281/zenodo.21383724