I keep thinking about how strange it is that something could know everything about the letter "H": its shape, its sound, how it fits into "hello," but never actually experience it. What does it even mean to learn something without sensing it?

I think about Mary's Room a lot. This scientist who's lived her whole life in a black and white room. She knows everything about color, wavelengths, light reflections, how our brains process different colors, but she's never seen color herself. When she finally steps out and sees red for the first time, something changes. There's a kind of knowledge that only comes through direct experience, something that all her intellectual understanding could never give her.

Language models like ChatGPT remind me of Mary before she sees color. They process letters, words, and patterns at an incredible level, but their understanding is purely diegetic, stuck inside the internal, propositional content of words. They know about the letter "H," how it fits in the alphabet, how it works in sentences, the statistics of how it appears. But can a model that has never seen the letter "H" in the real world, never interacted with its shape or seen it in context, ever really understand it like we do?

This leads us to the idea of supradiegetic knowledge—understanding that goes beyond just the internal logic of language and connects with how it actually exists in the world. It's the difference between knowing "H" is the eighth letter of the alphabet and knowing what it's like to see "H" on a page, spot it in your peripheral vision, tell it apart from other letters at a glance. That physical relationship with language is something we take for granted, but it's exactly what models like ChatGPT don't have.

The gap between diegetic and supradiegetic knowledge feels fundamental. Just like Mary can describe color but not know it until she sees it, language models can process words without ever really getting the sensory aspects that make those words meaningful. This raises deeper questions about what it means to understand language at all. For us, language is something we experience through sight, sound, and touch. For a model, it's just patterns of tokens, like seeing only the shadow of what we actually experience every day.

Looking at it this way, LLMs like ChatGPT are powerful but incomplete. They exist in this kind of linguistic limbo where they can work with words and sentences but miss the full picture of what language is. And like Mary, they stay in the dark unless we can somehow give them true sensory experience, if that's even possible at all.