05/05/2026
How LLMs Actually "Think"
I’ve been diving into the mechanics of Large Language Models (LLMs). It’s easy to treat these tools like magic, but the logic under the hood is fascinating. Here is how they actually process our world:
1. Meaning to Math
Computers don’t understand words; they understand numbers. Models use "Embeddings" to turn words into vectors. This allows the AI to mathematically understand that words like "King" and "Queen" are related because they share similar data points.
2. The Prediction Loop
When an LLM answers you, it isn't writing a full sentence at once. It is running a continuous loop, predicting the single most likely next word based on the patterns it learned during training. It does this over and over until the thought is complete.
3. The Context Window
Every model has a "Context Window," which acts like its short-term memory. Once a conversation exceeds this limit, the model has to let older information fade away to make room for the new. This is why long chats can sometimes feel like the AI is "forgetting" the beginning of the prompt.
Understanding these basics completely changes how I write my prompts.
