0 — How LLMs Actually Work (Quick, Casual Explainer)

You don't need a CS degree for this. You need the right picture in your head instead of a wrong one. Most people's mental model is "it's a search engine," or "it remembers me," or "it's kind of thinking like a person." All wrong. Here's what's actually going on, in plain terms.

01The one-liner

An LLM guesses the next word, over and over, based on everything written so far. That's it. That's the whole trick. Chat, code, "reasoning," all just that guess, repeated thousands of times per answer.

02ELI5: how it actually answers you

Imagine the world's best autocomplete. You type "I went to the," your phone suggests "store." An LLM does that, except:

it's insanely good at it, trained on a huge chunk of the internet
it doesn't stop at one word, it keeps guessing the next one, and the next, using its own previous guesses as new input
it can hold a much bigger picture in mind than "the last word or two." It looks back over your whole question and its whole answer so far, every single time it picks the next word

Heads upThere's no thinking, no checking facts, no understanding in the human sense. It's pattern-completion, done at a scale that makes the pattern completion look like reasoning.

03Where does the "knowledge" come from?

Training. Once. A long time before you ever chat with it.

The model reads an enormous pile of text (trillions of words: books, code, websites, articles) and plays one repeated game: cover up the next word, guess it, get told the right answer, adjust slightly to guess better next time. Do that billions of times and the model ends up with a huge internal sense of what word tends to follow what — at scale, that sense encodes grammar, facts, coding patterns, and reasoning-shaped habits.

That's it settled after training. Talking to it later doesn't teach it anything new or look anything up. It's running your question through the same frozen sense of what comes next that got built once.

Learning happens before the cutoff. After it, the model only gets what lands in the conversation.

CutoffThat's also why it has a knowledge cutoff and can't know anything after that, unless a tool (search, a fetched webpage) hands it fresh text directly in the conversation.

04Why does it sometimes just make stuff up?

Because guessing a plausible-sounding next word and guessing a true next word are literally the same operation to the model. There's no separate "I actually know this" switch. If something is a very common, confident-sounding pattern in its training, it'll say it just as confidently whether it's true or not. Hallucination isn't a glitch, it's the normal mechanism, just without a truth-checker attached.

Some newer training tries to teach it to sound less sure when it's actually less sure. Helps a bit. Doesn't fix the underlying issue.

The model sounds equally sure whether the pattern is true or not.

05Training has two very different phases

Two very different jobs, in order.

Phase 1, learn language by brute force. Read a giant pile of internet text, guess-the-next-word millions of times. This makes it fluent and knowledgeable, but not helpful. A raw model like this will happily respond to your question with another question, because that's a normal pattern in raw text.

Phase 2, turn it into an assistant. Two steps:

Show it a bunch of examples of "good assistant" conversations, so it learns the shape of a helpful reply.
Have it generate several possible answers to the same prompt, score which ones are actually better (correct code, correct math, or just "a human or another AI liked this one better"), and nudge it toward producing more of the good ones and fewer of the bad ones. Repeated a huge number of times.

That second step is also where "safety" gets trained in: bad-behavior answers just get marked as bad answers, same mechanism.

Fun factOne experiment (DeepSeek-R1) skipped step 1 entirely and just rewarded a model for getting the final answer right. The model taught itself to "think out loud" step by step first, purely because that strategy worked better. Nobody told it to do that.

06"Thinking" / reasoning mode, what is it really?

Same next-word-guessing loop. The only difference: the model was trained to write out a scratchpad of reasoning before giving its final answer, instead of jumping straight there. More words written before the answer means more chances to catch its own mistake, which means better results on hard problems like math and code. That's why "thinking longer" measurably helps. It's not doing anything fundamentally different — it's just giving itself more room to work the problem out in text first.

Same guessing loop, with room to work the problem out in text first.

07Why it forgets everything the moment you close the tab

There's no memory being saved anywhere inside the model. Every single reply is generated by re-reading the entire visible conversation from scratch: your messages, its own previous replies, all of it, every time. Close the chat and it's gone. Permanently.

Each reply starts from the full transcript. Nothing is stored in the model.

Anything that looks like the AI "remembering you" (ChatGPT's memory, Claude Code's memory files) is a bolted-on trick: some other piece of software writes a summary to a file, and quietly stuffs that summary back into the conversation next time, pretending it's memory. The model itself never learns a thing from talking to you.

Practical upshotThe model only ever knows two things, whatever got baked into it during training, and whatever is literally sitting in front of it in the current conversation right now. Nothing else exists to it.

08Where do web search, skills, and local files fit in, then?

None of it is training, and none of it changes the model's weights. It's all the same trick: get more relevant text into the context window before the model starts guessing the next word.

The model only reads and writes text. The harness does everything else.

Web search / fetching a pagethe harness runs the search, grabs the result text, and pastes it into the conversation as if you'd typed it in yourself. The model then does its normal "guess the next word" thing, except now the answer is sitting right there in its input.
Skillsa skill is a pre-written block of instructions on disk. When one triggers, the harness reads that file and drops its contents into context, same as pasting in a very detailed instruction sheet before you ask your question. The model isn't "loading a skill" the way software loads a plugin, it's just been handed more text to read before it answers.
Local files / project context (CLAUDE.md, memory files, a codebase)identical mechanism. Reading a file and putting its contents in context is the only way the model ever "sees" it. There's no separate channel; it's all just more tokens in the same input sequence.
Agents and tool callswhen the model "calls a tool," it's actually just outputting text in a specific format that the harness recognizes as a command. The harness runs the real action (search the web, edit a file, run a script) outside the model entirely, then feeds the result back in as more context. The model never touches your filesystem or the internet directly, it only ever produces text, and the surrounding software decides what to do with that text.

The pattern is always the same: training gives the model its baseline knowledge and behavior, frozen. Everything that looks like "live" knowledge or "using a tool" is the harness injecting fresh text into the one thing the model actually reads: the current context window.

09A few more things worth knowing, briefly

It doesn't see words, it sees "tokens,"chunks of a word, roughly. Common words are one chunk; weird words get chopped into pieces. This is why it's bad at spelling and counting-letter tasks: it's not looking at letters at all.
Longer conversations cost more and get slightly worse.Every reply reprocesses the whole conversation so far, so a long chat is slower and pricier. Stuff buried in the middle of a huge wall of text also tends to get less "attention" than stuff near the end, so dumping your entire codebase in isn't automatically better than giving it just the relevant bits.
Answers aren't always identical for the same question,because there's a small dial of randomness in how it picks the next word, called temperature. Turn that dial to zero and it gets much more repeatable.
Modern big models don't use their whole brain for every word.They're built from many smaller specialist sub-networks, and only a handful get switched on per word, called "mixture of experts." Lets a model be huge without being huge-slow.

It's a next-word guessing machine, extremely well trained twice over: once to be fluent, once to be helpful. It has no memory, no database, no "I don't know" button that always works. Every weird behavior you'll run into, confident wrong answers, getting worse in giant conversations, answers changing between runs, writing fluent code for a library that doesn't exist, comes directly from that mechanism.

AAppendix: the lingo, quickly

Where each term in the list below sits.

AI a broad umbrella term for software that does tasks normally needing human judgment. Everything below is one specific kind of AI.
LLM (large language model) the actual model discussed in this whole doc, the next-word guesser trained on huge amounts of text.
Token the chunk of text (roughly a word or word-piece) the model actually reads and generates, one at a time.
Model weights the huge set of numbers tuned during training. This is "the model." Nothing else is stored anywhere.
Context window everything the model can currently see in one go: your messages, its own replies, any files or search results pasted in. Its only source of "live" information. Deliberately curating what goes in here is its own discipline, see context engineering below.
Prompt the text you (or a system) send the model to get a response.
Prompt engineering optimizing the literal wording of a prompt (word choice, examples, output format) for a single call to the model. Stands on its own, doesn't require a harness, an agent, or any wrapper software, it's the same discipline whether you're calling the API directly or typing into a chat box. Deep dive: prompt-engineering.md.
Context engineering deciding what goes into the context window in the first place, not just how it's worded, minimal tool sets, just-in-time retrieval instead of front-loading everything, note-taking as durable memory. Deep dive: harness-engineering-vocabulary.md.
Training / pretraining the one-time process of tuning the weights on a giant pile of text. Happens long before you ever chat with it.
Fine-tuning / post-training further training on top of a pretrained model, usually to make it behave more like a helpful assistant rather than raw autocomplete.
Inference the actual act of running the model to generate a response. This is what happens every time you send a message; no weights change during this.
Hallucination a confident, fluent, wrong answer. Not a malfunction, just the normal guessing mechanism landing on something false.
Harness the software wrapped around the model (Claude Code, ChatGPT's app, an IDE plugin) that manages the conversation, runs tools, reads files, and decides what text to feed the model next. The model itself has none of this; the harness builds it. Deep dive: harness-engineering-vocabulary.md.
Tool call the model outputting a specific, structured piece of text that the harness recognizes as "run this action" (search the web, edit a file, run a command). The model requests it; the harness executes it and feeds the result back.
Agent a harness set up to run the model in a loop, letting it call tools, see results, and decide the next step on its own, repeatedly, until the task looks done. See subagents-and-delegation.md for how multiple agents split work.
Skill a pre-written instruction file the harness can load into context when a task matches, so the model gets detailed guidance without you typing it out every time. Deep dive: skill-libraries-and-marketplaces.md.
Loop running the same prompt or task repeatedly (on a timer, or self-paced), typically to check on something or make incremental progress over multiple turns. Building these deliberately (external progress files, explicit done-conditions instead of trusting self-report) is sometimes called loop engineering. Deep dive: loop-engineering-agent-loops.md.
Graph engineering coordinating multiple agents or loops as an explicit network of nodes (agents, steps) and edges (handoffs, conditions), instead of one agent looping alone, plus a separate sense of the term for storing knowledge as explicit linked relationships rather than flat searchable text. Works independently of any specific harness underneath. Newer, less settled term. Deep dive: graph-engineering-multi-agent-coordination.md.
RAG (retrieval-augmented generation) the formal name for "fetch relevant text from somewhere and paste it into context before answering." Web search and file lookups in this doc are both examples of RAG in practice. See dev-scoped-second-brain-rag.md for RAG over your own codebase/notes specifically.
MCP (Model Context Protocol) a standard way for a harness to connect a model to live external tools/data sources without custom integration code per tool. Deep dive: mcp-model-context-protocol.md.

↗Sources

Attention Is All You Need (Vaswani et al., 2017): the original transformer/self-attention paper.
TensorOps, LLM Mixture of Experts Explained: A 2026 Field Guide: MoE mechanics and 2026 frontier model parameter counts (DeepSeek-V4, Kimi K2, Qwen3, Mistral Large 3, Llama 4 Maverick).
Jiaxin Zhang, How Frontier Labs Train Large Language Models (2026): the pretraining to mid-training to SFT to RL (GRPO-family) to alignment pipeline, DeepSeek-R1's pure-RL reasoning emergence, calibration and safety-gating in RL reward design.