A lot of people are confused about the difference between a model and an agent, and most aren’t really seeing what has happened. So let me give you a historical perspective on the evolution from language models to agents. There have been at least three big jumps: 2020 with GPT-3, 2022 with ChatGPT, and 2025 to 2026 with Claude Code. This is a distilled walkthrough of the whiteboard talk.
2020: language can be modeled
Before 2020, language models were small. Then OpenAI published “Language Models are Few-Shot Learners” and the whole point was in one line: they trained GPT-3, a model with 175 billion parameters, ten times larger than any previous dense language model. They scaled all three axes at once: the size of the model, the amount of data, and the compute used to compress that data into the model.
The mechanism is simple to picture. Take all the text you can find on the internet: Common Crawl, torrents, Wikipedia read many times over. Sample a sequence from that corpus. For each sequence you already know the next token (a token is like a word, but not exactly). You feed the previous text (the context, the prompt) into the network and ask it to predict that next token, then push the prediction to be as close as possible to the real one.
Why was the scientific community stunned? Because language can be modeled bottom-up, from data. There are no hand-written rules, no separation between syntax and semantics: it all mixes together, and you totally delegate the understanding and production of language to the model. You can also read the whole process as compression: the loss is a compression error, and intelligence starts to look like pattern recognition and data compression. That raises hard philosophical questions, but the deal was real. And “few-shot” means the model doesn’t need to be fine-tuned for each task: it can figure out what to do directly from the prompt.
2022: making it conversational and aligned
GPT-3 had two problems that made it unfit for the mass market. First, it was pure completion, not conversational: it didn’t understand turns. Second, trained on the open web, it was completely uncensored and unaligned. Ask early GPT-3 “how do I build a bomb?” and, instead of refusing, it would happily continue the text: “yes, we should really find out how to build a bomb, let’s go ask someone.” It was continuing text, not following instructions.
ChatGPT was a fine-tuning of GPT-3 to fix exactly those two things: respect conversational turns, and be aligned and censored in the specific way OpenAI decided. There was an intermediate model called InstructGPT for those who want to go deeper. Part of how they did it is uncomfortable and worth remembering: OpenAI used Kenyan workers paid less than two dollars an hour to write turn-based conversations with a censoring profile around them (“user asks for the bomb, AI refuses even when the user begs”), so the model could be trained on that shape. On top of the supervised step came reinforcement learning from human feedback: a separate model is trained to predict how much a human would appreciate a response, and the language model is optimized to produce sequences that this feedback model rates highly, rather than just the literal next token. Every other company did the same afterwards.
2025 to 2026: from model to agent
The jump from ChatGPT to Claude Code is the jump from a model to an agent, and it needs a bit of care.
Everything still bottoms out in a string. Whether you send a list of messages, retrieved documents, or tool descriptions, the vendor serializes all of it into one string, feeds it to the model, and gets tokens back one at a time in a loop until the reply is complete. That has not changed since GPT-3. What changes is what we put into the context before we hand it over.
RAG was the first thing resembling an agent. Retrieval-augmented generation: instead of sending the prompt straight to the model, you first query a knowledge base (a search engine, a database, a folder of PDFs), pull back relevant chunks, and add that retrieved context to the conversation. The model then answers over the enriched string. It was a big deal in the first year after ChatGPT, though it was already in the literature. This is around when I published the Cheshire Cat, version one. It is essentially a language model plus an additional memory system.
Tools are the real agentic step. Into the context you serialize a list of functions the model may use: get_weather(city), open the door, write a file, whatever can run on your computer, each with its arguments and a description. The model never calls a function directly. It produces text, a tool call, which is basically a JSON object with the tool name and arguments (get_weather, city=Rome). The runtime around the model parses that and actually executes the function. So the model decides which function to call and with what arguments; the software runs it and feeds the output back into the context.
Verifiable rewards are why coding agents got so strong so fast. We now train models encapsulated in software on entire rollouts, long sequences of tool calls, and we reward the whole sequence based on whether it solved the problem. For code we have a hard correctness signal: the unit test passes or it doesn’t, the math proof checks or it doesn’t. The reward is distant from any single token, but it is still the same old backpropagation underneath. This verifiable-reward strategy is the big jump between 2025 and 2026.
The agentic loop
Put it all together and here is what happens when you type into Claude Code, Codex, or any of these harnesses. First a context-building step (this is what people call context engineering): load the agents.md, load the skill descriptions, load the MCP servers and the tools they expose, plus the whole conversation so far. Pass that context to the model. Then ask one question: did the model output a tool call?
If yes, the runtime runs the tool (for coding agents these are Unix commands, glob, grep, ls, bash, git, plus anything you add via MCP servers). The tool’s output is appended to the context and the cycle restarts. Coding agents are strong partly because the models were trained to use exactly these Unix commands and were rewarded for it. If the model outputs no more tool calls, it means it has finished (wrote the files, ran the search, produced the report) and returns a final text synthesis to the user. All those dots you see streaming by are tools chosen by the model and executed by the runtime.
That cycle is the now-famous agentic loop. In the Cheshire Cat version two I made a deliberately simple implementation: a while True that calls the model with the system prompt, the messages, and the tools; appends the returned message; and exits the moment there are no more tool calls, otherwise executing the tools and looping again. It’s genuinely easy once you see it from the inside.
So, to recap the three stages: the foundational model, the conversational model, and the agentic embedded model. And here is the takeaway: it is not about the models anymore. Wake up, people. It is about the software around the models. If you tried these agentic coding products even four or five months ago, you have no idea how much they have improved.
