Context Engineering Is the Real Skill Now (Prompt Wording Is Dead)
Clever prompt phrasing stopped mattering as models got better at intent. What decides an agent's output now is the context you assemble around it. Here is how to engineer that working set.
Two engineers give the same coding agent the same task. One types a tidy, polite request. The other types a blunt one with a typo in it. Same model, same temperature. The outputs come back nearly identical, and both are fine. Run that experiment a hundred times and the polishing of the words stops correlating with anything you can measure.
Now change one variable. One engineer’s agent has the right three files open, the failing test in view, and a short note about how this module is supposed to behave. The other’s agent is staring at forty files, a stale README, and a half-finished conversation from an hour ago. The words in the prompt are identical this time. The outputs are not even close.
That is the whole argument in one image. The prompt wording is no longer the lever. The context is.
What actually changed
For a couple of years, prompt engineering was a real edge. You could coax a meaningfully better answer out of a model by finding the magic framing: “you are an expert,” “think step by step,” “take a deep breath.” Those tricks worked because the models were weak at inferring intent. They needed you to hand-hold the shape of the reasoning.
That gap has mostly closed. A current frontier model reads a plain, direct instruction and figures out what you meant. It infers that you want the edge cases handled without you listing them. It picks a reasonable structure without a template. The phrasing lottery is over because the model stopped needing the ritual.
What did not get easier is deciding what the model gets to see. A model can only reason over what is in its context window at the moment it generates. Everything it knows about your specific problem, your codebase, the last error, the decision you made yesterday, has to be physically present in that window or it does not exist for that call. Assembling the right working set is the job now. That assembly has a name, and the name is context engineering.
“Prompt engineering was about wording the question. Context engineering is about controlling what the model can see when it answers. The second one is where all the leverage moved.
”
The context window is a scarce, expensive working set
The word “window” makes it sound generous. Think of it instead as a small desk that the model has to work at. Everything relevant to the task has to fit on the desk. Anything not on the desk might as well not exist. And every time the model does one unit of work, you pay for everything currently on the desk, whether it looked at that paper or not.
This is the mental shift most people miss. The context window is not free memory you fill up because you can. It is a working set you curate, and it is scarce in three ways at once.
- It is finite. Even large windows fill faster than you think once an agent starts reading files and dumping tool output. A repo of any real size does not fit, so “just give it everything” is not an option even when it sounds like the safe one.
- It is billed on every call. An agent task is not one model call, it is hundreds. Each loop iteration re-sends the current desk. A bloated context is not a one-time cost, it is a tax you pay on every single step of the loop.
- It degrades with size. This is the part people underestimate. A model’s ability to actually use what is in the window is not flat across the window. Bury the one important line in the middle of a hundred thousand tokens of noise and the model’s odds of using it drop. A bigger context can produce a worse answer.
So the goal is not maximum context. The goal is the minimum sufficient context: the smallest working set that still contains everything the model needs, and as little as possible of what it does not.
The components of good context
If context is the working set, engineering it means deliberately choosing what goes in from a handful of sources. Five of them matter most. A strong agent setup is really just these five, curated well.
Retrieval: bring the relevant, not the repo
Retrieval is how you get the right slice of a large body of knowledge onto the small desk. Retrieval-augmented generation, RAG, is the common form: you index your documents or your codebase, and at query time you pull only the passages that match the current task and place them in context.
The failure mode here is treating retrieval as “dump everything vaguely related.” Good retrieval is closer to a librarian than a firehose. You want the three functions that this change touches, not the whole module. You want the one design doc section that governs this decision, not the entire wiki. For code specifically, structural retrieval (pull the definition of the symbol being called, the callers of the function being changed) beats naive text similarity, because the thing the model needs is often the thing the current file references, not the thing that shares the most words with it.
Memory: what should survive between calls
The model forgets everything the instant a call ends. Memory is the deliberate decision about what survives. There are two kinds worth separating. Short-term memory is the running state of the current task: what has been tried, what failed, the plan so far. Long-term memory is the durable stuff that should persist across sessions: project conventions, the fact that this codebase uses a particular test runner, the decision you made last week and do not want to relitigate.
The engineering move is to make memory explicit rather than hoping it stays in the conversation. Write the durable facts to a file the agent reads at the start of every session. A short, curated AGENTS.md or a project notes file does more for consistency than any amount of re-explaining, because it lands the same durable context on the desk every time without you retyping it.
Tool definitions: the model’s hands, described precisely
Tools are how an agent does anything beyond talking: read a file, run a test, query a database, hit an API. The definitions of those tools live in context too, and their quality matters more than people expect. A tool with a vague description and loose parameters produces vague, wrong calls. A tool with a crisp name, a one-line description of exactly when to use it, and tightly typed parameters produces clean calls.
Treat tool definitions like an API you are designing for a slightly literal colleague. Fewer, sharper tools beat a sprawling toolbox where three of them overlap and the model has to guess which to reach for. And when a tool runs inside a sandbox where the agent can execute code safely, the returned output is itself context you are choosing to admit, which is the next problem.
Tool results: the sneaky context inflator
Here is where most agent contexts quietly rot. Every file the agent reads, every test log it captures, every API response it pulls back, gets appended to the working set and stays there for the rest of the session. Read a 2,000-line file to check one function and all 2,000 lines are now sitting on the desk, being re-sent and re-billed on every subsequent call, drowning the signal.
The discipline is to summarize and evict. Pull the file, extract the answer, and drop the raw dump. Capture the test failure, keep the error, discard the passing noise. A well-run agent loop treats tool output as perishable: useful this turn, mostly garbage the next.
Few-shot examples and structured state: shape without instruction
The last two are the highest-leverage per token. A couple of well-chosen examples of the exact output you want will steer a model harder than three paragraphs describing that output. Show one correct function in your house style and the model matches it. This is the cheapest way to encode taste. Structured state, meanwhile, is the running plan or task list the agent reads and updates, often a literal plan.md or a checklist. It keeps a long task coherent, because instead of relying on the fuzzy memory of the conversation, the agent re-reads its own explicit state each loop and knows precisely where it is.
The failure modes: how context goes bad
Knowing the components is half of it. The other half is recognizing the four ways a working set turns toxic, because each one has a different fix.
Long agent sessions degrade in predictable ways. The context that was clean at turn five is a swamp by turn eighty, and the model’s output quality tracks that decay closely. Four named failure modes cover almost all of it.
Layer 2 · Mechanismhow it actually works
Context rot. As the window grows, the model’s ability to use any given fact in it declines. Retrieval accuracy is not uniform across a long context, so the important line buried at position sixty-thousand gets effectively ignored. Symptom: the agent “forgets” a constraint it was told, even though the constraint is technically still in the window.
Distraction. Irrelevant but plausible material pulls the model off course. A stale file, an unrelated code path, an old part of the conversation about a different bug. The model has no reliable way to know that the thing on the desk is no longer relevant, so it reasons over it anyway.
Contradiction. Two things in context disagree: the README says one thing, the code does another; an earlier turn made a decision that a later tool result invalidates. The model now has to guess which to trust, and it often guesses wrong or produces mush that tries to honor both.
Cost. The pure economic failure. Even when a bloated context still produces a right answer, you paid to re-send all of it on every call. A session that could have run on a lean 8k-token working set instead drags 120k tokens through hundreds of calls. The output is fine and the bill is not.
Layer 3 · Math & where it breaksgo deeper
Why size hurts both quality and cost at once:
per-call cost ≈ context_tokens × in_price + output_tokens × out_price
session cost ≈ Σ over calls ( per-call cost )
usable signal ≈ relevant_tokens / total_tokens_in_windowTwo things move together as you bloat the window. context_tokens rises, so every one of the hundreds of calls costs more. And usable signal, the share of the desk that is actually relevant, falls, so the model fights more noise for a worse answer. Lean context is the rare lever that cuts the bill and lifts quality in the same move, which is why it is the first thing to reach for.
You can stop after Layer 1 and still be correct about context failure modes, just less complete.
A method to engineer context for a task
Enough theory. Here is the loop I actually run when setting up an agent for a non-trivial task. It is deliberately boring, because reliability comes from boring.
Write the durable context down, once
Before the first prompt, put the stable facts in a file the agent reads every session: the stack, the test command, the conventions, the two or three architectural decisions that are settled. This is long-term memory made explicit. You are refusing to re-explain the project on every run, and you are landing identical durable context on the desk each time so behavior stops drifting session to session.
Retrieve narrow, not wide
For the specific task, bring in only the files and passages that this change actually touches. Prefer structural retrieval over keyword similarity: the definition being called, the callers being affected, the one governing doc section. If you catch yourself pasting a whole directory in “to be safe,” stop. Wide context is the distraction failure mode waiting to happen.
Seed with an example, not a lecture
If the output has a shape (a component, a test, a migration), drop in one correct example of that shape in your house style. One good few-shot beats a paragraph of description and costs less to keep on the desk.
Give the agent explicit state to update
Have it maintain a plan or checklist it re-reads each loop. Structured state keeps a long task coherent and lets the agent recover its place after a detour, instead of relying on the fuzzy memory of a growing transcript.
Prune tool output aggressively
Treat every file read and test log as perishable. Summarize what matters, evict the raw dump, and clear the conversation between unrelated subtasks. If you run agents in a sandbox that executes code, this is where most of the bloat enters, so this is where most of the discipline pays off.
Watch the window, not the wording
When output degrades, resist the urge to rewrite the prompt. Look at the desk instead. Is there a contradiction between the doc and the code in context? A stale file distracting it? A buried constraint the model has stopped seeing? Nine times out of ten the fix is a change to the working set, not the phrasing.
Why this is the durable skill
Models will keep getting better at understanding your words. That is exactly why fussing over the words is a dead end: you are optimizing the thing the labs are automating away. What no model can do for you is decide which of your ten thousand files, which past decision, which example, which slice of state belongs on the desk for this specific task. That judgment is domain knowledge plus engineering, and it does not transfer to the model no matter how capable it gets.
There is a reason the same discipline shows up whether you are running a single agent or a team of agents that loop on a task, or letting agents execute code in a sandbox where tool output floods back in. In every one of those setups, the thing that separates a system that works from one that thrashes is not the cleverness of any single prompt. It is whether the right context reached the model at the right moment, and nothing else came along for the ride.
The prompt was always a proxy. What you were really trying to control was what the model paid attention to. Now you can control that directly. That is the job.
