In June 2025, Andrej Karpathy, OpenAI co-founder and former Tesla AI director, reframed how developers talk about building with AI: “context engineering is the delicate art and science of filling the context window with just the right information for the next step.” More than a year on, that’s become the closest thing the industry has to a working definition of the actual hard part of shipping AI agents.
This guide covers what it means, why it’s unavoidable, and the components it breaks down into.
What It Is, and Why It's Needed At All
Context engineering is deciding what an AI model sees at each step of a task — and what it doesn’t — so it reasons and acts reliably.
Anthropic’s engineering team frames the goal precisely: “finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome.” That’s narrow on purpose.
Large language models are stateless — every call replays the entire conversation from scratch — and every token in that replay competes for the same limited attention. Add more, and you don’t just spend tokens, you dilute focus on what’s already there. A Stanford-led study, ‘Lost in the Middle,’ found models reliably underuse information buried in the middle of a long context.
Context Engineering Vs. Prompt Engineering
Prompt engineering is the instruction — how you phrase one request. Context engineering is everything else the model sees: system instructions, tools, retrieved documents, history, memory. A better-phrased prompt can fix a single answer. It does nothing for an agent ten steps into a task, working from a history full of stale information and its own earlier mistakes. That’s a context problem, and no amount of instruction-tuning solves it.
Why Developers Should Actually Care
Lightrun’s 2026 State of AI-Powered Engineering Report (200 senior SRE/DevOps leaders) found 43% of AI-generated code changes still need debugging in production, and 97% said their AI SREs operate without significant visibility into production.
As Lightrun’s Or Maimon put it: “AI coding agents cannot see how their code behaves in running environments.” That’s not a reasoning failure — it’s a model confidently acting on an incomplete picture, because nobody engineered what it was allowed to see.
The Building Blocks of Context Engineering
| COMPONENT | WHAT IT IS | WHAT DEVELOPERS SHOULD ACTUALLY DO |
|---|---|---|
| System prompt | Standing rules and role definition, read fresh every turn. | Prune it on a schedule — Anthropic cut roughly 80% of Claude Code's own system prompt for a newer model and it lost nothing. |
| Tools | The defined actions and data sources the agent can call. | Keep the tool count small; merge or cut any two tools that could plausibly handle the same request. |
| Examples | A handful of sample interactions that demonstrate the behavior you want. | Curate three to five canonical examples instead of exhaustive rules, and retire ones that no longer match current behavior. |
| Retrieved context | Information pulled in upfront or just-in-time via a lightweight reference. | Default to just-in-time retrieval; reserve upfront loading for what's genuinely always relevant. |
| Conversation history | The running record so far — the component most prone to pollution. | When an agent repeats a failed fix, roll back to before it went wrong instead of letting it argue with its own mistake. |
| Memory / notes | Information the agent writes outside the window and reloads later. | Give long-running agents a persistent scratch file so they aren't re-deriving conclusions every reset. |
| Compaction | A periodic summary that replaces aging history once the window fills up. | Trigger it manually at a natural breakpoint — letting it fire mid-task loses the thread. |
| Sub-agent output | A condensed result from a specialized agent that explored in its own window. | Spawn one for open-ended exploration and require a short, distilled summary back, not its full working notes. |
Miss any one of these and the rest can’t compensate for it.
Both agents can reason equally well. Only one of them was given something worth reasoning about — the difference is entirely in what happens between the data and the context window, not in the model itself.
Why This Stopped Being a Developer-Only Problem
The moment AI systems moved from answering questions to taking actions — updating a record, triggering a workflow, approving a request — a context mistake stopped being a minor inconvenience and became an operational risk. An agent that gets one thing wrong acts on it, and every step after inherits the mistake as fact. Business teams adopting agentic AI are now running headfirst into the same problem developers have been fighting for over a year: an AI system that sounds certain is not the same thing as one that’s right.
Good agents aren't built on better models. They're built on better context.
Kenexai’s AgentWorks helps you build, deploy, and govern AI agents your business can actually trust.
Context Engineering FAQs:
No — RAG is one retrieval technique; context engineering is the broader discipline it belongs to, alongside tool design, memory, and compaction.
Usually not as a title. It’s shared between the developers building an agent and the data team maintaining what it draws from.
An open standard letting AI tools connect to external data through one consistent interface instead of a custom integration per tool.
Because a strong model reasoning over stale or ungoverned information still acts confidently and wrongly — the failure is almost always what it was given, not its reasoning.
Mainly through Horizon Context (governed definitions with access control and lineage) and managed MCP servers (a secure, standard connection for outside AI tools).
The bottom line
Context engineering isn’t a rebrand of prompt engineering — it’s data governance and software engineering meeting at the same point. The model was never the bottleneck. What it’s allowed to see, trust, and act on always was.
