Inside the Codex Agent Loop — How OpenAI's Coding Agent Actually Works

A Rare Look Inside the Machine

OpenAI engineer Michael Bolin has published the first in a series of technical deep-dives into how Codex CLI works — a level of transparency that's unusual even by open-source standards. The focus of this first post: the agent loop, the core logic responsible for orchestrating every interaction between user, model, and tools.

Codex CLI is OpenAI's cross-platform local software agent, designed to produce high-quality, reliable software changes while operating safely and efficiently on your machine. It launched in April 2025, and the team has since accumulated hard-earned lessons worth sharing.

The Loop, Explained

At its core, the agent loop is a five-phase cycle that repeats until the task is done:

• Prompt assembly: Build the prompt from instructions, tool definitions, environment context, and user input.
• Inference: Send the prompt to the model via the Responses API and receive either a final answer or a tool call request.
• Tool invocation: If the model requests a tool (e.g., run a shell command, read a file), execute it.
• Tool response: Feed the result back into the prompt and loop again.
• Assistant message: When no more tool calls are needed, return the final response to the user.

A single "turn" — one message from the user — can involve hundreds of these inference-tool iterations before producing a final response. The loop ends only when the model decides it's done.

The Quadratic Problem No One Talks About

Here's the uncomfortable truth about multi-turn AI agents: every new turn must include the full conversation history in the prompt. As the conversation grows, so does every request to the API. Left unchecked, this creates quadratic growth in the amount of data sent per conversation — meaning a long session sends exponentially more tokens than a short one.

OpenAI solves this with two mechanisms:

• Prompt caching: When a new prompt shares an exact prefix with a previous inference call, the API reuses intermediate computations instead of reprocessing everything. This converts quadratic performance back to linear. The trick: always put static content (instructions, tool definitions, sandbox config) at the beginning of the prompt, and variable content (user messages) at the end. Any change to early prompt segments triggers an expensive cache miss.
• Conversation compaction: When token usage hits a threshold, Codex automatically calls a specialized API endpoint that compresses the conversation history into a condensed summary, replacing the full history with a lighter representation while preserving context coherence.

Stateless by Design — and Why That Matters

Every Codex request ships the entire conversation history to the API rather than referencing a stored session ID. This is deliberate: stateless requests simplify the API provider's infrastructure and — crucially — support Zero Data Retention (ZDR) customers who require OpenAI to store no user data server-side.

ZDR customers don't lose access to prior turns either: encrypted reasoning content from previous turns can be decrypted server-side using the customer's own decryption key, which OpenAI persists separately from conversation data. It's an elegant solution to a hard compliance requirement.

Lessons Learned the Hard Way

Bolin doesn't shy away from the bugs. One early issue: Codex CLI's initial MCP (Model Context Protocol) support failed to enumerate tools in a consistent order, which silently invalidated the prompt cache on every call — destroying performance without any obvious error. The fix was straightforward once diagnosed, but finding it required careful tracing of cache behavior in production.

The openness here is notable. OpenAI confirmed it uses Codex internally to develop Codex itself — and still publishes the failure modes. That kind of honesty builds more confidence than polished marketing ever could.

The Bigger Picture

What Bolin's writeup makes clear is that a coding agent isn't magic — it's a loop with very carefully engineered guardrails. The magic comes from the model quality inside the loop, the context management around it, and the sandboxing that keeps it safe. Future posts in the series will cover tool integration, infrastructure architecture, and Codex's sandboxing model. For anyone building on the Responses API, the lessons here apply far beyond Codex itself.