Serving nearly a petabyte of Git data per day became a hard operational reality for OpenAI once autonomous coding workloads surged across its core repositories. The deployment of Codex reshaped the volume and velocity of internal development, creating high-concurrency demands that legacy source control architectures could not handle. When engineering organizations introduce autonomous agents into the software delivery loop, source control systems face a massive flood of concurrent branch updates and read operations.
OpenAI responded to this operational bottleneck by building Pando, a custom backend for Git repositories designed specifically to sustain agent-driven workloads. The system cut median push latency in production by nearly 5x while delivering approximately a petabyte of Git data daily. Pando achieved this throughput without breaking compatibility with existing Git clients or disrupting active development pipelines.
Monorepo Limits Under Agent Concurrency
Traditional source control systems stem from an era when developers exchanged kernel patches over email, an architecture designed around human rhythms rather than continuous machine execution. Over the past year, the scale of development driven by Codex generated simultaneous pushes, automated branch merges, and continuous fetches from research environments and testing pipelines. The cumulative overhead caused severe friction across monorepo infrastructure.
"We suffered slow fetches and pushes along with failed merges, with reliability at times falling below 90%."
This reliability drop revealed that standard Git server implementations were not equipped for thousands of concurrent machine actions. Did scaling agentic development mean abandoning the standard Git protocol entirely?
Decoupled Branches and Shared Packfiles
OpenAI avoided breaking existing workflows by retaining the Git protocol while completely redesigning the storage and synchronization layer. A Git repository fundamentally records file snapshots and history as a graph of immutable objects—blobs for file contents, trees for directory structures, commits pointing to root trees and parents, and tags. Branches and tags serve as references that point directly to specific commits. Git stores these elements either as loose files or packed together in packfiles, which compress similar objects as deltas against one another.
"First, Pando coordinates updates to each branch independently. Updates to unrelated branches can proceed in parallel, and a reader fetching a branch gets the history needed for that revision."
To eliminate write contention, Pando isolates branch coordination so updates to unrelated branches execute concurrently without locking the repository. For high-volume read traffic, Pando reuses existing packfiles across fetches rather than calculating bespoke compressed archives for every individual client. While clients might occasionally download redundant objects, the server conserves significant compute resources and maximizes network throughput.
Operational Sustainability of Agent Infrastructure
Pando now powers all active development within OpenAI's monorepo, proving that underlying developer tooling must adapt to the physical constraints of autonomous code generation. Organizations deploying continuous agentic workflows will inevitably run into similar throughput walls across their version control backends. Maintaining high developer and machine velocity requires investing in custom data distribution layers before infrastructure reliability degrades.
The catch
Nearly a petabyte of daily mono-repository traffic exposed how standard engineering tooling breaks under the relentless pace of autonomous systems. To keep latency down, the team accepted a clear trade-off: serving redundant objects to clients to save backend compute at the expense of network bandwidth. While the custom Pando storage layer resolved write locks, it remains a patchwork fix over a version control architecture originally engineered for slow human patch exchanges. Can optimizing legacy protocols sustain autonomous workloads when branch traffic scales further, or will the industry have to reinvent version control from scratch?
Our stories: GPT-5 Codex: Automating Technical Debt and Software Engineering (April 19).