Storage for a Billion: How OpenAI Keeps ChatGPT's Infrastructure Alive

From Local Code to Global Storage

OpenAI detailed Habitat, its distributed storage platform. Originally a Python library, it now serves over a billion ChatGPT users. The system handles 22 million requests per second, functioning as a globally distributed layer on top of traditional cloud storage.

The raw data reveals that the main scaling barrier wasn't just GPU farms or inference, but the ability to feed them data and quickly save conversation states.

Why Isolated Storage Is the Bottleneck

For developers, this move demonstrates that training models is only half the battle. Once you need to hold long contexts and serve planetary traffic, classic databases fail. The architecture now depends on how close you can get storage to the compute node, rather than just the model's brute strength.

Companies building their own on-premise agents or local clusters will hit the same limit: inference is slow when storage can't deliver context fast enough.

Infrastructure Reality and Giant Cloud Bills

The report exposes why operational costs are so massive. It's not just electricity for inference chips. The whole ecosystem is now a robust cloud provider in itself. OpenAI has moved past the research lab phase, becoming a global infrastructure player facing hyperscaler-tier problems.

This also highlights the limits for potential local competitors aiming to run a service for millions without the backing (and wallet) of Microsoft.

The Speed of Deploying New Architectures Will Decide

The success of Habitat or its successors will show whether OpenAI can become a sustainable business before the money runs out. The proof will be whether they can keep the unit cost per query low while moving such massive blocks of context windows across data centers.

Lilith's verdict

Forget Python code. This is proof that AI operates as heavy-duty data freight where even a one-second outage means billions in losses.