RAG data path calculator
Retrieval sits between every prompt and the GPU. Price what a slow object-storage path costs in stalled accelerator time.
Every retrieval-augmented request makes a round trip through embeddings, a vector index, and object storage before the GPU can start. That data path is invisible in most AI dashboards — until you price the accelerator time spent waiting on it. This calculator connects storage latency and throughput to the number that matters: GPU-seconds stalled per day.
01: The retrieval workload
02: What the waiting costs
The fleet, minus the waiting
Blue is generating. Amber is the accelerator-time equivalent spent stalled behind the retrieval path.
- Generating
- Stalled on retrieval
More Info
RAG turns every prompt into a storage workload: embed the query, hit the vector index, then fetch the winning chunks from an object store before a single token generates. Object stores were built for durability, not for millisecond fan-out—connection setup, TLS handshakes, and serial fetching all add latency that lands directly in time to first token and, in blocking stacks, in idle accelerator time. The same path decides how fast training data streams and how fast checkpoints restore.
Frequently asked questions
From AI factory math to AI factory control
The classic fix is a delivery tier in front of the object store: connection pooling and reuse, TLS resumption, TCP optimization, and multi-tenant policy—the same discipline load balancers have applied to web backends for twenty years, pointed at S3-compatible storage.