Google Cloud Targets AI Storage Bottlenecks With Purpose-Built Infrastructure to Improve Accelerator ROI
Google Cloud is expanding its AI storage infrastructure with Cloud Storage Rapid and Managed Lustre, aiming to reduce data bottlenecks, improve GPU utilisation and help enterprises maximise returns on expensive AI accelerators.
Xcademia Team

As enterprises invest in increasingly powerful AI accelerators, the ability to keep GPUs and other computing resources continuously occupied is becoming a major infrastructure challenge.
Google Cloud is highlighting a problem that can limit the return on investment from AI hardware: insufficient storage performance.
In a blog post published on 16 November 2026, Google Cloud explained how traditional storage architectures can create input/output (I/O) bottlenecks that prevent AI accelerators from accessing data quickly enough. When GPUs have to wait for datasets, model files or checkpoint operations, valuable computing capacity remains idle.
The company is addressing these challenges through purpose-built storage technologies, including
Cloud Storage Rapid, Rapid Cache and Google Cloud Managed Lustre.
The broader objective is to align storage performance with the requirements of modern AI workloads, particularly large-scale model training, real-time inference and increasingly complex agentic AI systems.
According to Google Cloud, AI infrastructure planning must move beyond simply purchasing more accelerators. Organisations also need to ensure that their storage and networking systems can supply data at the speed their computing infrastructure requires.
Why Storage Performance Has Become an AI Infrastructure Challenge
AI infrastructure depends on more than processing power.
Training and inference systems continuously move large volumes of information between storage systems, memory, networking infrastructure and accelerators.
When storage cannot deliver data quickly enough, GPUs and TPUs may remain underutilised even when organisations have invested heavily in computing hardware.
Google Cloud identifies insufficient storage throughput as a significant obstacle to AI infrastructure performance.
The issue becomes particularly important as enterprises expand their AI deployments and introduce more complex workloads.
The company also references Gartner's prediction that enterprises will deploy more than 150,000 agents in the coming years. Google argues that this growth will increase the demands on storage systems and data orchestration.
For organisations operating large AI environments, the challenge is therefore not simply acquiring additional hardware. It is ensuring that the entire infrastructure operates efficiently.
AI Training and Inference Require Different Storage Architectures
One of the central points in Google's announcement is that AI should not be treated as a single, general-purpose workload.
Different stages of the AI lifecycle create different data access requirements. Storage infrastructure must account for these differences.
AI model training: Managing high-throughput data and checkpoints
During model training, GPU clusters repeatedly access large datasets.
They also need to save checkpoints, which preserve the model's current state so that training can resume if a failure occurs.
Google Cloud explains that checkpoint operations can create significant pressure on conventional storage systems.
When thousands of GPUs attempt to save checkpoint data simultaneously, storage architectures may struggle to process the resulting requests.
Traditional cloud storage systems also perform administrative database transactions to maintain file records. During large checkpoint operations, these activities can contribute to delays and network congestion.
The consequences include:
Delayed checkpoint operations.
Increased waiting time for GPU clusters.
Slower recovery after infrastructure failures.
Reduced utilisation of expensive computing resources.
Google also highlights that slow checkpoint recovery can turn what should be a routine restart into a delay lasting several hours.
AI inference: Reducing latency and model loading delays
Inference workloads have different requirements.
Unlike training, inference involves responding to user requests in real time. AI services must load models, retrieve relevant information and process requests with minimal delay.
As inference deployments expand, storage systems must support rapid access to large model repositories.
Google identifies two important challenges:
Cold starts: GPUs may remain idle while large model files are loaded from storage.
Context management: Models processing context windows exceeding 100,000 tokens can consume significant accelerator memory.
Without an appropriate external caching mechanism, systems may need to recalculate previously processed inputs, resulting in redundant computing operations.
This makes storage latency and external caching important considerations when designing large-scale inference infrastructure.

Google Cloud Introduces Purpose-Built Storage Solutions for AI Workloads
To address these infrastructure challenges, Google Cloud is focusing on integrating compute, networking and storage more closely.
The company says its storage infrastructure is built on Colossus, Google's distributed file system, which also supports services such as Google Search, YouTube and Gemini.
Google states that its storage architecture has been developed in collaboration with frontier AI laboratories and its internal foundation model teams over the past two years.
It highlights three key storage technologies.
Cloud Storage Rapid
Designed for organisations using cloud-native object storage, Cloud Storage Rapid provides high-performance access to data stored in object storage for AI workloads.
According to Google, the service allows organisations to use existing object storage data without migrating it into separate specialised storage systems.
Google reports the following capabilities for a single Rapid bucket:
Up to 15 TB/s throughput.
Up to 20 million queries per second.
Sub-millisecond latency.
These capabilities are intended to support large-scale training datasets and checkpoint recovery operations.
Rapid Cache
Designed to accelerate read-intensive workloads by placing active data closer to TPU and GPU clusters.
Rapid Cache addresses the problem of slow model loading and cold starts.
Google says the service can deliver up to 2.5 TB/s of aggregate read throughput without requiring code changes.
The company also reports that it can reduce cold-start waiting periods from hours to minutes.
This is particularly relevant for AI environments where models must be loaded quickly across multiple locations.
Google Cloud Managed Lustre
A managed parallel file system designed for demanding AI and high-performance computing workloads.
Developed in partnership with DDN, Managed Lustre is designed to handle intensive read-and-write operations.
Google reports up to 10 TB/s throughput per instance.
The service targets workloads such as:
Large-scale model training.
Synchronous checkpoint operations.
Reinforcement learning.
High-performance computing workloads.
Google has also introduced two additional capabilities within its Managed Lustre offering:
Dynamic Tier: Consolidates frequently accessed and less frequently accessed data into a single namespace. Google says this is intended to reduce the performance limitations associated with traditional caching approaches.
Managed Lustre Cloud Storage Mirror: Automatically synchronises datasets in the background, reducing the need for manually written data-transfer scripts and helping ensure that datasets are ready before training begins.
The company did not provide specific pricing information in this announcement.

Google Cloud Highlights Enterprise Deployments and Customer Results
Google Cloud says its purpose-built storage infrastructure is already supporting demanding AI workloads across several organisations.
The announcement provides examples from five companies, highlighting improvements in throughput, latency, cache performance, recovery times and GPU utilisation.
Here are the key deployments described by Google.
Thinking Machines Lab
Thinking Machines Lab uses Cloud Storage Rapid Cache to manage demanding data-access workloads.
Google reports that the deployment achieved:
Read throughput peaks above 1.8 TB/s.
An 89% cache hit ratio.
Elimination of rate-limiting surge errors across its multi-region architecture.
These results demonstrate how caching can help maintain consistent data delivery during demanding AI workloads.
Shopify
Shopify deployed more than 50 unique Rapid Caches across its global infrastructure.
According to Google, the deployment achieved:
Peak throughput exceeding 2 TB/s.
Up to 90% latency reduction compared with its baseline.
The deployment addresses the data requirements associated with Shopify's expanding computing infrastructure.
Salesforce AI Research
Salesforce AI Research uses Managed Lustre as an external cache for demanding inference workloads.
Google describes the deployment as helping keep GPUs fully utilised while supporting complex large language model inference.
The announcement does not provide a specific numerical performance improvement for this deployment.
Atlassian
Atlassian adopted Managed Lustre on Google Kubernetes Engine (GKE).
Google reports that this implementation helped the company:
Restore interrupted workloads in five minutes following preemption events.
Achieve nearly five times higher throughput.
Improve GPU utilisation by 70% compared with NVMe-based systems.
The deployment illustrates the importance of storage performance and recovery processes in environments running AI workloads on Kubernetes.
Harmonic
Harmonic uses Managed Lustre to train mathematical reasoning models.
Google reports three outcomes:
50% fewer experiment interruptions.
Startup times reduced by half.
Data retrieval speeds increased by 15 times.
These results highlight how storage infrastructure can influence the efficiency of repeated AI training experiments.
All customer performance figures above are reported by Google Cloud in its announcement. They should be understood in the context of the individual deployments, not as universal performance guarantees.

Google's Recommendations for Building AI Infrastructure
Beyond introducing storage capabilities, Google Cloud outlines several recommendations for organisations designing AI infrastructure.
Evaluate token economics instead of focusing only on storage metrics
Organisations should examine how infrastructure performance affects the cost of delivering AI outputs.
Traditional storage measurements may not fully explain the financial impact of GPU idle time.
Google recommends evaluating token economics and identifying whether storage delays are reducing the effective utilisation of computing resources.
Avoid creating unnecessary copies of datasets
Google also recommends maintaining a unified data foundation rather than creating separate copies for every new AI workload.
Moving data into appropriate storage tiers when required can help reduce unnecessary data duplication and infrastructure complexity.
Prioritise reliability and cost management
AI infrastructure must support more than raw performance.
Google emphasises the importance of enterprise-grade reliability, built-in security and automated cost discipline when designing infrastructure for AI workloads.
The company argues that storage, networking and computing should be designed as interconnected parts of the same architecture rather than independent infrastructure investments.
What Google's Announcement Means for the Enterprise AI Infrastructure Market
Industry analysis
Google's announcement highlights a broader industry shift towards infrastructure optimisation as AI deployments become more demanding.
The early stages of enterprise AI investment often focus on acquiring GPUs, expanding computing capacity and deploying larger models. However, the performance of these systems also depends on how efficiently data moves between storage and compute resources.
This creates several important considerations for enterprise technology teams.
Storage is becoming a strategic AI investment.
Organisations may need to reconsider whether their existing storage architecture can support large-scale training, inference and agentic workloads.
Infrastructure decisions increasingly influence AI economics.
When expensive accelerators remain idle because of storage delays, organisations may not achieve the expected return from their hardware investments.
Different AI workloads require different infrastructure approaches.
Training, inference and reinforcement learning have different data access patterns. A single general-purpose storage architecture may not be suitable for every workload.
Data management and orchestration are becoming more important.
As organisations deploy more AI agents and distributed workloads, maintaining efficient data access across infrastructure environments becomes increasingly relevant.
For enterprises, this could mean that future AI infrastructure assessments will need to evaluate storage throughput, latency, checkpoint recovery, caching and GPU utilisation alongside accelerator specifications.
These are industry implications drawn from the announcement, rather than additional product commitments made by Google.
Source: Google Cloud Blog
About the Author