--- url: "https://www.xcademia.com/news/google-cloud-adds-day-0-support-for-reflection-beam-with-up-to-5x-higher-throughput" title: Google Cloud Adds Day 0 Support for Reflection Beam With Up to 5x Higher Throughput description: "Google Cloud adds day 0 support for Reflection Beam, with llm-d and NVIDIA Dynamo recipes reporting up to 5x higher throughput." publishedAt: "2026-10-19T10:30:01.326+00:00" updatedAt: "2026-10-19T23:50:42.333235+00:00" type: news category: "ai-ml" source_name: Google Cloud Blog source_url: "https://cloud.google.com/blog/topics/ai-infrastructure/day-0-support-with-5x-perf-for-reflection-beam-on-google-cloud" tags: - "#GoogleCloud" - "#ReflectionBeam" - "#AIInfrastructure" - "#AgenticAI" - "#LLMInference" - "#NVIDIADynamo" - "#llmd" - "#AIMachineLearning" --- # Google Cloud Adds Day 0 Support for Reflection Beam With Up to 5x Higher Throughput > Google Cloud has introduced day 0 deployment support for Reflection's Beam model, combining llm-d and NVIDIA Dynamo with infrastructure optimizations designed for multi-turn AI workloads. The company reports up to 5x higher throughput and 8x lower latency on specified hardware. Source: **Google Cloud Blog** · 19 October 2026 Google Cloud has announced day 0 support for Reflection's Beam, an open-weight model designed for software engineering and long-running AI agent tasks. The launch includes ready-to-run deployment recipes for Google Cloud AI Hypercomputer and NVIDIA AI infrastructure, using the llm-d and NVIDIA Dynamo serving stacks. The announcement focuses on a key challenge in deploying AI agents: improving inference efficiency requires more than an efficient model. Serving infrastructure must also handle repeated model calls, growing conversation histories, and large volumes of cached context without wasting GPU resources. Google Cloud says its optimized configurations can deliver up to 5x higher throughput and 8x lower latency on A4X Max instances. On G4 virtual machines, the company reports up to 2x higher throughput and 8x lower latency. These results are based on customer benchmarking, with each optimized configuration compared against a baseline running on the same hardware. ## Reflection Releases Beam With Open-Weight Checkpoints Beam is Reflection's first open-weight model. It targets software engineering, coding, and multi-step agentic workflows, giving developers, researchers, and enterprises the ability to use and customize the model under the Apache 2.0 license. According to Google Cloud, Beam has approximately 501 billion total parameters, with around 23 billion active parameters per token. The model also has an approximately one-million-token context window. The announcement states that Beam can solve reasoning and coding tasks using 3x to 4x fewer reasoning tokens than comparable open-weight models. Using fewer reasoning tokens can reduce the compute required during inference, although overall performance also depends on how the model is deployed and served. Beam is intended to work with existing coding environments and agent harnesses, including Reflection's Mirror CLI, Claude Code, Hermes, and OpenCode. The source describes use cases such as resolving GitHub issues, executing command-line and DevOps workflows, and completing tasks that require multiple steps. The model's BF16, FP8, and NVFP4 checkpoints are available through Reflection's Hugging Face page. ## Why Agentic AI Needs Serving-System Optimization AI agents often make repeated model calls within a single workflow. A coding agent, for example, may inspect files, propose code changes, run commands, process compiler output, and use those results to determine its next action. Each step can add more information to the conversation history. Over a long session, the accumulated context can grow to tens or hundreds of thousands of tokens. Even if a model uses fewer reasoning tokens, the serving system still has to process this expanding context efficiently. Google Cloud identifies two major infrastructure bottlenecks. ### 1. Repeated prompt processing increases compute demand In a conventional Kubernetes deployment, stateless round-robin routing can send consecutive requests from the same agent session to different model replicas. If the next replica does not have the previous request's cached context, it may need to process the historical prompt again. This repeats work that another replica may already have completed and can increase prefill processing demand. For long-running coding agents, repeated prompt processing can consume GPU capacity that could otherwise serve new or ongoing requests. ### 2. GPU memory limits can reduce cache reuse Serving systems use key-value (KV) caches to retain intermediate information from earlier processing. These caches help avoid recomputing the same context for subsequent requests. However, long-context sessions and concurrent users can consume substantial GPU high-bandwidth memory (HBM). When available memory is insufficient, cached blocks may be evicted. If a later request needs evicted context, the system may have to perform additional computation. Google Cloud's approach targets these issues through context-aware routing, cache offloading, and changes to how prompt processing and token generation are scheduled. ## Inside Google Cloud's Optimized Beam Serving Architecture Google Cloud's deployment recipes pair Beam with two serving options: - **llm-d with vLLM** - **NVIDIA Dynamo with SGLang** The architecture is designed around the repeated requests generated by multi-turn AI agents rather than treating each request as an isolated event. ### Prefix-aware routing and load management The routing layer tracks cached prompt prefixes and worker queue depth. When an agent submits a follow-up request, the router can direct it to a replica that already holds the relevant cached context. The architecture uses GKE alongside llm-d's Endpoint Picker (EPP) or the NVIDIA Dynamo router. Turn-priority flow control is intended to help manage queues and avoid overloading workers. This approach can reduce redundant prefill processing and make more effective use of existing cached context. ### Prefill and decode disaggregation Prefill and decode are two distinct stages of inference. Prefill processes the input prompt, while decode generates output tokens. The deployment recipes separate these stages into different worker pools. This is intended to prevent large prompt-processing workloads from interfering with token generation for active sessions. The source also describes speculative decoding, Expert Parallelism (EP), and Data-Parallel Attention as components of the optimized serving configurations. It identifies DFlash2 speculative decoding in the architecture description. ### Tiered KV-cache storage The architecture uses multiple storage tiers to retain useful context as agent sessions grow. Frequently accessed KV blocks remain in GPU memory. Selected blocks that no longer fit in GPU memory can be offloaded to host DRAM and local NVMe SSD storage. For the vLLM stack, the source identifies the OffloadingConnector. For SGLang, it identifies HiCache. Moving selected cached data beyond GPU memory is intended to reduce unnecessary recomputation during long-running sessions. The exact benefit depends on cache behavior, workload characteristics, and the deployment configuration. ![info-1](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1792405476507-info-1--223-.webp) ## Google Cloud Reports Performance Gains on A4X Max and G4 Google Cloud reports results from two optimized configurations designed for serving Reflection's Beam model. Configuration Hardware Reported result versus its own baseline NVIDIA Dynamo with SGLang A4X Max with 32 NVIDIA GB300 NVL72 GPUs Up to 5x higher throughput and 8x lower latency llm-d with vLLM G4 VMs with 8 NVIDIA RTX PRO 6000 GPUs Up to 2x higher throughput and 8x lower latency Google Cloud explicitly states that the configurations are not directly comparable. They use different hardware and serving stacks, and each reported result measures improvement against a baseline on the same hardware. The source discusses multi-turn coding and tool-use workloads, including the AgentX trajectory dataset, across increasing concurrency levels. It explains that adding replicas can give the routing system more opportunities to find reusable cached prefixes and more flexibility to separate prefill and decode processing. The results should be interpreted as reported benchmark outcomes, not guaranteed performance for every deployment. Google Cloud attributes the figures to customer benchmarking. ![info-2](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1792406702247-info-2--205-.webp) ## Deployment Resources Available at Launch Google Cloud has published deployment recipes and supporting resources for running Beam with the recommended serving stacks. The resources include configurations for A4X Max and G4 infrastructure, alongside open-source Kubernetes manifests, router configurations, and benchmarking scripts that developers can use on their own Google Kubernetes Engine (GKE) clusters. The announcement points developers to the following resources: - **AI Hypercomputer GPU recipes:** [GitHub repository](https://github.com/AI-Hypercomputer/gpu-recipes) - **A4X Max with NVIDIA Dynamo:** [Deployment recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/inference/a4xmax/disaggregated-serving/dynamo) - **G4 with llm-d:** [Deployment recipe](https://github.com/AI-Hypercomputer/gpu-recipes/tree/main/inference/g4/aggregated-serving/llm-d) - **llm-d guides:** [GitHub repository](https://github.com/llm-d/llm-d/tree/main/guides) - **Reflection Beam checkpoints:** [Hugging Face](https://huggingface.co/reflection) - **Beam in Google Cloud Model Garden:** [Model Garden listing](https://console.cloud.google.com/agent-platform/publishers/reflection/model-garden/beam) These resources give developers a starting point for deploying the model and evaluating the serving configurations. Teams still need to benchmark their own workloads to determine how the optimizations perform under their particular concurrency, context-length, and infrastructure requirements. ![info-3](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1792405428911-info-3--191-.webp) ## Analysis: What the Announcement Means for AI Infrastructure The announcement highlights a broader industry shift toward optimizing AI inference as a distributed systems problem. Model-level efficiency remains important, but the infrastructure must also manage context reuse, memory pressure, request routing, and the competing demands of prompt processing and token generation. For enterprises deploying AI coding agents, these techniques could help reduce repeated computation and improve responsiveness during long-running workflows. The practical results, however, depend on the workloads and configurations involved. There are also operational considerations. Teams evaluating a more specialized serving architecture should assess its resource requirements, deployment complexity, monitoring needs, and performance under realistic traffic. The announcement does not provide a universal cost comparison or guarantee the same results for every workload. The published recipes provide a basis for testing the approach, but customer benchmark results should not be treated as a substitute for independent validation on an organization's own infrastructure. ## Conclusion Google Cloud's day 0 support brings Reflection's Beam model together with optimized serving recipes based on llm-d and NVIDIA Dynamo. The architecture addresses common challenges in multi-turn AI inference through prefix-aware routing, prefill and decode separation, speculative decoding, and tiered KV-cache storage. Google Cloud reports up to 5x higher throughput and 8x lower latency on A4X Max, and up to 2x higher throughput and 8x lower latency on G4, relative to baselines on the respective hardware. Because the configurations differ, these results should be assessed separately. For AI infrastructure teams, the main takeaway is that efficient model reasoning is only one part of production performance. How the serving system routes requests, reuses context, and manages memory can also influence the responsiveness and efficiency of long-running AI agents. ## Original source https://cloud.google.com/blog/topics/ai-infrastructure/day-0-support-with-5x-perf-for-reflection-beam-on-google-cloud ## Tags `#GoogleCloud` · `#ReflectionBeam` · `#AIInfrastructure` · `#AgenticAI` · `#LLMInference` · `#NVIDIADynamo` · `#llmd` · `#AIMachineLearning` --- ## About this content This Markdown news article is the citation-grade twin of [Google Cloud Adds Day 0 Support for Reflection Beam With Up to 5x Higher Throughput](https://www.xcademia.com/news/google-cloud-adds-day-0-support-for-reflection-beam-with-up-to-5x-higher-throughput). It is published by **Xcademia** (UK Companies House 12322710) and is available for AI search engines and large language models to index, summarise, and cite. When citing or quoting, please attribute *Xcademia* and link back to the source URL above. - Source: https://www.xcademia.com/news/google-cloud-adds-day-0-support-for-reflection-beam-with-up-to-5x-higher-throughput - Publisher: Xcademia, https://www.xcademia.com - Catalogue index: https://www.xcademia.com/llms-full.txt