Beijing — The next phase of artificial intelligence is moving beyond screens, texts, and cloud services into homes, factories, roads, hospitals, and space facilities. This transition depends on embodied intelligence: systems that connect perception, language, reasoning, and action in a continuous physical loop. A newly released paper by Xu Mengwei and Xu Daliang of the School of Computer Science at Beijing University of Posts and Telecommunications examines the system-level challenges and opportunities facing on-device embodied intelligence. The work argues that embodied intelligence can no longer rely primarily on remote cloud inference. Network jitter, disconnections, and unpredictable latency can degrade control performance or cause task failure. Instead, large embodied models must reside on the robot itself and complete closed-loop control locally.
The paper begins with the Moravec paradox, which observes that abstract reasoning tasks that appear difficult to humans are often relatively easy for computers, while perception, motion, and physical interaction that even children can perform remain extremely difficult to automate. Over the past decade, deep learning and large models have made substantial progress in language, vision, and code, yet they still struggle to drive robots in open physical environments for grasping, navigation, collaboration, and maintenance. As artificial intelligence expands from digital services into real-world interaction, embodied intelligence is becoming a core challenge for the coming decade. An embodied intelligence system must convert model capabilities into executable actions under continuous observation, limited computation, and real-time constraints. Its effectiveness will determine whether large models can evolve from digital assistants into physical agents.
1. Two Model Architectures Shape the Future of Embodied Intelligence
Recent embodied intelligence architectures have evolved along two representative technical routes. The first is the vision-language-action model, or VLA. A VLA uses a vision-language model as its backbone, encodes camera observations, language instructions, and historical states into a unified representation, and directly outputs robot actions. The second is the world-action model, or WAM. A WAM is trained primarily to predict future states, such as images, latent representations, or other forms of future observation. Building on the idea of world models, it is further optimized for real robot action generation. Together, these two routes are pushing embodied intelligence from task-specific policies toward general foundation models, allowing robots to understand instructions, anticipate environmental changes, and generate continuous actions in more complex and open settings.
Despite rapid architectural progress, embodied intelligence still faces critical system-efficiency challenges before it can move from laboratories to real deployment. A robot interacting with the physical world must make decisions continuously. Network jitter directly degrades control performance, and disconnection can cause task failure. This is why embodied intelligence is shifting from cloud-based remote inference toward models that reside on the device and complete closed-loop control locally. However, stable robot manipulation typically requires control frequencies of 10 Hz to 100 Hz. Billion-parameter-scale VLA models and video-diffusion-based WAM models often cannot achieve the required inference efficiency on end-side computing platforms. The paper therefore discusses and anticipates efficient on-device embodied intelligence systems from both algorithm and system perspectives.
2. VLA Inference and Its Bottlenecks
A VLA model is commonly formed by cascading a visual encoder, a vision-language backbone, and an action head. During inference, the robot’s current visual observation is first converted by the encoder into visual tokens. These tokens, together with tokenized language instructions, are fed into the backbone for a forward computation, often called the prefill stage. This stage produces contextual representations that fuse scene understanding with instruction intent. The action head then uses those representations as conditions to generate low-level control commands, usually in the form of action chunks that output several future action steps at once and execute them sequentially. Early approaches discretized continuous actions and generated them autoregressively token by token. More recent high-performance models increasingly adopt diffusion or flow-matching action experts. These methods start from random noise and generate continuous action vectors through multi-step denoising iterations. Because they model continuous control and multimodal action distributions well, they are becoming mainstream.
The end-side bottlenecks of VLA models are interrelated and jointly constrain system efficiency. First, the visual encoder and backbone carry the main computational cost. A single image frame is often expanded into hundreds to more than a thousand visual tokens, far more than language and action tokens. This pushes attention computation and feed-forward network operations in the prefill stage to a high level. Because each memory access in this stage is accompanied by many floating-point operations, performance is limited mainly by compute rather than memory bandwidth. According to roofline-model analysis, visual encoding and the backbone can be classified as compute-bound. In contrast, the multi-step denoising process of the action expert repeatedly reads model weights and key-value caches at each step, while the computation per step is relatively limited. Its performance is therefore constrained by memory bandwidth.
Measurements on two types of end-side graphics processing units further show this split. On Jetson Thor, the action expert latency share of the π0.5 model is about 38 percent. On a consumer-grade RTX 5090, the action head latency share of GR00T N1.7 rises to 59.2 percent, but its model compute utilization is only 13.2 percent, while the backbone reaches 33.0 percent. The more fully the backbone is accelerated, the more prominent the relative overhead of action generation becomes. Simply raising peak compute cannot effectively remove this bottleneck.
This two-stage structure, with a compute-bound backbone and a memory-bound action expert, directly causes a mismatch between control frequency and model inference latency. Unoptimized 3B to 7B parameter VLA models on edge platforms typically reach only about 6 Hz to 19 Hz. On Jetson Thor, the official reference implementation of π0.5 reaches an equivalent inference frequency of only 5.6 Hz. Even after system optimization, it rises only to 9.3 Hz, far below the 30 Hz target desired for dexterous manipulation. In addition, the prefill stage and the action-generation stage focus on different tokens. The former emphasizes global scene understanding, while the latter focuses on local regions closely related to the current operation. Therefore, directly transferring visual token pruning methods designed for general vision-language models to VLA scenarios often removes key tokens on which actions depend, harming task success rates. This phenomenon requires compression strategies for on-device VLA models to be action-aware.
3. WAM Inference and Its Bottlenecks
WAM inference differs from VLA inference. In embodied settings, world models have two typical uses. One is as an action generator that predicts the future and directly outputs actions. The other is as a rollable environment simulator that performs lookahead rollouts over candidate action sequences and selects the best one. In either use, the computational cost is concentrated in future prediction. Future frames or latent states must be obtained through multi-step diffusion sampling, so a single prediction is already expensive. Long-horizon rolling generation for planning further amplifies this cost. Memory usage grows linearly with the number of rollout steps, and prediction errors accumulate, together limiting usable planning depth. Generating frame by frame in pixel space is far more expensive than rolling in latent space.
Measurements of Cosmos3-Nano also show that, in end-to-end latency, the four-step denoising loop contained in a single action-chunk generation accounts for about 78 percent on Jetson Thor. Unlike the action expert in a VLA, the WAM generation module already exhibits compute-bound characteristics at batch size one. Its arithmetic intensity is about 926 floating-point operations per byte, far above the Jetson Thor performance bottleneck turning point of about 450 floating-point operations per byte. Its model compute utilization reaches 65.8 percent. Therefore, its latency is mainly determined by the raw compute demand of multi-step denoising, and simply increasing memory bandwidth yields limited benefit. More seriously, the model has a memory constraint: peak memory for a single inference reaches the order of 30 GiB, and it cannot run on a 32 GB consumer-grade RTX 5090 because of memory overflow. Moreover, world models currently capable of real-time rolling still mostly rely on server-grade GPUs. Compressing them to end-side devices and completing rolling generation within the control cadence remains an insufficiently solved problem.
| Metric | π0.5 on Jetson Thor | GR00T N1.7 on RTX 5090 | Cosmos3-Nano on Jetson Thor |
|---|---|---|---|
| End-to-end latency | 124.6 ms | 20.5 ms | 8,121 ms |
| Equivalent frequency | 8.00 Hz | 48.80 Hz | 0.12 Hz |
| Backbone/conditioning stage | 76.8 ms, 61.6 percent | 8.3 ms, 40.8 percent | 1,824 ms, 22.5 percent |
| Action head/generation stage | 47.8 ms, 38.4 percent | 12.1 ms, 59.2 percent | 6,298 ms, 77.5 percent |
| Arithmetic intensity of that stage | 50.8 FLOP/byte | 135.8 FLOP/byte | 926.2 FLOP/byte |
| Device performance turning point | 450.4 FLOP/byte | 139.4 FLOP/byte | 448.9 FLOP/byte |
| Model compute utilization of that stage | 7.6 percent | 13.2 percent | 65.8 percent |
| Bottleneck judgment | Memory-bound | Memory-bound | Compute-bound |
4. Efficient Algorithms for Embodied Intelligence
Efficient algorithms reduce inference cost without relying on specialized hardware. They include methods that directly modify the model and methods that reshape the inference execution process. Both are essentially algorithm-level means. Because the diffusion action expert in VLA models and the video diffusion transformer in WAM models both use iterative diffusion as their computational core, the paper first discusses shared diffusion-generation acceleration strategies and then examines model-specific optimizations that exploit embodied structure.
4.1 General Algorithm Optimizations
Few-step denoising is one of the most direct routes. Diffusion and flow-matching generation require multi-step iterative denoising, and the number of steps almost linearly determines inference latency. Consistency and flow-matching distillation methods compress a multi-step teacher model into a single-step or few-step student model while preserving action-generation quality as much as possible. For example, OneDP uses a loss that minimizes the score difference of the diffusion-chain Kullback-Leibler divergence to distill a multi-step diffusion policy into a single-step generator, increasing action prediction frequency from 1.5 Hz to 62 Hz on an NVIDIA V100 GPU. Because actions are temporal signals, FreqPolicy further applies consistency constraints in the frequency domain to safeguard single-step action quality. Streaming Diffusion Policy maintains a variable-noise action buffer, fully denoises only the action currently to be executed, and retains noise states for subsequent actions, producing executable actions with less computation.
Reducing weight or activation precision is another acceleration route orthogonal to step reduction. Post-training quantization methods for video diffusion transformers, such as ViDiT-Q and PTQ4DiT, can compress weights and activations to 4 to 8 bits, effectively reducing memory-access overhead. Quantization on the VLA side must be tightly coupled with control tasks and must use manipulation success rate as a core constraint.
Diffusion feature caching is also important. During iterative denoising, intermediate features in adjacent steps or adjacent generation blocks are often highly similar. Reusing these features can skip redundant computation without retraining the model. General diffusion caching methods are difficult to transfer directly to policies and world models because of architectural differences, so targeted designs have been proposed. Block-wise adaptive caching, for example, designs a block-level adaptive cache mechanism for diffusion policies and points out that feature similarity across time steps and network blocks changes non-uniformly.
Asynchronous action-chunk execution helps hide inference latency. Action chunking improves throughput but introduces reaction lag at chunk boundaries. The inference latency needed to generate the next chunk may prevent a robot from responding in time in dynamic environments. The basic idea of asynchronous execution is to generate the next chunk while executing the current chunk, hiding inference latency inside execution. Because this method only involves generation and execution scheduling of chunks, it applies to any policy that outputs action chunks. A representative work, real-time chunking execution, formulates the problem as image inpainting. It freezes already committed actions as known quantities and treats the remaining parts as unknown quantities to be inpainted, using guidance terms to ensure smooth transitions. The method requires no training, adapts to arbitrary diffusion or flow-matching policies, and can tolerate more than 300 ms of inference latency.
4.2 VLA-Specific Optimizations
VLA-specific optimizations share a common feature: they explicitly exploit structured characteristics of robots, such as temporal continuity and action relevance. They can be grouped into efficient architectures, action-aware compression, action-generation acceleration, and inference-execution acceleration.
Efficient architectures reshape model structure to address the high overhead of backbones and encoders. To alleviate the quadratic complexity of self-attention, RoboMamba replaces the standard attention backbone with a linear-complexity state-space model. To reduce overall model size, TinyVLA and SmolVLA adopt sub-billion-parameter backbones and pair them with diffusion action experts for lightweight operation. To address the problem that visual tokens dominate prefill computation, FastVLM starts from encoder design and directly reduces the number of visual tokens, significantly lowering first-token latency.
Action-aware compression differs from general model compression. It uses manipulation success rate, rather than reconstruction accuracy, as the evaluation criterion and allocates limited bit or token budgets to the parts most critical for action generation. In quantization, saliency-aware quantized imitation learning uses action saliency and a quantization-robust behavior-cloning loss to concentrate precision budgets on key states that strongly affect control performance. It speeds up a four-bit model by about 2.5 times on a Jetson GPU with nearly no performance loss. BitVLA further adopts native ternary weights, reducing model memory overhead to only 1.4 GB, equivalent to one eleventh of the original OpenVLA model. In pruning, VLA-Pruner addresses the semantic-action gap between attention distributions in the prefill stage and the action-decoding stage. It performs dual-level scoring for each token according to both semantic importance and action relevance, preserving key tokens on which actions truly depend while removing redundancy.
Action-generation acceleration directly shortens the decoding process of the action expert or autoregressive action head. One class of work reduces iteration steps or enables parallel decoding. CEED-VLA, for example, uses consistency distillation and early-exit strategies to reform Jacobi decoding, achieving about 4.1 times acceleration for OpenVLA inference on an NVIDIA H100 GPU. Another class improves action representation and decoding form. OpenVLA-OFT replaces autoregressive discrete generation with parallel decoding, action chunking, and continuous action representation. It achieves about 26 times acceleration for OpenVLA inference on an NVIDIA A100 GPU and reaches a 25 Hz action control frequency on a real ALOHA robot. FAST compresses action token sequences through the discrete cosine transform, reducing generation cost at the representation level.
Inference-execution acceleration reshapes decoding and serving execution without changing the model itself. Speculative action decoding mitigates the serial bottleneck of token-by-token generation through a draft-verify mechanism. Spec-VLA and FLASH cover autoregressive and diffusion action heads, respectively. FLASH uses parallel verification of flow consistency in the action expert and achieves about 3.04 times acceleration on a single RTX 4090D GPU. Key-value cache reuse exploits inter-frame redundancy to reduce computation. VLA-Cache reuses key-value representations of static visual tokens that change very little between adjacent frames and selectively recomputes task-relevant tokens, reducing per-step visual processing overhead. It achieves up to about 1.7 times acceleration for OpenVLA inference on a single RTX 4090 GPU. ActionFlow further packs and schedules the memory-bound decoding stage and the compute-bound prefill stage across control steps, bringing compute and bandwidth resources closer to saturation simultaneously.
4.3 WAM-Specific Optimizations
The end-side efficiency of WAM models ultimately depends on how to generate actions from a world model at lower computational cost. The paper characterizes WAM action-generation paradigms along two orthogonal dimensions. The first is the temporal coupling between prediction and action: a sequential mode that completes future prediction first and then derives actions, and a joint mode that produces prediction and action in the same generation process. The second is the maturity of the latent state on which actions depend: whether actions depend on fully denoised prediction results or on intermediate representations that have not completed denoising. On this two-dimensional division, the paper adds two orthogonal cost-reduction paths: using latent-space representations instead of explicit pixel generation, and execution-level optimization for long-horizon rollouts.
In the sequential paradigm, early implementations rely on complete future-state prediction. Fidelity is high, but computational cost is large. Seer, for example, predicts future frames and then derives actions through an inverse dynamics model. DreamVLA instead predicts compact world-knowledge representations such as dynamic regions, depth, and semantics, rather than reconstructing an entire future image, reducing prediction cost.
In the joint paradigm, future states and actions are generated together through a shared denoising or forward process, allowing both to share physical dynamics priors. PAD jointly denoises images and actions in a single diffusion process, while WorldVLA models actions and the world model autoregressively. The limitation of this paradigm is that actions must wait until the full denoising process ends. Producing actions early from intermediate representations within a joint framework, combining information sharing and low latency, still lacks a mature representative work and constitutes an open direction worth exploring. Unified World Models has already set independent noise timesteps for each modality, providing foundational support for early action generation from intermediate representations.
Latent-space world modeling is a more fundamental cost-reduction idea: abandon pixel-level generation and complete prediction in latent space. LAPA learns discrete latent actions from unlabeled video and uses them for pretraining. FLARE aligns policy features with latent embeddings of future observations without generating full future pixel representations. Both avoid explicit pixel-level prediction, effectively reducing the computational burden of world models at the representation and generation levels.
At the execution level, streaming rollouts and long-horizon memory use joint denoising, attention sinks, and historical memory compression to suppress long-horizon error accumulation and manage growing key-value caches. Rolling Forcing adopts three designs. First, it applies increasing noise to multiple frames within a window through joint denoising, updating the entire window at once rather than sampling frame by frame, relaxing strict causal constraints between adjacent frames. Second, it uses an attention sink to retain the key-value state of the first frame as a global anchor over the long term and adjusts its rotary position encoding offset so that relative positions remain bounded. Third, it applies few-step distillation over a significantly extended denoising window, conditioning on self-generated history to mitigate exposure bias.
5. On-Device Embodied Intelligence Inference Systems
Inference engines and runtimes for end-side embodied workloads have emerged rapidly in recent years, and the open-source ecosystem has begun to take shape. Some are specialized frameworks optimized across the full stack for physical-intelligence models. Others are general bases evolved from end-side large language models and covering multiple heterogeneous backends. Still others are end-to-end reference implementations maintained by the robot-learning community. The paper systematically measures single-request latency of four typical embodied models on two classes of end-side GPUs, Jetson Thor and RTX 5090: π0, π0.5, GR00T N1.7, and Cosmos3-Nano.
| Model | Device | LeRobot | PhyAI | FlashRT | vla.cpp | realtime-vla | mllm |
|---|---|---|---|---|---|---|---|
| π0 | Jetson Thor | 321.0 ms | 121.5 ms | 61.0 ms | 267.8 ms | 116.5 ms | NA |
| π0 | RTX 5090 | 127.5 ms | 31.5 ms | 22.2 ms | 61.6 ms | 32.7 ms | NA |
| π0.5 | Jetson Thor | 179.5 ms | 107.7 ms | 86.9 ms | 309.1 ms | 123.8 ms | NA |
| π0.5 | RTX 5090 | 52.1 ms | 28.6 ms | 28.5 ms | 54.2 ms | 27.5 ms | NA |
| GR00T N1.7 | Jetson Thor | 133.5 ms | 95.4 ms | 65.8 ms | 164.6 ms | NA | NA |
| GR00T N1.7 | RTX 5090 | 47.0 ms | 20.6 ms | 19.3 ms | 32.7 ms | NA | NA |
| Cosmos3-Nano | Jetson Thor | NA | 8,121.4 ms | 8,027.5 ms | NA | NA | NA |
| System | Supported models | Supported precision | Hardware backends | Cloud-edge collaborative inference |
|---|---|---|---|---|
| PhyAI | VLA and WAM | BF16, FP8, INT8, NVFP4/MXFP4/INT4 | Jetson Thor, RTX 5090/4090, A40/A100, H20/H100 | Yes, with DP/TP/CFG parallelism |
| mllm | General multimodal large language models | W4A8, W8A8, W4A16 | Arm CPU, Qualcomm Hexagon NPU, Ascend NPU, OpenCL, CUDA | No |
| LeRobot | VLA and WAM | FP32/BF16 | Jetson Thor, RTX 5090/4090, A40/A100, H20/H100 | Yes |
| vla.cpp | VLA | W2A8 | Jetson Orin Nano/Thor | Yes |
| realtime-vla | VLA | BF16 | Jetson Thor, RTX 4090/5090 | No |
| FlashRT | VLA and WAM | BF16, FP8, NVFP4, INT8/INT4 | Jetson Thor/Orin, RTX 5090/4090 | No |
5.1 System Profiles
PhyAI is a high-performance inference framework for physical-intelligence models, including VLA and WAM. Its most prominent feature is that a single code path spans end-side embodied hardware, edge servers, and the cloud. Model evaluation, cloud reinforcement-learning replay, edge GPU serving, and robot-body deployment share the same checkpoint and the same inference path, eliminating the need to write separate inference programs for each scenario. The framework supports low-bit inference optimizations such as NVFP4 and INT4, and implements diffusion transformer caching, operator fusion, and separate deployment of the vision-language backbone and diffusion action expert. The model adaptation layer is decoupled from the runtime. A new architecture only requires a new adapter, while scheduling, memory management, operator selection, and parallel serving are provided uniformly by the runtime. On Jetson Thor, PhyAI compresses π0.5 single-request latency from the official reference implementation’s 179.5 ms to 107.7 ms, about 9.3 Hz, a 1.67 times speedup. This approaches the 10 Hz lower bound needed for manipulation. On a consumer-grade RTX 5090, latency falls further to 28.6 ms, about 35 Hz, entering the real-time control frequency range needed for dexterous manipulation.
mllm is a multimodal large-model inference engine for mobile devices. It is implemented in C/C++, does not depend on a heavy runtime, and supports multiple weight and activation quantization modes such as W4A8, W8A8, and W4A16. Its notable advantage is broad backend coverage. In addition to CUDA and OpenCL, it supports Qualcomm Hexagon neural processing units, Ascend NPUs, and Arm CPUs, which are end-side heterogeneous compute resources not yet covered by the other frameworks. The llm.npu system built on this engine uses prompt chunking, outlier extraction, and block-level out-of-order scheduling to address NPU unfriendliness toward dynamic shapes and activation outliers, raising prefill for billion-parameter models to more than 1,000 tokens per second. At present, mllm does not include VLA or WAM models, so its latency entries are not applicable in the measurement table. However, it can serve as a general base for hosting the vision-language backbone of VLA models and migrating toward NPUs.
LeRobot is an end-to-end robot-learning open-source framework maintained by HuggingFace. It targets real robot scenarios, is compatible with low-cost and edge hardware, and provides checkpoint loading, data processors, training code, reference implementation behavior, and a general asynchronous policy inference stack. It has integrated SmolVLA, Transformer-based action chunking, diffusion transformers, π0, and other models that can run directly on end-side devices. The framework’s positioning covers the full model lifecycle and provides reference implementations rather than pursuing extreme operator-level optimization. It therefore mainly uses FP32/BF16 reference implementations. Measured π0.5 inference latency is 179.5 ms on Jetson Thor, about 5.6 Hz, and 52.1 ms on RTX 5090, about 19.2 Hz.
vla.cpp is the first unified C++ runtime at the ggml/llama.cpp level with native support for flow matching and diffusion action experts. It covers multiple VLA architectures. The runtime supports key-value reuse of the vision-language prefix across solver steps, avoiding repeated prefix computation at each denoising step. It uses ternary 1.58-bit W2A8 weight quantization to compress memory access and memory usage, allowing it to run on tightly memory-constrained devices such as the 8 GB Orin Nano. It supports cloud-edge separated deployment through a ZeroMQ client/server form. Measured π0.5 latency is 309.1 ms on Jetson Thor, about 3.2 Hz, and 54.2 ms on RTX 5090, about 18.4 Hz, indicating that broad portability comes at some latency cost.
realtime-vla decomposes the π0.5 model into 24 general matrix multiplication-like operators and applies Triton kernel optimization level by level to achieve fully streaming inference. It currently covers π0, π0.5, and DM0. The framework does not introduce quantization and relies only on BF16 kernel tuning and streaming scheduling for acceleration, so accuracy is fully aligned with the reference implementation. Measured π0.5 latency is 123.8 ms on Jetson Thor, about 8.1 Hz, and 27.5 ms on RTX 5090, about 36.4 Hz.
FlashRT is a real-time inference engine for small-batch, low-latency workloads such as VLA and WAM. It supports multiple low-precision options including FP8, NVFP4, and INT8/INT4. Hardware backends cover Jetson Thor, RTX series, and Jetson AGX Orin. In addition to VLA, it already supports WAM models such as Cosmos3. Measured π0.5 latency is 86.9 ms on Jetson Thor, about 11.5 Hz, and 28.5 ms on RTX 5090, about 35.1 Hz.
5.2 Comparative Findings for Embodied Intelligence Systems
The six systems have different strengths. Specialized runtimes are indeed faster in specific combinations. FlashRT reaches 86.9 ms and 65.8 ms for π0.5 and GR00T N1.7, respectively, on Jetson Thor. realtime-vla reaches 27.5 ms for π0.5 on RTX 5090. These results are slightly better than PhyAI’s 107.7 ms, 95.4 ms, and 28.6 ms. However, their applicability is clearly narrower. realtime-vla covers only the π series and DM0. It and vla.cpp do not support world-action models, and neither they nor FlashRT provide cloud-edge collaborative inference. Although mllm has the broadest backend coverage, it does not yet include embodied models.
By contrast, within the six systems tested, PhyAI is the only framework that provides measured data for all four models, covers both VLA and WAM, and supports cloud-side scaling through data parallelism, tensor parallelism, and classifier-free-guidance parallelism. Its latency is in the same tier as the fastest specialized runtimes. On RTX 5090 with π0.5, it is only 0.1 ms slower than FlashRT and 1.1 ms slower than realtime-vla. Therefore, within the tested scope, PhyAI achieves a good tradeoff between coverage and latency. It uses one code path and one checkpoint to cover end-side, edge, and cloud deployment scenarios, avoiding repeated inference-program implementation for different environments while maintaining latency at the level of specialized runtimes.
6. Outlook: Algorithm-System-Chip Co-Design for Embodied Intelligence
Although the paper has surveyed a rich set of optimization methods at both algorithm and system levels, on-device embodied intelligence still has a clear gap from practical deployment. Even after optimization, existing systems often cannot stably reach the 10 Hz needed for manipulation, let alone the 30 Hz or higher frequency desired for operation tasks. The measurements confirm this gap. The official reference implementation of π0.5 on Jetson Thor reaches only 5.6 Hz. Even after optimization it reaches only 9.3 Hz. Cosmos3-Nano is even lower, at 0.12 action chunks per second, and cannot run at all on a consumer-grade RTX 5090 because of memory overflow. The root causes differ between the two model classes. For VLA action generation under the single-sample batch processing that robots must use, memory access and startup overhead become the main bottlenecks. Model compute utilization is only 7.6 percent on Jetson Thor, and the action head of GR00T N1.7 on RTX 5090 is only 13.2 percent. For WAM, the generation module already exhibits compute-bound characteristics at batch size one, with utilization as high as 65.8 percent. Its latency is mainly determined by the raw compute demand of multi-step denoising. The former needs reduced memory access and startup overhead; the latter needs reduced denoising compute demand.
6.1 Evaluation Layer: Build Embodied End-to-End Performance Metrics
All optimization depends on correct measurement. Current evaluation of embodied large-model inference still mainly uses latency and energy metrics from traditional large models, lacking an embodied-specific perspective. Such metrics neither characterize the coupling between control frequency and action success rate nor reflect real-time and energy constraints under a perception-action closed loop. They therefore cannot effectively guide optimization for manipulation tasks. The paper argues that it is urgent to customize measurement methods and metric systems from an embodied perspective, jointly considering control frequency, action success rate, end-to-end latency, and closed-loop energy efficiency. Only when evaluation leads can subsequent algorithm and system optimizations have clear objectives and comparable benchmarks.
6.2 Algorithm Layer: Few-Step Generation and Compression Targeting Action Quality
Given the two root causes, algorithm-layer optimization should also differ. For the memory-bound VLA action expert, benefits mainly come from compressing denoising steps to a single step and reducing memory access for weights and key-value caches through low-bit quantization. For the compute-bound WAM generation module, effort should focus on reducing or simplifying denoising steps and compressing latent-space dimensions, because cost grows with raw compute demand. More importantly, the optimization objective must change. Diffusion caching and quantization are important for reducing memory usage and improving speed, but attempts to apply them to VLA and WAM remain limited, and most still use general objectives such as minimizing quantization error or reconstruction error. In embodied settings, minimal error is not equivalent to optimal manipulation. A more reasonable approach is to use action success rate directly as the optimization objective and evaluation criterion, making compression and acceleration responsible to actions rather than only to pixels or tensors.
6.3 System Layer: Specialized Runtimes and Multi-Model Coordination
Dedicated embodied inference runtimes such as vla.cpp and realtime-vla are only beginning. Most real robot deployments still borrow general large-model engines not designed for embodied scenarios, lacking targeted optimization for control-frequency constraints and the perception-action closed loop. There remains an obvious gap between the acceleration claimed by algorithms and their actual execution efficiency on real central processing units, GPUs, and NPUs. Algorithm-system cross-layer co-design is urgently needed to close this gap. On the other hand, an end-side embodied device often runs more than a control model. It may also host an OMNI-style general interaction large model. Existing work mostly considers single-model execution. How control models and interaction models can be collaboratively scheduled and executed in parallel on limited end-side resources is a key system problem that has not been sufficiently studied.
6.4 Chip Layer: Unified Accelerators for Vision-Language Models and Diffusion Transformers
Specialized chips for embodied workloads are almost absent. The core difficulty is heterogeneous execution demand. The vision-language backbone in a VLA is compute-bound, while the diffusion action expert is memory-bound and requires multi-step iteration. Their execution patterns differ significantly. How to make a single accelerator efficiently support both compute modes and natively support embodied and diffusion-specific operators such as adaptive normalization is a key architectural problem. Overall, work at this layer remains scarce and early, making it the most promising and most urgent direction in the full stack.
7. Conclusion
On-device embodied intelligence is redefining the system foundation for robots entering the real physical world. By moving large embodied models from cloud-based remote inference to local closed-loop execution on the device, embodied intelligence can reduce dependence on unstable networks and enable reliable control at 10 Hz to 100 Hz. Yet the paper shows that VLA and WAM models introduce severe computational and memory-access overhead through multimodal perception, diffusion-based action generation, and world-model rolling. These overheads conflict with limited power, constrained memory, and real-time control requirements on end-side hardware. The proposed direction is to treat action success rate, closed-loop latency, and energy efficiency as joint optimization objectives and to pursue algorithm-system co-design around few-step generation, action-aware compression, cache reuse, asynchronous chunk execution, and specialized runtimes. For embodied intelligence to become practical, progress must span evaluation, algorithms, systems, and chips. The challenge is not only to make models smaller or faster, but to make every layer of the embodied intelligence stack accountable to physical action, real-time control, and successful operation in the real world.