In recent years, the field of embodied artificial intelligence has witnessed a paradigm shift with the emergence of Vision-Language-Action (VLA) models. These models aim to unify visual perception, language understanding, and action generation into a single end-to-end framework, enabling an embodied robot to interpret complex instructions and interact with the physical world in a flexible manner. As an embodied robot must operate in unstructured, dynamic environments, the ability to accurately represent and generate actions becomes the central bridge between semantic understanding and physical execution. This survey provides a comprehensive overview of action representation and generation strategies in VLA models for embodied robot, focusing on the fundamental design choices, their trade-offs, and emerging research frontiers.
1. Evolution and Landscape of VLA Models for Embodied Robot
The general architecture of a VLA model consists of three core components: a vision encoder, a language encoder, and an action decoder. The vision encoder processes raw pixel data from cameras into structured feature representations, while the language encoder converts natural language instructions into vectorized forms. The action decoder, which is the most distinctive part for embodied robot, transforms the fused multimodal information into concrete control commands, such as joint angles, end-effector poses, or wheel velocities. Early work like RT-1 demonstrated that by discretizing continuous robot actions into tokens, a standard Transformer can learn to perform a wide range of real-world manipulation tasks. RT-2 further established that internet-scale knowledge from vision-language pre-training can be transferred to robot control, enabling emergent reasoning capabilities. OpenVLA and Octo later introduced open-source VLA models with more scalable and efficient designs, while recent models such as Groot N1 adopt hybrid architectures that combine high-level planning with low-level diffusion control. This rapid evolution highlights the growing importance of action representation and generation as the core differentiating factors among VLA systems for embodied robot.
2. Action Representation in VLA Models
Action representation defines how the physical commands to be executed by an embodied robot are encoded. A well-designed representation must cope with the high-dimensional, continuous, and multimodal nature of robot actions, where multiple valid solutions may exist for the same task. Two dominant paradigms have emerged: discrete action representation and continuous action representation.
2.1 Discrete Action Representation
Discrete action representation quantizes the continuous action space into a finite set of tokens, effectively converting the robot control problem into a sequence modeling task. RT-1 pioneered this approach by uniformly binning each action dimension into 256 intervals, so that every continuous value maps to an integer ID. Gato further unified all input and output modalities into flat token sequences, allowing a single Transformer to process text, images, and actions. RT-2 extended this idea by jointly fine-tuning a vision-language model to directly output action tokens, leveraging the model’s common sense and reasoning abilities. Q-Transformer incorporated offline reinforcement learning within a discrete token framework, enabling robust policy learning from mixed-quality data.
More recent discrete VLA models have expanded the scope to humanoid robots and virtual environments. Humanoid-VLA adapts the autoregressive token paradigm to 24-dimensional whole-body posture control, while JARVIS-VLA generates keyboard and mouse actions for game playing. However, discrete representations inevitably introduce quantization error, which can be unacceptable for tasks requiring sub-millimeter precision. This limitation has motivated the development of continuous action representations that preserve fidelity and support rich multimodality.
| Year | Model | Core paradigm | Platform | Action space type | Action dims | Number of bins | Challenges |
|---|---|---|---|---|---|---|---|
| 2022 | RT-1 | Imitation learning | Mobile manipulator | End-effector pose + base | 11 | 256 | Performance ceiling of imitation learning; limited generalization |
| 2022 | Gato | General supervised learning | Sawyer arm, etc. | End-effector velocity + gripper | 5 | 1024 | Context length constraint; slow inference |
| 2023 | RT-2 | VLM co-finetuning | Mobile manipulator | End-effector pose + base | 11 | 256 | Limited physical skill; high compute cost |
| 2023 | Q-Transformer | Offline RL | Mobile manipulator | End-effector pose + gripper | 8 | 256 | Reward design constraints; high-dimensional action limitation |
| 2024 | OpenVLA | VLM fine-tuning | Multiple arms | End-effector pose + gripper | 7 | 256 | Single image support; low inference efficiency |
| 2025 | Humanoid-VLA | Language-motion alignment | Humanoid robot | Whole-body posture | 24 | 1024 | Limited data quality; relies on low-level RL |
| 2025 | JARVIS-VLA | ActVLP | Virtual agent | Keyboard and mouse | — | 51 | Slow inference; gap with human players |
2.2 Continuous Action Representation
Continuous action representation directly models the probability distribution of actions in the original continuous space, avoiding the quantization errors introduced by binning. The key challenge is action multimodality: for the same task, there exist infinitely many valid trajectories. A simple regression model trained with mean squared error tends to average all correct answers, leading to mode collapse and producing infeasible actions. To address this, continuous methods learn a full probability distribution over actions, from which a concrete trajectory can be sampled at execution time.
ACT uses a conditional variational autoencoder (CVAE) to model a latent variable space that captures the diversity of valid actions. The CVAE encoder maps an expert action sequence to a latent variable, while the decoder generates predicted actions conditioned on observations and latent samples. Diffusion Policy introduced iterative denoising as a powerful way to learn complex multimodal action distributions. Starting from random noise, a noise prediction network progressively refines the action trajectory under the guidance of visual observations. Octo demonstrated that diffusion policies can be scaled across different robot embodiments. The \(\pi_0\) model adopted flow matching, a more efficient generative paradigm that learns a vector field to directly transform noise into action trajectories. DexVLA and HybridVLA further combine these continuous generation approaches with plug-and-play diffusion experts or hybrid architectures.
| Year | Model | Core paradigm | Platform | Action dims | Representation type | Challenges |
|---|---|---|---|---|---|---|
| 2023 | ACT | Imitation learning | Arms | 14 | Conditional VAE | Hardware limitations; perception challenges |
| 2024 | Octo | Imitation learning | Arms | 7/14 | Conditional diffusion | Poor wrist camera handling; demonstration data dependency |
| 2024 | \(\pi_0\) | VLM fine-tuning | Arms, mobile robot | 18 | Conditional flow matching | Dependence on large high-quality demonstration data |
| 2025 | HybridVLA | Co-training | Arms | 7/14 | Hybrid generation | Inference speed constraints |
| 2025 | DexVLA | Embodied curriculum learning | Arms, dexterous hands | — | Multi-head diffusion | Limitations in contact-rich scenarios |
3. Action Generation Strategies in VLA Models
Action generation strategies determine how a VLA model converts multimodal inputs into action sequences. The selection of a strategy directly impacts the quality, speed, diversity, and robustness of an embodied robot’s behavior. This section discusses three major categories: autoregressive generation, non-autoregressive generation, and hybrid generation.
3.1 Autoregressive Generation Strategy
Autoregressive generation follows a sequential, element-by-element prediction paradigm. In the context of VLA models, an autoregressive policy can be formulated as:
\[
\pi(a_t | p, s_{\le t}, a_{<t})
where \(p\) is the language instruction, \(s_{\le t}\) is the history of observations, and \(a_{<t}\) \(s_1,="" \dots,="" a="" action="" actions.="" an="" autoregressive="" causal="" conditional="" decoders="" denotes="" dependency.="" ensure="" factorized="" generated="" into="" is="" joint="" masked="" models="" modern="" of="" p="" previously="" probabilities:
\[
\log p_\theta(s_1, \dots, s_L) = \sum_{l=1}^{L} \log p_\theta(s_l | s_{<l},
For a batch of sequences with indices \(b\), a mask function \(m(b,l)\) indicates whether token \(l\) comes from an action or other modality. The training loss is:
\[
\mathcal{L}_{\text{AR}} = -\frac{1}{|B|} \sum_{b=1}^{|B|} \sum_{l=1}^{L} m(b,l) \log p(s_l^{(b)} | s_{<l}^{(b)})
Autoregressive methods, as used in RT-1, RT-2, OpenVLA, and VIMA, have demonstrated strong scalability and generalization. However, they suffer from slow sequential decoding, typically operating at 3–5 Hz, which is insufficient for real-time control of an embodied robot. Moreover, the discretization required for tokenized actions leads to precision loss.
3.2 Non-Autoregressive Generation Strategies
Non-autoregressive strategies aim to break the sequential bottleneck by generating actions in parallel or via iterative refinement. They are particularly suited for continuous action representation and can achieve much higher inference frequencies.
3.2.1 CVAE-Based Probabilistic Generation
The CVAE-based strategy models the distribution of actions in a latent space to address multimodality and mode collapse. The CVAE encoder \(q_\phi(z | o_t, a_{t:t+k})\) compresses an expert action sequence into a latent variable \(z\). The decoder \(\pi_\theta(a_t | o_t, z)\) generates action predictions conditioned on observations and sampled latent variables. The training objective maximizes the evidence lower bound (ELBO), which consists of a reconstruction loss and a regularization loss:
\[
\mathcal{L}_{\text{recon}} = \text{L1}(a_{t:t+k}, \hat{a}_{t:t+k})
\]
\[
\mathcal{L}_{\text{reg}} = \text{KL}(q_\phi(z | o_t, a_{t:t+k}) \| \mathcal{N}(0, I))
\]
The total loss is:
\[
\mathcal{L} = \mathcal{L}_{\text{recon}} + \beta \mathcal{L}_{\text{reg}}
\]
where \(\beta\) balances reconstruction fidelity and prior adherence. ACT is a representative model that combines CVAE with action chunking to predict short sequences of future actions, achieving high precision in bimanual manipulation tasks.
3.2.2 Diffusion-Based Iterative Generation
Diffusion-based strategies model action generation as an iterative denoising process. A noise prediction network \(\epsilon_\theta\) is trained to remove noise added to action trajectories. Given a Gaussian noise vector \(A^K\), the denoising step is:
\[
A^{k-1} = \alpha A^k – \gamma \epsilon_\theta(A^k, O_t, k) + \mathcal{N}(0, \sigma^2 I)
\]
where \(\alpha, \gamma, \sigma\) are noise schedule parameters. The training loss is a simple mean squared error between predicted and actual noise:
\[
\mathcal{L}_{\text{diff}} = \mathbb{E} \left[ \| \epsilon_\theta(A^k, O_t, k) – \epsilon^k \|^2 \right]
\]
Diffusion Policy pioneered this approach, and subsequent models such as MDT, RDT-1B, and CogACT have adopted the Diffusion Transformer architecture for better scalability. Diffusion-based strategies generate highly smooth and diverse action trajectories, but they require multiple iterative steps, leading to high computational cost and latency.
3.2.3 Flow Matching Generation
Flow matching offers an efficient alternative to diffusion by learning a vector field that transports noise to true data. Define a probability path between noise \(A_0 \sim \mathcal{N}(0, I)\) and real action \(A_1\). The model learns a velocity field \(v_\theta(A_t, O_t, t)\) by minimizing the flow matching loss:
\[
\mathcal{L}_{\text{FM}} = \mathbb{E}_{t, A_1, A_0} \left[ \| v_\theta(A_t, O_t, t) – (A_1 – A_0) \|^2 \right]
\]
At inference, actions are generated by integrating the ODE:
\[
A_{t+\delta} = A_t + \delta \cdot v_\theta(A_t, O_t, t)
\]
Typically, only 10 integration steps are needed. The \(\pi_0\) model uses flow matching as its core generation technique, enabling high-quality continuous control for general embodied robot tasks. GraspVLA further combines flow matching with progressive action generation for efficient grasping.
3.3 Hybrid Generation Strategies
Hybrid strategies combine multiple generation principles within a single VLA model to capitalize on their complementary strengths. A common design pairs an autoregressive high-level planner with a diffusion-based low-level controller. The autoregressive component is responsible for logical reasoning and long-horizon task decomposition, while the diffusion component generates smooth, precise, and multimodal low-level actions.
HybridVLA is a representative example. It integrates collaborative diffusion and autoregressive generators in a unified LLM, allowing both strategies to jointly predict low-level actions and then fuse them adaptively. This architecture has demonstrated superior robustness and generalization in single-arm and bimanual manipulation tasks, outperforming previous SOTA models.
Key challenges in hybrid generation include the temporal and spatial misalignment between high-level planning and low-level control, as well as the grounding of abstract symbolic subtasks into concrete numeric commands. Future solutions may involve asynchronous execution frameworks and shared representation spaces that bridge discrete and continuous domains.
3.4 Comparative Analysis of Generation Strategies
To illustrate the practical impact of different action representation and generation choices, consider a representative task: wiping coffee stains from a table. With discrete action representation and autoregressive generation, the VLA model predicts a sequence of discrete action tokens step-by-step, producing a structured but somewhat jerky motion. In contrast, with continuous action representation and diffusion or flow matching generation, the model starts from random noise and iteratively refines a complete smooth trajectory guided by current visual observations. The resulting motion is fluid, precise, and naturally adapted to the shape of the stain.
| Strategy | Representation | Inference speed | Precision | Multimodality | Complexity |
|---|---|---|---|---|---|
| Autoregressive | Discrete tokens | Low | Medium | Limited | Simple |
| CVAE | Continuous | High | High | Good | Moderate |
| Diffusion | Continuous | Low/Medium | Very high | Excellent | High |
| Flow matching | Continuous | Medium/High | Very high | Excellent | High |
| Hybrid | Mixed | Medium | High | Good | Very high |
4. Evaluation of VLA Models
To systematically assess the scalability, generalization, and lifelong learning capabilities of VLA models for embodied robot, standard benchmark datasets are crucial. Two widely used benchmarks are LIBERO and Open X-Embodiment.
4.1 LIBERO Benchmark
LIBERO (Lifelong Learning Benchmark for Robot Manipulation) is designed to study lifelong learning and knowledge transfer. It comprises 130 tasks organized into four suites, each focusing on different types of knowledge transfer: spatial relations, procedural knowledge, object concepts, and mixed knowledge. All tasks provide expert demonstrations collected via teleoperation. The primary evaluation metric is task success rate.
| Action type | Model | Average success rate (%) |
|---|---|---|
| Continuous | Diffusion Policy | 72.4 |
| Continuous | Octo | 75.1 |
| Continuous | DiT Policy | 82.4 |
| Continuous | OpenVLA-OFT | 95.4 |
| Continuous | \(\pi_0\) | 94.2 |
| Discrete | OpenVLA | 76.5 |
| Discrete | WorldVLA | 79.1 |
The results clearly show that continuous action models, especially those using diffusion and flow matching generation, achieve higher average success rates than discrete action models. This suggests that continuous representations offer better precision and expressiveness for embodied robot manipulation.
4.2 Open X-Embodiment Dataset
Open X-Embodiment is a large-scale, open dataset that aggregates robot interaction data from many institutions and robot platforms. Its goal is to train generalist policies that can transfer across embodiments, tasks, and environments. All data are converted to the standardized RLDS format.
| Action type | Model | Average success rate (%) |
|---|---|---|
| Continuous | Octo-Base | 16.8 |
| Continuous | \(\pi_0\) | 70.1 |
| Discrete | RT-1 | 6.8 |
| Discrete | TraceVLA | 42.0 |
| Discrete | RT-1-X | 53.4 |
| Discrete | RT-2-X | 60.7 |
| Discrete | OpenVLA | 27.7 |
The best continuous model \(\pi_0\) outperforms the best discrete model RT-2-X, indicating a higher performance ceiling for advanced continuous generation. However, Octo-Base performs poorly despite being continuous, highlighting that model architecture and training data matter more than the representation type alone.
5. Challenges and Opportunities
Despite significant progress, the action representation and generation in VLA models for embodied robot still face profound challenges and exciting opportunities.
5.1 Integration with World Models
Combining VLA models with learned world models is a promising direction. World models can predict future states given candidate actions, enabling an embodied robot to perform predictive planning rather than reactive control. This allows long-horizon tasks, simulated rehearsal, and proactive avoidance of failures. The integration of world models with action generation is expected to significantly advance the capabilities of embodied robot systems.
5.2 Breaking Through Traditional Modular Architectures
Traditional robotics relies on separated perception-planning-control pipelines, which suffer from integration gaps and limited adaptability. VLA models offer an end-to-end learning paradigm that directly maps visual and language inputs to actions, eliminating intermediate state conversions and information loss. This unified framework enables the discovery of deeper implicit associations among perception, language, and action, leading to better generalization and robustness compared with modular systems.
5.3 Efficient Generation for Real-Time Control
To overcome the speed bottleneck of existing strategies, new efficient generation techniques are being developed. Parallel decoding strategies, as demonstrated in Groot N1, generate blocks of action tokens concurrently, reducing latency by 60% and enabling 100 Hz control. Compression techniques like FAST operate in the frequency domain to compress long action sequences into fewer tokens, achieving up to 15× inference acceleration. These methods make it feasible to deploy powerful VLA models on resource-constrained embodied robot platforms.
5.4 Cross-Embodiment Generalizable Representations
A key frontier is the development of robot-agnostic action representations that work across different morphologies. Instead of outputting joint angles specific to a particular robot, an abstract action representation could encode intent or end-effector behavior in a canonical form. The ultimate vision is that a VLA model trained once can transfer its skills to a wheeled robot, a quadruped, or a humanoid after only a few minutes of calibration. This would greatly accelerate the deployment of embodied robot systems. The current limitation of imitation learning—being bounded by demonstration data quality—also motivates integrating reinforcement learning for self-improvement.
5.5 Safety and Reliability in Open Worlds
Ensuring safety and reliability in open, dynamic, and unpredictable environments is the most severe challenge for practical deployment. Many safety mechanisms rely on hard-coded thresholds, limiting adaptability. Visual perception degrades under poor lighting or occlusion, and collision prediction accuracy in dynamic spaces is still only about 82%. Emergency stop functions exhibit delays of 200–500 milliseconds, which can be fatal in high-speed operations. Addressing these issues requires robust perception, anticipatory safety models, and low-latency executables, all of which are active research areas for embodied robot.
5.6 Computational and Energy Consumption
The large parameter sizes of advanced VLA models (up to 7B) demand more than 28 GB of GPU memory, far exceeding the capacity of mainstream edge AI platforms like NVIDIA Jetson. Diffusion-based generation incurs approximately 3× the compute cost of autoregressive decoding. For battery-powered mobile robots, energy consumption per inference is a critical constraint. Additionally, the end-to-end black-box nature makes debugging and safety verification difficult. The heavy reliance on large-scale, high-quality demonstration data further increases costs. Efficient model compression, hardware-software co-design, and better data-efficient learning are needed to overcome these barriers.
6. Conclusion
This survey has examined the evolving landscape of action representation and generation in VLA models for embodied robot. The field is clearly shifting from discrete tokenization with autoregressive decoding toward continuous representations powered by generative modeling, such as diffusion and flow matching, to achieve higher fidelity and diversity. Hybrid strategies that combine high-level autoregressive planning with low-level diffusion control represent a promising direction for balancing speed, precision, and robustness. As the community continues to address challenges in real-time control, cross-embodiment generalization, safety, and computational efficiency, VLA models will become increasingly capable of enabling embodied robot to operate autonomously in complex, open-ended real-world environments.