In recent years, artificial intelligence has found increasingly broad applications in space operations, ranging from autonomous scientific target selection on Mars rovers to intelligent in-orbit servicing systems. As space missions evolve toward greater complexity and autonomy, the demand for versatile robotic platforms capable of performing diverse operational tasks has become ever more prominent. Among various robotic architectures, the humanoid robot stands out due to its unique advantages in generality, adaptability, and dexterity. Unlike traditional planetary rovers or fixed-base manipulators, a humanoid robot can directly interface with tools, equipment, and workstations designed for human astronauts, thereby substantially reducing the costs associated with spacecraft design and maintenance. Furthermore, when deployed on extraterrestrial surfaces, the humanoid robot’s anthropomorphic limb structure and whole-body motion control capabilities enable it to perform a wide spectrum of locomotion and manipulation skills, making it exceptionally well-suited for navigating complex, unstructured environments.
However, equipping a humanoid robot with a diverse repertoire of whole-body motion skills while maintaining smooth transitions between those skills remains a formidable challenge. Reinforcement learning has demonstrated remarkable progress in humanoid robot skill acquisition. Several approaches have enabled bipedal robots to achieve stable walking in outdoor environments, high-dynamic motions such as running and jumping, and even autonomous standing-up from arbitrary postures. Nevertheless, these methods typically target a single or narrowly defined set of skills, training and deploying one policy per skill. The question of how to generate a single policy model capable of mastering multiple skills simultaneously, while ensuring both the quality of individual skill execution and the coherence of transitions between skills, is still open.
In this study, we address this challenge by proposing a single model imitation learning for multi-skill efficiency framework, referred to as SMILE. Our approach integrates goal-conditioned reinforcement learning with generative adversarial imitation learning, augmented by two key innovations: preference-based reward shaping and a failure-frequency-based priority sampling mechanism. Through extensive simulation experiments on the BICE-Rob humanoid robot, we demonstrate that SMILE enables the acquisition of diverse whole-body skills such as standing, squatting, walking, obstacle jumping, stooping for detailed inspection, and object picking, while achieving coherent transitions between skills. The proposed method achieves a success rate of 93.33%, significantly outperforming ablation baselines and conventional reinforcement learning approaches.
1 Problem Formulation
We model the multi-skill learning task for a humanoid robot as a finite Markov decision process, denoted by the tuple \( (\mathcal{S}, \mathcal{A}, \mathcal{R}, \mathcal{P}, \gamma) \). Here, \( \mathcal{S} \) denotes the state space, \( \mathcal{A} \) represents the action space, \( \mathcal{R} \) is the reward function, \( \mathcal{P} \) is the state transition function, and \( \gamma \in (0,1] \) is the discount factor.
At each time step \( t \), the humanoid robot observes a state \( s_t \) consisting of a proprioceptive state \( s_t^p \) and a goal state \( s_t^g \). The robot executes an action \( a_t \) determined by its policy \( \pi \), and receives a scalar reward \( R(s_t^p, s_t^g, a_t) \). The next proprioceptive state \( s_{t+1}^p \) is sampled according to the transition function \( \mathcal{P} \), while the next goal state \( s_{t+1}^g \) is obtained from the reference motion. Each episode begins from an initial state \( s_0 \sim \mathcal{I} \), where \( \mathcal{I} \) is the initial state distribution. The policy \( \pi \) interacts with the environment to generate a trajectory \( \tau_i = (s_0^i, a_0^i, R_1^i, s_1^i, a_1^i, R_2^i, \ldots) \).
The optimization objective is to maximize the expected cumulative discounted reward:
$$ J(\pi) = \mathbb{E}_{\tau \sim \pi} \left[ \sum_{t=0}^{T-1} \gamma^t R(s_t^p, s_t^g, a_t) \right] \tag{1} $$
where \( T \) denotes the episode horizon and \( \mathbb{E}_{\tau \sim \pi}(\cdot) \) represents the mathematical expectation over all trajectories generated by policy \( \pi \).
To evaluate the performance of the learned policy, we adopt the task success rate \( E_s \) as the primary evaluation metric. The success rate measures whether the humanoid robot can maintain whole-body stability and synchronize with the reference motion. Specifically, if at any time instant the average Euclidean distance between the positions of all joints of the humanoid robot and the corresponding joints of the reference motion exceeds 0.5 m, the episode is considered a failure; otherwise, it is considered a success.
For this study, we selected ten essential skills that a humanoid robot should possess for space-related applications. These skills are summarized in Table 1.
| No. | Skill name | Application scenario |
|---|---|---|
| 1 | Standing | Maintaining balance in upright posture while performing coordinated upper-limb motions or dexterous manipulations |
| 2 | Squatting | Picking up samples from an extraterrestrial surface or accessing interfaces at the bottom of maintenance equipment |
| 3 | Walking | Transitioning among forward, backward, and lateral walking for autonomous charging, emergency stopping, or obstacle avoidance |
| 4 | Carrying | Transferring supply boxes in a space station or transporting equipment and construction materials on an extraterrestrial surface |
| 5 | Stooping for inspection | Bending over or stooping to observe instruments inside a cabin or inspect the surrounding environment |
| 6 | Long-duration cruising | Conducting patrol inspection between orbital modules or long-distance patrol and exploration on planetary surfaces |
| 7 | Multi-directional picking | Picking up objects at different positions and grasping handrails or fixed points in various directions |
| 8 | Stand-sit transition | Transitioning postures at a control console or cockpit, or switching from various standing postures to sitting to conserve energy |
| 9 | Obstacle jumping | Rapidly leaping over gullies, pits, and other obstacles to improve mobility efficiency and obstacle-crossing capability |
| 10 | High-dynamic striding | Rapidly switching support points to step over obstacles or uneven terrains quickly |
2 BICE-Rob Humanoid Robot Platform
The BICE-Rob humanoid robot system is designed with a three-tier architecture comprising a perception layer, a decision layer, and a motion control layer. The perception layer integrates a binocular camera, a depth camera, a LiDAR sensor, an inertial measurement unit (IMU), and a dedicated perception computer. The decision layer consists of a main decision computer that processes sensory data, generates task commands autonomously or parses teleoperation instructions, and dispatches commands to the motion control layer. The motion control layer, housed in a dedicated motion control computer, computes joint-level control commands and sends them to the actuators.
BICE-Rob stands at 1.50 m tall and weighs 55 kg. The robot is actuated by electric motors across 55 degrees of actuation, comprising: two 13-DOA BICE five-finger anthropomorphic dexterous hands (20-DOF each including the wrist), two 14-DOA anthropomorphic arms, a 2-DOA neck, a 1-DOA waist, 3-DOA hips, 1-DOA knees, and 2-DOA ankles. This configuration provides high anthropomorphic fidelity in overall morphology. The peak torque of the hip and knee joints is 340 N·m, enabling exceptional explosive power and locomotion agility during high-dynamic maneuvers. Additionally, BICE-Rob possesses teleoperation capability to fulfill requirements of extravehicular maintenance and planetary surface exploration tasks.
3 The SMILE Method
The proposed SMILE method combines goal-conditioned reinforcement learning with generative adversarial imitation learning to achieve efficient multi-skill whole-body motion policy generation. The core components include a discriminator network, a policy network, a value network, a reference motion sampling strategy, and a reward shaping mechanism. The training procedure alternates between updating the discriminator and optimizing the policy. The discriminator learns to distinguish between state-action pairs generated by the policy and those from the reference motion dataset, thereby gaining the ability to discriminate between different motion styles. Concurrently, guided by the imitation reward, discriminator reward, preference reward, and physical constraint reward, the policy is continually refined in an attempt to fool the discriminator, thereby producing motions that closely resemble human behavior. This adversarial training paradigm enables the policy to master diverse human-like skills.
3.1 Human Motion Retargeting
Since the skeleton topology, degrees of freedom, and joint axis conventions of the humanoid robot differ from those of human motion capture data, a retargeting procedure is required. Our retargeting pipeline consists of two stages. In the first stage, we optimize the shape parameters of the SMPL model to make the joint positions of the SMPL model, under a reference posture, as close as possible to the corresponding joint positions of BICE-Rob. This minimizes motion distortions in the subsequent mapping stage that could arise from body proportion differences. In the second stage, we select a subset of joint positions as keypoints and establish the spatial position matching relationship between BICE-Rob and the human motion data. Inverse kinematics is then solved to obtain the joint angles of the humanoid robot, ensuring that the spatial trajectories of BICE-Rob align with the human demonstrations.
3.2 Network Architecture
The policy network is a six-layer multilayer perceptron (MLP) with 2048, 1536, 1024, 1024, 512, and 512 neurons in each layer, respectively. All hidden layers use the SiLU activation function, while the output layer applies a linear mapping. The value network adopts an identical six-layer MLP structure. The discriminator network contains two hidden layers with 1024 and 512 neurons, respectively, and uses the ReLU activation function. The loss function of the discriminator is designed based on the AMP framework. The policy network is trained using the proximal policy optimization algorithm.
| Component | Architecture | Activation | Learning rate |
|---|---|---|---|
| Policy network | MLP 2048-1536-1024-1024-512-512 | SiLU (hidden), Linear (output) | 5×10⁻⁵ |
| Value network | MLP 2048-1536-1024-1024-512-512 | SiLU (hidden), Linear (output) | 5×10⁻⁵ |
| Discriminator | MLP 1024-512 | ReLU (hidden) | 1×10⁻⁴ |
| Discount factor \(\gamma\) | 0.99 | ||
| GAE parameter \(\lambda\) | 0.95 | ||
| Clip ratio \(\epsilon_{\text{clip}}\) | 0.2 | ||
3.3 State and Action Spaces
The proprioceptive state \( s_t^p \in \mathbb{R}^{298} \) represents the actual motion state of BICE-Rob, including the posture, 3D position, angular velocity, and linear velocity of all joints:
$$ s_t^p \triangleq [r_t, \ p_t, \ \omega_t, \ v_t] \tag{2} $$
where \( r_t \), \( p_t \), \( \omega_t \), and \( v_t \) denote the posture (in 6D representation for training stability), position, angular velocity, and linear velocity of each joint at the current time, respectively.
The goal state \( s_t^g \in \mathbb{R}^{480} \) is designed to measure the discrepancy between the actual motion and the reference motion:
$$ s_t^g \triangleq \left( \hat{r}_{t+1} \odot r_t, \ \hat{p}_{t+1} – p_t, \ \hat{v}_{t+1} – v_t, \ \hat{\omega}_t – \omega_t, \ \hat{\theta}_{t+1}, \ \hat{p}_{t+1} \right) \tag{3} $$
where \( \hat{r}_{t+1} \), \( \hat{p}_{t+1} \), \( \hat{v}_{t+1} \), and \( \hat{\omega}_t \) denote the reference posture, position, linear velocity at the next time step, and the reference angular velocity at the current time step, respectively. The symbol \( \odot \) denotes the relative rotation error between the reference posture \( \hat{r}_{t+1} \) and the current posture \( r_t \). We adopt the convention that quantities with a hat ( \( \hat{\cdot} \) ) are ground-truth values from the motion dataset, while quantities without additional symbols are measured values in the simulation system.
In the action space, we do not consider the degrees of freedom of the neck, dexterous hands, wrists, and ankle inversion/eversion. The action is defined as the target joint positions for the PD controllers:
$$ a_t \in \mathbb{R}^{19} \tag{4} $$
3.4 Reward Function Design
The reward function comprises four components: imitation reward, discriminator reward, preference reward, and physical constraint reward. The imitation, discriminator, and physical constraint rewards follow the design principles established in prior work. The imitation reward includes six sub-terms: joint angle imitation reward, joint angular velocity imitation reward, position imitation reward, posture imitation reward, linear velocity imitation reward, and angular velocity imitation reward. Each sub-term is computed via an exponential decay function. The discriminator reward is derived from the discriminator output — when the state distribution generated by the humanoid robot policy closely matches the reference motion distribution, the discriminator assigns a higher reward. The physical constraint reward encourages the robot to complete tasks with lower energy consumption and to reduce high-frequency foot oscillations.
The key novelty lies in the introduction of the preference reward, which constrains behaviors that the discriminator may deem acceptable but that humans would consider unreasonable. The preference reward consists of three components: a tripping penalty \( r_b \), a foot-sliding penalty \( r_h \), and a foot-orientation reward \( r_o \).
The tripping penalty \( r_b \) penalizes signs of tripping (such as toe-stubbing causing forward torso lean) or actual falls, encouraging the robot to avoid severe imbalance or falling. The foot-sliding penalty \( r_h \) penalizes obvious slippage of the feet relative to the ground during stance, promoting stable foot contact without frictional sliding. The foot-orientation reward \( r_o \) penalizes non-vertical orientations of the humanoid robot’s feet, encouraging the soles to remain flat on the ground with the foot normal direction as aligned with gravity as possible, thereby avoiding toe-only contact or single-edge foot contact.
The composite preference reward is formulated as:
$$ r_h = k_b r_b + k_h r_h + k_o r_o \tag{5} $$
where the individual terms are:
$$ r_b = \mathbf{I}\left( \| F_{xy}^{feet} \|_2 > 5 \ \| F_z^{feet} \| \right) \tag{6} $$
$$ r_h = \| v_{feet} \|_2^2 \times \mathbf{I}\left( \| F_{feet} \|_2 > 1 \right) \tag{7} $$
$$ r_o = \| g_{xy}^{l\_feet} \|_2^2 + \| g_{xy}^{r\_feet} \|_2^2 \tag{8} $$
In the above, \( \mathbf{I}(\cdot) \) denotes the indicator function, \( F_{feet} \), \( F_{xy}^{feet} \), and \( F_z^{feet} \) represent the contact force, the horizontal component of the contact force, and the vertical component of the contact force of both feet, respectively. The term \( v_{feet} \) denotes the velocity of the feet, and \( g_{xy}^{l\_feet} \) and \( g_{xy}^{r\_feet} \) are the horizontal projections of the gravity vector on the left and right feet, respectively.
The coefficients \( k_b \), \( k_h \), and \( k_o \) are hyperparameters that balance the relative contributions of each preference sub-term. We used \( k_b = 2.0 \), \( k_h = 0.5 \), and \( k_o = 1.0 \).
| Reward type | Symbol | Description |
|---|---|---|
| Imitation reward | \( r_{t}^{imp} \) | Exponential decay functions for joint angles, joint angular velocities, positions, postures, linear velocities, and angular velocities |
| Discriminator reward | \( r_{t}^{d} \) | Based on discriminator output, encourages matching the reference motion distribution |
| Preference reward | \( r_{t}^{h} \) | Tripping penalty, foot-sliding penalty, and foot-orientation reward to enforce human-preferred behaviors |
| Physical constraint reward | \( r_{t}^{c} \) | Penalizes energy consumption and high-frequency foot oscillations |
3.5 Initialization and Termination Conditions
The initial state distribution and termination conditions have a critical influence on the effectiveness of multi-skill learning. We employ reference-state initialization: a state is randomly sampled from a reference motion segment as the initial state of the humanoid robot at each episode. This approach reduces policy exploration difficulty and improves learning efficiency. Additionally, an early termination mechanism is introduced during training: if the average deviation between the humanoid robot’s joint positions and the reference motion exceeds 0.25 m, the current episode is terminated with a failure signal.
3.6 Failure-Frequency-Based Priority Sampling
To mitigate the problems of catastrophic forgetting, severe fluctuations in learning curves, and convergence difficulties when a single policy model learns multiple skills, we extend the idea of prioritized experience replay to the imitation learning domain and propose a failure-frequency-based priority sampling method. The fundamental concept is to automatically evaluate task success rates during training and prioritize the sampling of reference motion samples that the policy struggles to master.
Initially, the policy learns all reference motions through uniform sampling. When the policy exhibits repeated failures on certain samples, the sampling probability of those challenging samples is progressively increased. Meanwhile, samples that have already been mastered are assigned a smaller sampling probability to consolidate existing skills and prevent complete forgetting.
During training, the sampling probability of reference motion samples is given by:
$$ P_i = \frac{(\rho_i^k + \varepsilon)^{\alpha}}{\sum_{j} (\rho_j^k + \varepsilon)^{\alpha}} \tag{9} $$
where \( \varepsilon \) is a smoothing term that prevents samples with zero failure count from never being sampled, \( \alpha \) is a power parameter that adjusts the degree of prioritization, and \( \rho_i^k \) represents the cumulative failure count of sample \( i \) at the \( k \)-th evaluation. The update rule for \( \rho_i^k \) is:
$$ \rho_i^k \leftarrow \eta \rho_i^{k-1} + \Delta_{fail} \tag{10} $$
In the above, \( \Delta_{fail} = 1 \) indicates that sample \( i \) resulted in failure at the \( k \)-th evaluation, otherwise \( \Delta_{fail} = 0 \). The parameter \( \eta \in (0,1] \) is a decay factor that controls the influence of historical failure counts. We set \( \eta = 0.95 \), \( \alpha = 0.6 \), and \( \varepsilon = 1 \times 10^{-6} \) in our experiments.
4 Simulation Results and Analysis
All simulation experiments were carried out in Isaac Gym. The policy runs at 30 Hz while the physics simulation operates at 60 Hz. Training was performed on a workstation equipped with an Intel i9-13900K CPU and an NVIDIA RTX 4090 GPU. From the AMASS motion capture dataset, we selected 15 groups of representative diverse human demonstrations covering the ten skills listed in Table 1. These include stable standing, squatting, carrying, multi-directional picking, stooping for detailed inspection, forward walking, lateral-to-forward walking transition, backward-to-forward walking transition, long-duration cruising, stepping in place, obstacle jumping, aerial turning, standing-to-high-dynamic-striding transition, walking-to-high-dynamic-striding transition, and stand-sit transitions.
The BICE-Rob humanoid robot successfully learned a wide range of skills through the SMILE method. For multi-directional object picking, the humanoid robot demonstrated coordinated whole-body movements including waist rotation, leg adjustment, and foot repositioning to achieve postural adjustments and turns in different directions. During forward and backward walking transitions, BICE-Rob exhibited human-like heel-to-toe gait patterns: the forward foot’s heel lands first, the center of mass gradually shifts, and the rear foot lifts off in a heel-to-toe sequence. For lateral-to-forward walking transitions, the humanoid robot initially faces forward and progressively completes the transition from lateral to forward gait through coordinated whole-body movements. During the turning process, one foot remains flat on the ground while the other rotates on its toes, matching common human turning characteristics.
In the stooping-to-inspect skill, since BICE-Rob’s waist has only one rotational degree of freedom, the bending motion is accomplished through coordinated whole-body joint movements, primarily relying on hip joint flexion to achieve trunk forward tilt. The high-dynamic striding skill demonstrates the robot’s ability to achieve the desired leg-raising height through coordinated whole-body motion: the support leg is on its toes while the other leg rapidly lifts to a high position. During this motion, the humanoid robot briefly exhibits a flight phase before the support leg regains contact with the ground. The obstacle-jumping skill successfully demonstrates both feet simultaneously leaving the ground. The humanoid robot is also capable of performing a series of static actions, including various standing postures, squatting, and sitting in different orientations. Notably, it achieves a unique human-like sitting posture where the upper body leans backward while maintaining stability, demonstrating the robot’s whole-body balance control capability under complex static postures.
4.1 Ablation Study
To verify the contribution of each core component of SMILE, we conducted ablation experiments by removing the failure-frequency-based priority sampling and the preference reward, respectively. The learning curves are illustrated in the accompanying figure. Table 4 summarizes the quantitative comparison results.
| Method | Success rate | Average episode length | Discriminator reward |
|---|---|---|---|
| SMILE (full) | 93.33% | 309 steps | ~1.6 |
| Without preference reward | 53.33% | 177 steps | ~0.6 |
| Without priority sampling (uniform) | 73.33% | — | — |
The ablation results demonstrate the importance of both proposed components. When the preference reward is removed, the success rate drops dramatically from 93.33% to 53.33%, and the average episode length decreases from 309 to 177 steps. The discriminator reward also saturates at a much lower value (~0.6 compared to ~1.6 for the full SMILE method), indicating that the generated policies deviate significantly from human motion patterns without preference guidance. When uniform sampling replaces the failure-frequency-based priority sampling, the success rate reaches only 73.33%, a 20% drop compared to SMILE.
The preference reward contributes to three major advantages in the multi-skill learning task:
1. Significantly improved task success rate. The failure-frequency-based priority sampling method clearly accelerates learning and improves the final success rate. SMILE reaches roughly 93.33% success within about 4500 evaluations, whereas uniform sampling plateaus at 73.33%. This improvement stems from the adaptive re-weighting of samples: the policy allocates more training effort to skills it has not yet mastered, preventing early-skill dominance.
2. Enhanced training efficiency and stability. The success rate, average episode length, and total reward growth trends are all faster and more stable for SMILE. In the initial phase (1500 evaluations), SMILE already achieves 73.33% success, while the variant without preference reward only reaches 33.33%. Moreover, SMILE’s learning curves exhibit no significant drops or rebounds, whereas the variant without preference reward shows notable fluctuations around 1500 evaluations. This demonstrates that the preference reward reduces early training instability and the probability of converging to local optima.
3. Improved human-likeness of multi-skill policies. The discriminator reward curve indicates that SMILE quickly surpasses 1.0 within approximately 580 training iterations and steadily rises to about 1.6 by the end of training. In contrast, the variant without preference reward increases slowly and only reaches 0.6. Since the discriminator reward directly reflects the similarity between the generated state distribution and the human demonstration data distribution, higher values indicate that the policy generates motions that are closer to human behavior patterns.
4.2 Comparison with Goal-Conditioned Reinforcement Learning
To evaluate whether goal-conditioned reinforcement learning methods can solve the challenge of balancing motion execution quality and transition coherence by simply adjusting reward functions and fine-tuning policies, we designed a comparative experiment against OmniH2O, a representative goal-conditioned reinforcement learning approach. The training procedure was divided into two stages. In the first stage, we trained BICE-Rob to learn diverse motion skills using the reward function and hyperparameters from OmniH2O, yielding an initial policy. In the second stage, we fine-tuned this initial policy by replacing the reward function with the imitation reward, physical constraint reward, and preference reward from SMILE — deliberately excluding the discriminator reward. This stage was designed to investigate whether a reinforcement learning method, given a reasonable base skill set and optimized rewards, could master more complex, diverse, and highly dynamic skills.
The learning curves showed that the initial policy exhibited a slow but consistent upward trend in both average episode length and reward, eventually stabilizing at relatively low values. This indicates that the initial policy could only master a small subset of skills. The second-stage fine-tuning curves showed rapid early improvement, demonstrating the benefit of starting from a pre-trained policy. However, the curves were notably more volatile, indicating that without the discriminator reward constraining motion quality, the policy’s capacity to learn diverse skills remained fundamentally limited.
Visual inspection of the fine-tuned policy’s behavior revealed three critical deficiencies:
1. Substantially degraded motion execution quality. For skills with pronounced dynamic characteristics, BICE-Rob struggled to reproduce the key features of the reference motions and instead resorted to small shuffling steps or torso swaying. For example, the obstacle-jumping skill, which should exhibit the key feature of both feet simultaneously leaving the ground, was replaced by a sequence of small shuffling steps with no flight phase. This motion distortion demonstrates that without effective discriminator constraints, the humanoid robot tends to find low-effort shortcuts that fail to capture the essence of the demonstrated skill.
2. Clearly reduced motion execution efficiency. When performing skills that involve large strides or rapid movements — such as walking, high-dynamic striding, long-duration cruising, and stooping for detailed inspection — BICE-Rob often exhibited small step sizes and frequent foot adjustments. This resulted in unsmooth motion transitions, sluggish behavior, and severely limited overall task performance. The robot’s movement efficiency was significantly lower than that of the SMILE-trained policy, directly impacting the quality of skill transitions.
3. Regression or forgetting of previously learned skills. Because the fine-tuning stage reduced the reward for maintaining a conservative standing posture and large-step movements, BICE-Rob exhibited noticeable regression on skills that the initial policy had already mastered. For instance, the forward and backward walking skills that were performed smoothly at large step sizes in the initial policy degraded to inefficient small shuffling steps after fine-tuning. This observation indicates that while the humanoid robot acquired new skills, it often did so in non-standard, inefficient ways, leading to a trade-off problem where improving one skill comes at the expense of others.
The core issue underlying these deficiencies is the absence of a discriminator reward to serve as an explicit constraint on imitation quality. Without this signal, the humanoid robot is prone to executing tasks in degraded or inefficient modes, significantly limiting the overall policy performance. Our SMILE method, by leveraging the adversarial training between the policy and discriminator, establishes a clear quality signal for imitation. The preference reward then further guides the optimization away from unreasonable motions that might fool the discriminator but violate human preferences, thus resolving the trade-off between motion quality and transition coherence.
5 Discussion
The results presented in this study provide strong evidence that the proposed SMILE method effectively addresses the two key challenges in multi-skill humanoid robot policy learning. First, the preference-based reward mechanism helps the policy navigate the conflicting requirements of different skills. Static skills such as standing and squatting require maintaining static equilibrium conditions, whereas high-dynamic skills such as obstacle jumping and high-dynamic striding require deliberately breaking static equilibrium and quickly restoring stability after rapid, agile maneuvers. The preference reward shapes the policy optimization landscape to balance these conflicting demands, reducing the risk of policy degradation.
Second, SMILE achieves a favorable trade-off between motion execution quality and transition coherence. By combining the discriminator reward (which enforces similarity to human demonstrations) with the preference reward (which encodes human preferences for reasonable motion), the policy can generate high-quality motions for each individual skill while ensuring coherent, coordinated transitions between skills. The success rate of 93.33% achieved by SMILE outperforms both the ablation baselines (53.33% without preference reward, 73.33% without priority sampling) and the goal-conditioned reinforcement learning baseline, which showed significant performance degradation during fine-tuning.
It is worth noting that the SMILE method exhibits good scalability. Although the current study validates the approach on a relatively limited skill set, the same policy generation methodology can be extended to larger, more diverse datasets. The preference reward and failure-frequency-based priority sampling continue to play guiding roles across a broader range of skills, suggesting that high learning efficiency and policy performance can be maintained even as the number of skills increases. Future research should investigate the scenario of continuously growing skill sets, where the humanoid robot faces more pronounced inter-skill conflicts while needing to preserve existing capabilities.
Looking forward, we plan to explore simulation-to-reality transfer methods, leveraging a teacher-student paradigm to improve the practical applicability of the proposed approach on the physical BICE-Rob platform. The gap between simulation and reality remains a critical barrier for deploying humanoid robot policies in real-world space missions. Domain randomization, system identification, and adaptive control techniques will be essential to bridge this gap and ensure robust performance in real environments.
6 Conclusion
In this study, we proposed SMILE, a single-model multi-skill efficient imitation learning method for humanoid robot whole-body motion policy generation. By integrating goal-conditioned reinforcement learning with generative adversarial imitation learning, and introducing preference-based reward shaping and failure-frequency-based priority sampling, SMILE achieves the following key contributions:
1. It enables a single policy model to acquire diverse whole-body skills including standing, squatting, walking, obstacle jumping, stooping, and object picking, with an overall success rate of 93.33%.
2. It effectively mitigates the optimization conflicts arising from the distinct motion characteristics of different skills, allowing the policy to balance static stability requirements with dynamic agile maneuvers.
3. It achieves a favorable trade-off between motion execution quality and transition coherence, ensuring that individual skills are executed at high quality while transitions between skills remain smooth and coordinated.
The ablation experiments confirm that both the preference reward and the failure-frequency-based priority sampling are essential components. Removing either component leads to substantial degradation: the success rate drops to 53.33% without the preference reward and to 73.33% without priority sampling. The SMILE method also demonstrates superior performance compared to goal-conditioned reinforcement learning baselines, which exhibit motion degradation, reduced efficiency, and skill regression when fine-tuned for multi-skill learning tasks.
The proposed method offers a novel perspective on humanoid robot multi-skill imitation learning and provides a technical foundation for future deployment in space station operation and extraterrestrial exploration tasks where humanoid robots must autonomously adapt to diverse, unpredictable environments while maintaining robust and human-like whole-body control.