A new simulation study reports that a single policy model can enable a humanoid robot to learn multiple whole-body motion skills, including standing, squatting, walking, obstacle jumping, stooping for inspection, and object picking, while maintaining coherent transitions between those skills. The proposed method, named single model imitation learning for multi-skill efficiency, or SMILE, combines goal-conditioned reinforcement learning with generative adversarial imitation learning. According to the reported results, the humanoid robot policy achieved an overall success rate of 93.33% in simulation, and the method was designed to address two persistent difficulties in multi-skill humanoid robot learning: conflicts caused by the different motion characteristics of different skills, and the trade-off between motion quality and transition coherence during skill switching.
The research focuses on humanoid robot autonomy for space-related and extraterrestrial scenarios. The paper notes that humanoid robots offer advantages in generality, adaptability, and dexterity compared with planetary rovers and space manipulators. Inside a space station, a humanoid robot could directly use tools, equipment, and interfaces designed for astronauts, reducing the need for additional adaptation. On extraterrestrial surfaces, a humanoid robot with human-like limb structure and highly flexible whole-body motion control could acquire more diverse whole-body and mobile manipulation skills, allowing it to operate in complex and unstructured environments. The reported motivation is that a humanoid robot must master diverse skills and transition coherently among them to support tasks such as space station maintenance, lunar research station construction, and deep-space exploration.
- Humanoid Robot Multi-Skill Learning Challenge
The paper describes a central challenge for humanoid robot learning: when a single policy model is trained to learn diverse whole-body motion skills at the same time, it is difficult to simultaneously ensure high action completion quality and coherent transitions between different skills. Static skills tend to require the humanoid robot to satisfy static balance conditions. High-dynamic motion skills require the humanoid robot to actively break static balance, execute rapid and flexible movements, and then quickly recover stability. These different motion characteristics can create clear conflicts during policy optimization, making it difficult for a single humanoid robot policy to learn diverse skills efficiently.
The study also identifies a second challenge: the trade-off between action quality and coherence during skill transitions. If transition coherence is emphasized too strongly, the completion quality of an individual skill may decline and the humanoid robot may fail to meet the task requirements. If individual skill completion quality is pursued too narrowly, transitions may become incoherent or unstable, and the humanoid robot may lose balance. The reported work therefore aims to generate a multi-skill policy that allows the humanoid robot to learn diverse whole-body motion skills while maintaining both transition coherence and action completion quality.
To address these issues, the proposed SMILE method integrates goal-conditioned reinforcement learning and generative adversarial imitation learning. The method introduces preference-based rewards into the reward function to guide the policy in accounting for differences among skill characteristics and to reduce the risk of convergence to suboptimal solutions. It also adopts a failure-frequency-based priority sampling method that adaptively increases the sampling probability of samples with poor policy performance. The reported goal is to improve both the efficiency and the performance of learning diverse skills in a single humanoid robot policy model.
- Humanoid Robot System and Task Definition
The humanoid robot platform described in the study is the BICE-Rob system. Its overall architecture is divided into a perception layer, a decision layer, and a motion control layer. The perception layer includes sensors such as stereo cameras, depth cameras, lidar, and inertial measurement units, together with perception computers. The decision layer receives perception data, supports autonomous task instruction generation, processes teleoperation commands, and distributes instructions to the motion control layer. The motion control layer receives commands, solves and generates control instructions, and sends them to the joints of the humanoid robot.
The BICE-Rob humanoid robot is reported to be 1.50 m tall and to weigh 55 kg. Its joints are motor-driven, and it has 55 drive degrees of actuation. The configuration includes a 13-DOA BICE five-finger dexterous hand with 20 degrees of freedom, 14-DOA human-like dual arms, a 2-DOA neck, a 1-DOA waist, a 3-DOA hip, a 1-DOA knee, and a 2-DOA ankle. The paper states that the humanoid robot has strong biomimetic characteristics and is intended to meet both dexterous fine manipulation and high-dynamic motion requirements. The peak torque of the hip and knee joints is reported as 340 N·m, supporting burst force and mobility during high-dynamic motion. The humanoid robot also supports teleoperation for human-in-the-loop fine operation.
The task is modeled as a finite Markov decision process with a state space, action space, reward function, state transition function, and discount factor. The state includes a proprioceptive state and a goal state. At each time step, the humanoid robot observes the state, executes an action determined by the policy, receives a scalar reward, samples the next proprioceptive state according to the transition function, and obtains the next goal state from the reference motion. The optimization objective is to maximize the expected cumulative discounted reward over trajectories generated by the policy. The reported task evaluation metric is success rate, which assesses whether the humanoid robot can maintain whole-body stability and synchronize with the reference motion. A failure is recorded if, at any time, the average Euclidean distance between the humanoid robot joint positions and the corresponding reference joint positions exceeds 0.5 m; otherwise, the episode is considered successful.
The study selected ten representative skills intended to reflect basic requirements for humanoid robot autonomy and self-preservation in space-related tasks. These skills and their application scenarios are summarized in the following table.
No. Skill Application scenario 1 Standing Maintain balance in an upright posture while completing coordinated upper-limb motion or dexterous operation. 2 Squatting Pick up samples from an extraterrestrial surface or maintain equipment at a low interface. 3 Walking Transition among forward, backward, and lateral walking for autonomous charging, emergency stopping, obstacle avoidance, or detouring. 4 Carrying Transfer supply boxes in a space station or transport equipment and construction materials on an extraterrestrial surface. 5 Close inspection Bend or stoop to observe instruments inside a cabin or inspect the surrounding environment. 6 Long-duration cruising Inspect orbital modules or conduct long-distance inspection and exploration on an extraterrestrial planetary surface. 7 Multi-directional picking Pick up objects in different directions and grasp handrails or fixed points in different directions. 8 Sit-to-stand transition Change posture at a control console or in a cockpit, or move from different standing postures to sitting to save energy. 9 Obstacle jumping Quickly jump over trenches, pits, or other obstacles to improve mobility efficiency and obstacle-crossing capability. 10 High-dynamic striding Quickly switch support points and rapidly cross obstacles or uneven terrain. - SMILE Method for Humanoid Robot Whole-Body Motion
The SMILE method is described as a single-model multi-skill efficient imitation learning approach. Its core components include a discriminator, a policy network, a value network, a reference motion sampling method, and a reward shaping method. The discriminator and the policy network are optimized alternately. The discriminator distinguishes trajectories generated by the policy from reference trajectories and continuously updates its ability to discriminate different motion styles. At the same time, the humanoid robot policy is optimized under the combined guidance of imitation rewards, discriminator rewards, preference rewards, and physical constraint rewards. The policy attempts to deceive the discriminator so that its generated motions become more human-like. This adversarial training process is intended to help the policy acquire diverse human-like skills.
Human motion retargeting is used to map human demonstration data to the humanoid robot. The process addresses differences in skeleton topology, drive degree-of-freedom count, and joint axis definitions. First, the shape parameters of a skinned multi-person linear model are optimized so that, in the same reference pose, joint spatial positions are as close as possible to the corresponding joint positions of the BICE-Rob humanoid robot. This reduces motion distortion caused by differences between the human model and the humanoid robot in body-part size proportions. Second, after obtaining a human model matched to the humanoid robot body size, selected joint positions are used as key points to establish spatial position matching between the BICE-Rob humanoid robot and the human motion data. Inverse kinematics is then used to obtain humanoid robot joint angles so that the spatial trajectory of the humanoid robot is consistent with the human demonstration.
The policy network is a six-layer multilayer perceptron with 2048, 1536, 1024, 1024, 512, and 512 neurons. The hidden-layer activation function is SiLU, and the output layer uses linear mapping. The value network uses the same six-layer multilayer perceptron structure. The discriminator network contains two hidden layers with 1024 and 512 neurons, respectively, and uses ReLU activation. The discriminator loss is designed and calculated according to the adversarial motion priors framework, and the discriminator network parameters are updated based on this loss. The policy network is trained with proximal policy optimization.
The proprioceptive state represents the current actual motion state of the BICE-Rob humanoid robot and includes the posture, three-dimensional position, angular velocity, and linear velocity of all joints. The goal state measures the difference between the actual motion and the reference motion. It includes the relative rotation error between the reference posture and the current posture, position differences, linear velocity differences, angular velocity differences, and the next reference posture and position. The action is defined as the target joint position of a proportional-derivative controller. The study does not consider the drive degrees of freedom of the neck, dexterous hand, wrist, or ankle inversion and eversion.
The reward function includes four parts: imitation reward, discriminator reward, preference reward, and physical constraint reward. The imitation reward includes six subterms: joint angle imitation, joint angular velocity imitation, position imitation, posture imitation, velocity imitation, and angular velocity imitation. Each subterm is calculated through an exponential decay function. The discriminator reward is obtained from the discriminator output; when the state distribution generated by the humanoid robot policy is closer to the reference motion state distribution, the discriminator gives a higher reward. The physical constraint reward encourages the humanoid robot to complete tasks with lower energy consumption and reduces high-frequency foot jitter.
The preference reward is introduced to constrain behaviors that are recognized by the discriminator but appear unreasonable from a human perspective. It represents a reward signal reflecting human subjective preference and directly characterizes behavior features that humans consider good or bad. The preference reward includes a tripping penalty term, a sliding penalty term, and a dual-foot orientation reward term. The tripping penalty term penalizes signs of tripping, such as toe contact that causes forward body lean, or actual falls, encouraging the humanoid robot to avoid severe imbalance or falling. The sliding penalty term penalizes noticeable foot slip relative to the ground during support, encouraging the humanoid robot to maintain stability when the sole contacts the ground and avoid friction-based sliding, thereby producing a more human-like gait. The dual-foot orientation reward term penalizes non-vertical foot orientation and encourages the humanoid robot to keep the sole flat against the ground so that the normal direction of the sole is as aligned with gravity as possible. This helps avoid toe contact, single-edge contact, and similar postures.
For initialization and termination, the method uses reference state initialization. A state is randomly sampled from a reference motion segment as the initial state of each humanoid robot episode, reducing policy exploration difficulty and improving learning efficiency. An early termination mechanism is also used during training. When the average deviation between the humanoid robot joint positions and the reference motion exceeds 0.25 m, the current episode is terminated.
- Failure-Frequency-Based Priority Sampling for Humanoid Robot Skills
To avoid catastrophic forgetting, severe fluctuation of learning curves, and convergence difficulty when a single policy model learns multiple skills, the study applies the idea of prioritized experience replay from reinforcement learning to imitation learning and proposes a failure-frequency-based priority sampling method. The basic idea is to automatically evaluate task success rate during training and preferentially use reference motion samples that the policy finds difficult to master. Initially, the policy learns all reference motions through uniform sampling. When the policy repeatedly fails on certain samples, the sampling probability of these difficult samples is continuously increased. Samples that have already been mastered are assigned a smaller sampling probability to consolidate existing skills and avoid complete forgetting.
During training, the sampling probability of a reference motion is computed from the accumulated failure count with a smoothing term, a power parameter that adjusts the priority level, and a normalization over all samples. The failure count is updated with a decay coefficient so that the method can track recent difficulty while retaining prior information. The paper states that this sampling strategy improves the humanoid robot policy’s performance on challenging samples and helps the single policy model learn diverse skills more efficiently.
- Simulation Setup and Humanoid Robot Skill Acquisition
Simulation experiments were conducted in Isaac Gym. The policy runs at 30 Hz, and the simulation runs at 60 Hz. Training used an Intel i9-13900K CPU at 5.80 GHz and an NVIDIA RTX 4090 GPU. To evaluate the SMILE method, the study selected 15 representative diverse human demonstration groups from the AMASS dataset. These groups include stable standing, squatting, carrying, multi-directional picking, stooping for detailed inspection, forward walking, lateral-to-forward walking transition, backward-to-forward walking transition, long-duration cruising, marching in place, obstacle jumping, airborne turning, standing-to-high-dynamic-stride transition, walking-to-high-dynamic-stride transition, and sit-to-stand transition.
The reported visualizations show that the BICE-Rob humanoid robot trained with SMILE acquired diverse motion skills. In multi-directional picking, the humanoid robot used whole-body coordination, including waist rotation, leg adjustment, and foot placement changes, to achieve posture adjustment and turning in different directions. This demonstrates the humanoid robot’s whole-body motion coordination capability. In forward and backward walking transitions, the humanoid robot displayed human-like heel-to-toe gait features during forward walking: the heel of the front foot landed first, the center of gravity gradually shifted, and the rear foot left the ground from heel to toe. When quickly transitioning from forward walking to a stationary state, the humanoid robot used simultaneous toe contact of both feet to achieve an emergency stop and then switched to backward walking, showing human-like gait features in which the front foot lands with the heel and the rear foot changes from toe contact to heel contact.
In lateral-to-forward walking, the humanoid robot initially faced forward and gradually completed the transition from lateral gait to forward gait through whole-body coordination. During turning, one foot was placed flat on the ground while the other foot used toe contact to rotate the body, which is consistent with common human turning characteristics. In stooping for detailed inspection, because the BICE-Rob humanoid robot has only one rotational degree of freedom at the waist, the stooping motion was completed through coordinated whole-body joints, mainly relying on hip flexion to tilt the trunk forward. This posture reflects the humanoid robot’s coordinated whole-body control under limited degrees of freedom.
In high-dynamic striding, the humanoid robot achieved the desired leg-lift height through whole-body coordination. The support leg used toe contact, and the other leg was rapidly lifted. During the motion, the humanoid robot briefly entered a double-foot airborne state and then the support leg touched the ground again to continue the striding motion. This reflects the humanoid robot’s whole-body coordination, dynamic stability, and balance control. In obstacle jumping, the humanoid robot successfully achieved a double-foot airborne state. In a series of static motions, the humanoid robot completed different standing postures, squatting motions, and sitting postures facing different directions. The humanoid robot was able to achieve a distinctive human-like sitting posture in which the upper body leaned backward to maintain stability, placing higher requirements on whole-body balance control and demonstrating the humanoid robot’s control capability for whole-body coordination and posture maintenance in complex static postures.
- Ablation Results for Humanoid Robot Multi-Skill Learning
Ablation experiments were performed to evaluate the influence of core SMILE components, specifically the reference motion sampling method and the preference reward function. The reported learning curves show that the failure-frequency-based priority sampling method has clear advantages in improving the success rate and accelerating training efficiency when the humanoid robot learns diverse skills. The SMILE policy rapidly reached a success rate of about 93.33% during training, whereas the uniform sampling policy generation method reached a maximum of 73.33%, a difference of 20 percentage points. In terms of average episode length, the SMILE method produced longer episodes and faster growth during training. The total reward trend further verified the advantage of the failure-frequency-based priority sampling method: the reward value under SMILE increased significantly faster than under uniform sampling and stabilized at a higher level in the later training stage.
The preference reward ablation showed three main advantages. First, it significantly improved task success rate. The SMILE method achieved a success rate of about 93.33%, compared with a maximum of 53.33% for the method without preference reward, an improvement of 40 percentage points. In terms of average episode length, the SMILE method reached a maximum of 309 steps, while the method without preference reward reached only 177 steps. This indicates that the preference reward effectively reduced the probability of premature failure of the humanoid robot and allowed it to fully explore and exploit a larger state-action space. The average reward curve also showed that the SMILE method consistently achieved higher average reward, indicating that the preference reward worked together with the imitation reward, discriminator reward, and physical constraint reward to guide policy optimization. The combined analysis of success rate, average episode length, and average reward indicates that the SMILE policy had longer episodes and higher average reward, significantly improving the humanoid robot’s success rate in learning diverse whole-body motion skills.
Configuration Reported peak success rate Reported average episode length Reported discriminator reward trend SMILE method 93.33% Up to 309 steps Rapidly exceeded 1.0 at about 580 steps and stabilized around 1.6 near the end of training Without preference reward 53.33% Up to 177 steps Grew slowly and reached only about 0.6 Without priority sampling 73.33% Not specified in the excerpt Not specified in the excerpt Second, the preference reward improved training efficiency and stability. The SMILE method showed a clearly faster and more stable growth trend in success rate, average episode length, and average reward. In the early training stage, at the 1500th evaluation, the SMILE method already reached a success rate of 73.33%. At the 3000th evaluation, it further increased to 86.66%, and at the 4500th evaluation, it reached the final 93.33%. In contrast, the method without preference reward reached only 33.33% at the 1500th evaluation and did not reach its highest success rate of 53.33% until the 16500th evaluation. At the same time, the average episode length and average reward of the SMILE method grew faster and more stably, without obvious decline or fallback, whereas the method without preference reward showed obvious fluctuation near the 1500th evaluation. This indicates that the preference reward effectively reduced policy performance fluctuation during training, reduced the probability of early training failure or local optima, and significantly improved training efficiency and stability.
Third, the preference reward improved the human-like performance of the multi-skill policy. The discriminator reward curve shows that the SMILE method rapidly exceeded a discriminator reward of 1.0 in the early training stage, at about 580 steps, and steadily increased to about 1.6 near the end of training. In comparison, the discriminator reward of the method without preference reward grew slowly and finally reached only about 0.6. The discriminator reward directly reflects the similarity between the generated policy state distribution and the human demonstration data distribution. A higher discriminator reward means that the policy-generated motions are closer to human behavior patterns and have more obvious human-like characteristics. The method without preference reward, lacking explicit human preference guidance, produced motions with a larger gap from human behavior patterns, limiting the human-like characteristics and overall performance of the policy.
Overall, the reported ablation results indicate that the preference reward in SMILE effectively improved task success rate, accelerated training efficiency and stability, and enhanced the human-like characteristics of the policy, comprehensively improving the overall performance of the humanoid robot multi-skill policy.
- Comparison with Goal-Conditioned Reinforcement Learning for Humanoid Robot Control
To examine whether a goal-conditioned reinforcement learning method can resolve the difficulty of balancing motion coherence and completion quality when a humanoid robot learns diverse skills through reward function adjustment and policy fine-tuning, the study designed a comparison with a reinforcement learning method. The comparison included a method trained according to the original reinforcement learning reward function and hyperparameters to obtain an initial policy for the BICE-Rob humanoid robot learning diverse motion skills. Then, on the basis of the initial policy, the reward function was adjusted to be consistent with the imitation reward, constraint reward, and preference reward used in SMILE, while still excluding the discriminator reward, and policy fine-tuning was performed. This second stage aimed to examine whether, after the humanoid robot had acquired certain basic skills, optimizing only the reward function through policy fine-tuning could enable the reinforcement learning method to master more complex, diverse, and highly dynamic motion skills.
The learning curves reported for the comparison show that the average episode length and reward value of the initial policy exhibited a slow upward trend, and the policy gradually stabilized in the later training stage, but the final average episode length and corresponding reward value were both low. This indicates that the initial policy could master only a small number of skills. Compared with the initial policy, the second-stage learning curve showed a clear difference: in early training, both episode length and reward value increased rapidly and clearly exceeded those of the initial policy. This indicates that the fine-tuning method based on the initial policy could quickly improve the humanoid robot’s imitation performance for diverse skills. However, overall, the second-stage learning curve fluctuated more severely. This phenomenon indicates that, without the discriminator reward constraining action performance, the policy’s ability to learn diverse skills was limited.
According to the reported forward-running visualization of the policy generated by the goal-conditioned reinforcement learning method, the BICE-Rob humanoid robot adopted a conservative standing-first style in the initial policy, with a certain ability to imitate static skills. It prioritized lower-limb stability and minimized lateral or large center-of-gravity shifts. This strategy avoided small stomping-step adjustments, and the overall motion was smoother. However, it had several shortcomings: it overemphasized standing still in place and performed poorly in imitating lateral movement, turning, or large dynamic motions; the lower-limb motion was relatively rigid and difficult to adapt to actions requiring larger steps or rapid foot adjustment, such as lateral walking, obstacle jumping, and multi-directional object picking; and action completion was insufficient, often using waist rotation or large arm swings to replace leg movement, resulting in imitation failure.
After the second-stage fine-tuning training, the BICE-Rob humanoid robot was able to attempt more diverse action sequences. However, because it lacked the discriminator reward’s explicit constraint on imitation quality, the fine-tuned policy had several problems. First, action completion was clearly reduced. When demonstrating motion skills with significant dynamic features, the humanoid robot found it difficult to accurately reproduce key action features and tended to use small shuffling steps or body swaying as substitutes. For example, obstacle jumping should have the key feature of both feet leaving the ground simultaneously, but the humanoid robot replaced the airborne process with continuous small shuffling steps. This action deformation phenomenon indicates that the lack of effective discriminator reward constraints can easily lead to a clear decline in humanoid robot action completion.
Second, action execution efficiency was significantly reduced. When executing tasks involving larger strides or rapid movement, such as walking, high-dynamic striding, long-duration cruising, and stooping for detailed inspection, the humanoid robot often showed too-small steps and frequent foot adjustment, resulting in unsmooth action transitions and slow speed. This severely limited the overall performance of the humanoid robot in multi-skill tasks. Third, original actions degraded or were forgotten. Because the reward function in the fine-tuning stage reduced the reward for standing-first behavior and large-step movement, the BICE-Rob humanoid robot showed obvious degradation when executing actions that the initial policy had already mastered relatively well. For example, forward and backward walking, which the initial policy could perform smoothly with large strides, degraded into inefficient small shuffling steps after fine-tuning. This phenomenon indicates that although the humanoid robot acquired new skills, it often implemented them in non-standard and inefficient ways, leading to a trade-off problem in the overall policy.
The comparison results therefore indicate that goal-conditioned reinforcement learning alone, through reward function adjustment and policy fine-tuning, is insufficient to effectively resolve the conflicts caused by different skill characteristics during policy optimization. The humanoid robot found it difficult to balance skills requiring stability and balance, such as standing, carrying, and walking, with skills requiring flexibility and large dynamic adjustment, such as multi-directional object picking, obstacle jumping, and high-dynamic striding. As a result, action completion and movement efficiency decreased. The paper attributes the root cause to the lack of discriminator reward as an explicit constraint on action imitation quality, which makes it easy for the humanoid robot to execute tasks in a reduced-completion or inefficient mode, limiting the overall performance of the policy.
- Implications for Humanoid Robot Space Applications
The reported SMILE method is positioned as a contribution to whole-body motion strategy generation for humanoid robots. By combining goal-conditioned reinforcement learning with generative adversarial imitation learning, the method aims to improve the humanoid robot’s adaptability to different skill characteristics and to produce policies with higher action completion quality and coherence during multi-skill transitions. Compared with policy generation methods without preference reward, the reported approach significantly improved task success rate through preference reward guidance, improved training efficiency and stability, and enhanced the human-like performance of the policy.
The study states that the proposed SMILE method may provide a technical foundation for deploying multi-skill humanoid robot whole-body motion strategies in future space station and extraterrestrial exploration mission scenarios. Future work is expected to focus on simulation-to-reality transfer methods and to use a teacher-student paradigm to further improve the adaptability and engineering feasibility of the proposed method on the real BICE-Rob humanoid robot platform. The paper also notes that the SMILE method has good scalability. Although validation was performed on a relatively limited skill set, a similar policy generation approach could be applied to larger-scale or more diverse datasets. Because the preference reward and failure-frequency-based priority sampling method can play a guiding role across different skills, the method is expected to maintain high learning efficiency and policy performance even when the number of skills increases further.
Future research is also expected to explore how a humanoid robot can respond to continuously increasing new skill learning demands while ensuring that existing skills are retained, especially when conflicts among skill characteristics become more significant. The reported work therefore contributes to the broader effort to develop humanoid robot policies that can learn diverse whole-body motion skills in a single model, transition smoothly among skills, and support complex autonomous tasks in demanding environments.
- Humanoid Robot Skill Set and Reported Evaluation Summary
The reported study provides a structured evaluation of a humanoid robot learning framework for multi-skill whole-body motion. The humanoid robot is trained in a single policy model using human demonstration data. The SMILE method uses adversarial imitation learning to encourage human-like motion, preference rewards to encode human judgment about unreasonable or unstable behavior, and failure-frequency-based priority sampling to focus learning on difficult reference motions. The reported simulation results indicate that the humanoid robot acquired standing, squatting, walking, carrying, close inspection, long-duration cruising, multi-directional picking, sit-to-stand transition, obstacle jumping, and high-dynamic striding skills, and that the policy achieved coherent transitions between different skills.
The reported success rate of 93.33% under SMILE contrasts with 53.33% when the preference reward was removed and 73.33% when the failure-frequency-based priority sampling method was removed. These ablation results are presented as evidence that both components contribute to the performance of the humanoid robot multi-skill policy. The study also reports that the discriminator reward under SMILE exceeded 1.0 at about 580 steps and stabilized around 1.6, while the method without preference reward reached only about 0.6. The average episode length under SMILE reached 309 steps, compared with 177 steps without preference reward. Training milestones show that SMILE reached 73.33% success at the 1500th evaluation, 86.66% at the 3000th evaluation, and 93.33% at the 4500th evaluation, whereas the method without preference reward reached 33.33% at the 1500th evaluation and did not reach its peak 53.33% until the 16500th evaluation.
The paper concludes that the SMILE method improves the humanoid robot’s adaptability to different skill characteristics, enables higher action completion quality and coherence during multi-skill transitions, and provides a new approach for humanoid robot multi-skill imitation learning. The reported findings are based on simulation experiments in Isaac Gym, and future work is directed toward simulation-to-reality transfer and deployment on the real BICE-Rob humanoid robot platform. The overall contribution is a single-model method for generating multi-skill whole-body motion strategies for humanoid robots, with reported evidence that preference-based reward shaping and failure-frequency-based priority sampling can help a humanoid robot learn diverse skills while mitigating skill conflicts and catastrophic forgetting.