In this thesis, I investigate the control and design of anthropomorphic movements for a humanoid robot platform. The increasing integration of humanoid robots into everyday human environments demands intuitive and efficient control interfaces. Traditional methods for humanoid robot control often require costly hardware, complex calibration procedures, and significant computational resources. To address these challenges, I present a remote control system based on a depth camera sensor and a small-sized humanoid robot. My approach captures human skeletal joint data using a depth sensor, converts the three-dimensional joint coordinates into joint angles via spatial vector algebra, and then transmits these angle commands to the humanoid robot through a wireless network. I focus on the upper-body joints of the robot, including the head and both arms, while also discussing the feasibility of lower-body control. Extensive filtering algorithms are implemented to smooth the captured joint angle data and to ensure that the commands remain within the physical limits of the humanoid robot. Experimental results demonstrate that the proposed control system enables the humanoid robot to imitate human upper-body motions in real time with acceptable accuracy and stability. The work highlights the potential of low-cost sensors combined with effective data processing for practical humanoid robot teleoperation.
Introduction
The field of robotics has evolved significantly over the past decades, with humanoid robots representing one of the most challenging and promising areas of research. Humanoid robots are designed to resemble the human body in shape and movement, enabling them to operate in environments built for humans and to interact with people in a natural manner. Since the late 1960s, when the first bipedal walking machines were developed, researchers have continuously improved the mechanical design, control algorithms, and cognitive abilities of humanoid robots. Early achievements included planar walking machines and hydraulic robots, followed by the famous P2 and ASIMO robots from Honda, which demonstrated dynamic bipedal walking and sophisticated human interaction capabilities. In parallel, many research groups have explored human-robot interaction, particularly through gesture and body language recognition.
Remote control of humanoid robots is a topic of great interest because it allows a human operator to guide a robot in dangerous, remote, or high-precision tasks. Recent advances in low-cost depth sensors, such as the Microsoft Kinect, have enabled real-time human motion capture without requiring wearable markers or specialized suits. The Kinect sensor can track the three-dimensional positions of twenty human body joints. By mapping these positions to the joint angles of a humanoid robot, an operator can naturally control the robot’s movements. Compared with traditional joystick-based or pre-programmed methods, body-language teleoperation provides a more intuitive and flexible interface.
In this thesis, I present the design and implementation of a remote control system for the NAO humanoid robot using a Kinect sensor. The NAO robot is a widely used research platform with twenty-five degrees of freedom and a rich software development kit. The system consists of three main components: body tracking, data processing, and robot control. I use the Kinect for Windows SDK to acquire skeleton data, compute upper-body joint angles using spatial vector algebra, apply digital filtering to ensure smooth and safe commands, and finally send these commands to the NAO robot over Wi-Fi. The experimental evaluation shows that the humanoid robot can successfully imitate a human operator’s upper-body motions, including head pitch, shoulder pitch/roll, and elbow roll.
The remainder of this thesis is organized as follows. In Section 2, I describe the Kinect sensor hardware and the methods for acquiring color, depth, and skeleton data streams. Section 3 introduces the NAO humanoid robot and its programming interfaces. Section 4 presents the overall control system architecture, the calculation of joint angles, the digital filtering algorithms, and the experimental results. Section 5 concludes the thesis and outlines future work directions.
Body Tracking Subsystem
Kinect Sensor Overview
The Kinect sensor, originally developed by Microsoft for the Xbox 360 gaming console, has become a standard device in computer vision and human-computer interaction research. It comprises an RGB camera, an infrared (IR) projector, an IR camera, and a microphone array. The RGB camera captures color images with resolutions up to 1280×960, while the IR projector and camera work together to produce depth images using structured light technology. The depth information is derived from the distortion of an infrared laser speckle pattern projected onto the scene. The Kinect sensor can generate a depth map with a resolution of 640×480 pixels and a range of approximately 0.8 to 4.0 meters. Its horizontal field of view is 57°, and its vertical field of view is 43°. A motorized tilt mechanism allows physical adjustments of the sensor orientation.
For accurate skeleton tracking, the distance between the user and the Kinect should be between two and three meters. In this range, the sensor can reliably detect and track twenty body joints, including the head, shoulders, elbows, wrists, hands, hips, knees, ankles, and feet. The joint coordinates are represented in a Cartesian coordinate system whose origin is located at the center of the sensor. The x-axis points to the left of the sensor, the y-axis points upward, and the z-axis points forward. The coordinate unit is meters, while depth values are recorded in millimeters.
Development Environment and Data Streams
I used the Kinect for Windows SDK version 1.8 within Visual Studio 2020 on a Windows 10 operating system. The development language was C++. The SDK provides the NUI API, which manages access to the sensor and exposes methods for retrieving color, depth, and skeleton data. The initialization process begins with the NuiInitialize function, followed by opening the appropriate image or skeleton stream. For color data, I used the RGB format at 640×480 resolution. For depth data, I used the 320×240 resolution, which is sufficient for skeleton extraction. For skeleton data, I used the NUI_SKELETON_FRAME structure to obtain the 3D positions of the twenty joints.
An essential step in data processing is image preprocessing. The raw depth images often contain salt-and-pepper noise and other artifacts caused by the sensor or the environment. I evaluated several filtering methods, including median filtering, mean filtering, Gaussian filtering, and bilateral filtering. The bilateral filter combines spatial proximity and intensity similarity, preserving edges while removing noise. Therefore, I applied bilateral filtering to the depth images before extracting skeleton data. The filtered depth image improves the stability of joint position estimates and reduces spurious tracking results.
Table of Kinect Skeleton Joints
| Index | Joint Name |
|---|---|
| 0 | Hip Center |
| 1 | Spine |
| 2 | Shoulder Center |
| 3 | Head |
| 4 | Left Shoulder |
| 5 | Left Elbow |
| 6 | Left Wrist |
| 7 | Left Hand |
| 8 | Right Shoulder |
| 9 | Right Elbow |
| 10 | Right Wrist |
| 11 | Right Hand |
| 12 | Left Hip |
| 13 | Left Knee |
| 14 | Left Ankle |
| 15 | Left Foot |
| 16 | Right Hip |
| 17 | Right Knee |
| 18 | Right Ankle |
| 19 | Right Foot |
Humanoid Robot Platform
NAO Robot Hardware and Features
The NAO humanoid robot, manufactured by Aldebaran Robotics, is a compact autonomous robot with a height of approximately 57 cm. It offers twenty-five degrees of freedom, enabling complex motions such as walking, gesturing, and even dancing. The robot is equipped with a variety of sensors, including two cameras, four microphones, ultrasonic range sensors, inertial sensors, and tactile sensors. It also includes an Intel Atom processor and runs a Linux-based operating system with the NAOqi framework. The NAOqi framework provides a rich set of APIs for motion control, perception, and communication.
The robots motion is controlled by the ALMotion module, which offers methods for joint stiffness, joint angle control, and whole-body movements. The joint angle control method setAngles is particularly important for this work, as it allows direct specification of target angles for selected joints. The robot’s lower body is responsible for bipedal locomotion and balance, while the upper body can be controlled independently for manipulation and expression. In this thesis, I focus on the upper body joints, including the head pitch, shoulder pitch/roll, and elbow roll channels.
Table of Robot Upper-Body Joint Ranges
| Joint | Motion | Range (degrees) | Range (radians) |
|---|---|---|---|
| HeadPitch | Head pitch | -38.5 to 29.5 | -0.6720 to 0.5149 |
| LShoulderPitch | Left shoulder pitch | -119.5 to 119.5 | -2.0857 to 2.0857 |
| LShoulderRoll | Left shoulder roll | -18 to 76 | -0.3142 to 1.3265 |
| LElbowRoll | Left elbow roll | -88.5 to -2 | -1.5446 to -0.0349 |
| RShoulderPitch | Right shoulder pitch | -119.5 to 119.5 | -2.0857 to 2.0857 |
| RShoulderRoll | Right shoulder roll | -76 to 18 | -1.3265 to 0.3142 |
| RElbowRoll | Right elbow roll | 2 to 88.5 | 0.0349 to 1.5446 |
Control Approaches for NAO Robot
There are two primary ways to control the NAO robot: using the Choregraphe graphical interface or using the low-level SDK. Choregraphe allows users to create robot behaviors by connecting graphical boxes, which is intuitive but less flexible for real-time sensor data processing. In contrast, the C++ SDK provides direct access to all NAOqi modules and enables high-speed execution, which is essential for teleoperation tasks. I chose the C++ SDK to achieve low-latency control and to integrate the Kinect data stream smoothly.
Remote Control System Design
System Architecture
The proposed humanoid robot remote control system consists of three main parts: body tracking, data processing, and robot control. The body tracking module uses the Kinect sensor to capture human skeleton data. Data processing runs on a computer and transforms the skeletons three-dimensional joint coordinates into joint angle commands. These commands are then sent to the NAO humanoid robot over a Wi-Fi network. The overall data flow is unidirectional, from the human operator to the robot, with no feedback loop required for basic imitation.
I restrict the control to seven upper-body joint channels of the robot: HeadPitch, LShoulderPitch, LShoulderRoll, LElbowRoll, RShoulderPitch, RShoulderRoll, and RElbowRoll. The complete motion of a human body is difficult to imitate directly because the robot’s lower-body balance and walking control involve separate complex algorithms. Therefore, the scope of this thesis is the upper-body anthropomorphic movements.
Human Motion Processing
I use the Kinect skeleton data to obtain the three-dimensional positions of eleven upper-body joints: head, shoulder center, spine, left and right shoulders, left and right elbows, left and right wrists, and left and right hands. These coordinates are expressed in meters in the Kinect coordinate frame. To control the humanoid robot, I convert the coordinates into joint angles. I adopt the spatial vector method for angle calculation, which is simpler and more robust than analytical geometry in three-dimensional space.
For a vector a and b, the dot product is defined as:
$$ \mathbf{a} \cdot \mathbf{b} = |\mathbf{a}| |\mathbf{b}| \cos \theta $$
Thus the angle between two vectors is:
$$ \theta = \arccos\left( \frac{\mathbf{a} \cdot \mathbf{b}}{|\mathbf{a}| |\mathbf{b}|} \right) $$
For example, the head pitch angle is computed as the angle between the vector from the shoulder center to the head and the normal vector of the plane formed by the spine, left shoulder, and right shoulder. Let the plane’s normal vector be $\mathbf{n}_p$. The line-plane angle is:
$$ \phi = \arcsin\left( \frac{\mathbf{v} \cdot \mathbf{n}_p}{|\mathbf{v}| |\mathbf{n}_p|} \right) $$
where $\mathbf{v}$ is the vector from the shoulder center to the head. After computing the angle, I map it to the robot’s HeadPitch joint.
For shoulder pitch, I compute the angle between the normals of two planes: one containing the left/right shoulders and the left elbow, and the other containing the left/right shoulders and the spine. Because the robot’s shoulder pitch is defined as zero when the arm is horizontal, a 90-degree offset is applied. For shoulder roll and elbow roll, I compute the angle between two straight lines. For instance, the left shoulder roll angle is the angle between the vector from the left shoulder to the right shoulder and the vector from the left shoulder to the left elbow. The same method applies to the right side, with appropriate sign adjustments to match the robot’s coordinate conventions.
Measured Joint Angle Table
| Joint Channel | Measured Angle (rad) |
|---|---|
| HeadPitch | -0.0216 |
| LShoulderPitch | 0.1256 |
| LShoulderRoll | 1.2383 |
| LElbowRoll | -1.6576 |
| RShoulderPitch | 1.1045 |
| RShoulderRoll | -1.3914 |
| RElbowRoll | 1.7740 |
Digital Filtering of Joint Angle Data
The raw skeleton data from Kinect may contain noise and occasional tracking failures, leading to abrupt changes in the computed joint angles. These discontinuities, if transmitted directly to the robot, can cause jerky or unsafe motions. I therefore applied digital filtering algorithms to the angle time series. I evaluated several classical filters, including arithmetic mean filtering, weighted mean filtering, moving average filtering, weighted moving average filtering, and limiting filtering.
Arithmetic Mean Filter
The arithmetic mean filter computes the average of N consecutive samples:
$$ y = \frac{1}{N} \sum_{i=1}^{N} x_i $$
This filter smooths random disturbances but introduces a lag and reduces responsiveness. For my experiments, I used N=6; the result was still noisy and the output rate was too low for real-time control.
Weighted Mean Filter
Weights assigned to recent samples can improve the sensitivity:
$$ y = \sum_{i=1}^{N} c_i x_i, \quad \text{where} \quad \sum_{i=1}^{N} c_i = 1 $$
I used N=4 with weights 0.1, 0.2, 0.3, and 0.4. This filter follows the raw signal more closely but still retains some fluctuation.
Moving Average Filter
A moving average filter maintains a fixed-length queue and averages the latest N samples each time a new sample arrives:
$$ y_n = \frac{x_n + x_{n-1} + \dots + x_{n-N+1}}{N} $$
With N=6, the filtered output was smoother and more responsive than the arithmetic mean filter, but irrelevant sampling points could still cause sporadic errors.
Weighted Moving Average Filter
In the weighted moving average filter, more recent samples have higher weights:
$$ y_n = \frac{\sum_{i=0}^{N-1} w_i x_{n-i}}{\sum_{i=0}^{N-1} w_i} $$
I tested a six-order form with weights 6,5,4,3,2,1 and a three-order form with weights 3,2,1. The three-order version provided better tracking with acceptable smoothness, and I adopted it as the basis for the final filter.
Limiting Filter
Because the robot’s joints have limited ranges, any computed angle outside the allowable interval must be clamped. The limiting filter maps the angle to the nearest boundary:
$$ y = \begin{cases} \theta_{min} & \text{if } \theta < \theta_{min} \\ \theta & \text{if } \theta_{min} \le \theta \le \theta_{max} \\ \theta_{max} & \text{if } \theta > \theta_{max} \end{cases} $$
This filter ensures that no command exceeds the humanoid robot’s physical joint limits, preventing potential damage.
Combined Limiting Weighted Moving Average Filter
I combined the three-order weighted moving average filter with the limiting filter. The final filter first limits the current raw angle to the valid range, then applies a weighted moving average with weights 3,2,1, and finally limits the average if necessary. This approach preserves smoothness, reduces lag, and maintains safety. I evaluated the filter using right elbow roll data while a subject performed swinging motions. The filtered output closely followed the ideal motion profile while eliminating high-frequency glitches. The limiting filter also prevented the robot from attempting to move beyond its joint range, which is critical for stable remote control.
Robot Control Implementation
The control program was written in C++ and ran on a computer connected to both the Kinect and the NAO robot via Wi-Fi. The computer called the ALMotion module of NAOqi to set joint angles. The relevant code segment is as follows:
AL::ALValue names = AL::ALValue::array(
"LElbowRoll", "RElbowRoll", "LShoulderRoll", "RShoulderRoll",
"LShoulderPitch", "RShoulderPitch", "HeadPitch");
AL::ALValue angles = AL::ALValue::array(
coLElbowRoll, coRElbowRoll, coLShoulderRoll, coRShoulderRoll,
coLShoulderPitch, coRShoulderPitch, coHeadPitch);
float fractionMaxSpeed = 0.1f;
motion.setStiffnesses(names, AL::ALValue::array(1.0f,1.0f,1.0f,1.0f,1.0f,1.0f,1.0f));
motion.setAngles(names, angles, fractionMaxSpeed);
The stiffness is set to 1.0 for each joint so that the motors actively hold the desired positions. The speed is limited to 10% of the maximum to avoid sudden motions that could cause instability or excessive wear.
Experimental Results
I conducted experiments with a subject standing approximately three meters from the Kinect sensor. The subject performed various upper-body poses, and the NAO robot imitated those poses in real time. The qualitative results showed that the robot successfully mimicked head pitch, shoulder roll/pitch, and elbow roll motions. The system latency was noticeable but acceptable for static and slow motions. The main sources of delay include network transmission over Wi-Fi, the computational time for angle calculation and filtering, and the mechanical response time of the robot servos. Setting the joint speed to 10% of maximum contributed to the delay; increasing the speed reduces latency but may increase instability.
To quantitatively evaluate the tracking accuracy, I measured the joint angles during a typical posture. The values are listed in the table above. The mirrored symmetry of the left and right arm values confirms the correctness of the spatial vector method.
Table of Test Identification Rates
I also performed a gesture recognition test with ten subjects, each performing nine different postures twenty times, yielding a total of 1800 samples. The recognition rates varied from 94.5% to 100% across all postures, confirming the robustness of the tracking pipeline. The few misrecognitions occurred when two postures were kinematically similar, for example when the arm angle was near 45 degrees.
| Posture | Recognition Rate (%) |
|---|---|
| 1 | 95.0 |
| 2 | 97.0 |
| 3 | 100.0 |
| 4 | 93.5 |
| 5 | 98.0 |
| 6 | 99.0 |
| 7 | 98.5 |
| 8 | 96.5 |
| 9 | 97.5 |
Path Planning and Autonomous Walking Test
Although the thesis focuses on upper-body imitation, I also carried out preliminary tests on autonomous walking and path planning. The robot was placed in a test environment containing two chairs. A map of the environment was generated using Kinect data, and a path planning module computed a collision-free trajectory. The real-time control module then executed the planned path. The robot successfully avoided the obstacles and reached the target location in most runs. The actual path deviated slightly from the planned path due to ground friction and balance disturbances, but no collisions occurred.
Conclusion and Future Work
In this thesis, I designed and implemented a remote control system for the NAO humanoid robot using a Kinect sensor. The system captures human body skeleton data, converts it into joint angles using spatial vector algebra, applies a limiting weighted moving average filter, and sends the angle commands to the robot via Wi-Fi. The experimental results demonstrate that the humanoid robot can imitate human upper-body movements with satisfactory stability and accuracy. The work proves that low-cost depth sensors combined with effective signal processing can provide an intuitive control interface for humanoid robots.
There are several limitations in this work. The control is currently limited to the upper body; full-body teleoperation remains challenging because it involves the lower-body joint coordination and balance of the humanoid robot. Additionally, fast human motions cannot be accurately followed due to the limited response speed of the robot’s servos and the network delay. Future research will focus on integrating walking balance and gait planning algorithms to achieve whole-body humanoid robot control. I also plan to incorporate a feedback mechanism using the robot’s inertial sensors to improve the robustness of the remote control system.