The operational reliability of space robotic systems is paramount for the success of long-duration, complex on-orbit servicing (OOS) missions. Among these systems, the dexterous robotic hand represents a critical advancement, transitioning from single-function tools to multi-fingered, highly articulated end-effectors capable of performing intricate tasks such as grasping, screwing, and manipulating interfaces. However, the space environment presents a uniquely hostile set of challenges—extreme thermal cycles, intense ionizing radiation, plasma interactions, and persistent micro-vibrations—that can severely degrade sensor performance. Joint angle sensors within a dexterous robotic hand are particularly vulnerable, susceptible to faults including signal drift, distortion, temporary loss, or permanent failure. Such sensor faults directly compromise the integrity of feedback control loops, leading to degraded performance or complete mission failure. Traditional fault-tolerant approaches, heavily reliant on precise analytical models or fixed-threshold detection, struggle with the time-varying dynamics of a dexterous robotic hand and the unpredictable nature of on-orbit anomalies. This paper addresses this critical gap by proposing a novel, data-driven framework for joint sensor fault detection and reconstruction, ensuring continuous and reliable operation.
The proposed methodology is centered on an Online Conditional Variational Autoencoder (Online-CVAE) synergistically integrated with a Bayesian Adaptive Threshold (BAT) mechanism. The core innovation lies in creating a self-adapting system that learns the nominal kinematics of the dexterous robotic hand, quantifies its own predictive uncertainty, and dynamically adjusts its fault-detection boundaries in real-time. The CVAE provides a probabilistic foundation, learning to reconstruct joint angles from correlated sensory inputs (like tendon displacements) while outputting both a mean prediction and a variance representing epistemic uncertainty. The BAT mechanism leverages this uncertainty to perform Bayesian inference on the prediction error, allowing the fault detection threshold to expand or contract intelligently based on the model’s confidence. Furthermore, to combat the inevitable drift in system characteristics over long missions, an online incremental learning framework based on Elastic Weight Consolidation (EWC) is embedded, enabling the CVAE model to update its parameters with new, fault-free data without catastrophically forgetting previously learned knowledge. This holistic approach moves beyond static models, offering a resilient and adaptive solution for the sensor fault tolerance of a dexterous robotic hand in the demanding space environment.
Methodological Framework
1. Probabilistic Kinematics Modeling with CVAE
The foundation of our fault-tolerant system is a Conditional Variational Autoencoder (CVAE) designed to learn the complex, nonlinear mapping between the actuator space (tendon displacements) and the configuration space (joint angles) of the dexterous robotic hand. Unlike deterministic models, the CVAE learns a *probability distribution* over possible joint states, conditioned on the observed actuator inputs. This provides not only a reconstructed joint angle but also a crucial measure of the model’s confidence for each prediction.
Network Architecture: The implemented CVAE features a specialized structure for time-series data. The encoder takes a sequence of measured joint angles $j_{1:T}$ and tendon displacements $m_{1:T}$ over a sliding window. A bidirectional LSTM layer captures temporal dependencies, followed by fully connected layers that map the learned features to the parameters of a latent Gaussian distribution: a mean vector $\boldsymbol{\mu}_z$ and a log-variance vector $\log \boldsymbol{\sigma}^2_z$. The latent variable $\mathbf{z}$ is sampled using the reparameterization trick:
$$\mathbf{z} = \boldsymbol{\mu}_z + \boldsymbol{\sigma}_z \odot \boldsymbol{\epsilon}, \quad \boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$$
The decoder is architected with two parallel output heads. Given the sampled latent variable $\mathbf{z}$ and the condition sequence $m_{1:T}$, it outputs the parameters of the reconstruction distribution: the mean predicted joint angle $\hat{\boldsymbol{\mu}}_m$ and its associated variance $\hat{\boldsymbol{\sigma}}^2_m$. This decoupled design explicitly separates state estimation from uncertainty quantification.
Loss Function: The model is trained by maximizing the Evidence Lower Bound (ELBO). The total loss function $\mathcal{L}$ is a weighted sum of three components:
$$\mathcal{L} = \mathcal{L}_{recon} + \lambda_{kl} \mathcal{L}_{kl} + \lambda_{var} \mathcal{L}_{var}$$
The reconstruction loss $\mathcal{L}_{recon}$ is the negative log-likelihood of the measured joints under the predicted Gaussian distribution:
$$\mathcal{L}_{recon} = \frac{1}{T} \sum_{t=1}^{T} \left[ \frac{(j_t – \hat{\mu}_{m,t})^2}{2\hat{\sigma}^2_{m,t}} + \frac{1}{2} \log \hat{\sigma}^2_{m,t} \right]$$
The Kullback-Leibler divergence $\mathcal{L}_{kl}$ regularizes the latent space, encouraging the encoder’s distribution $q_\phi(\mathbf{z} | m_{1:T}, j_{1:T})$ to align with a standard normal prior $p(\mathbf{z})$:
$$\mathcal{L}_{kl} = D_{KL}(q_\phi(\mathbf{z} | m_{1:T}, j_{1:T}) \, || \, \mathcal{N}(0, \mathbf{I}))$$
A variance regularization term $\mathcal{L}_{var}$ prevents the model from trivially minimizing $\mathcal{L}_{recon}$ by predicting excessively large variances. It penalizes both the magnitude and the abrupt changes in predicted variance:
$$\mathcal{L}_{var} = \alpha_1 \cdot \frac{1}{T}\sum_{t=1}^{T} (\hat{\sigma}^2_{m,t})^2 + \alpha_2 \cdot \sum_{t=2}^{T} \left( \log \frac{\hat{\sigma}^2_{m,t}}{\hat{\sigma}^2_{m,t-1}} \right)^2$$
The training parameters, optimized for the kinematics of the dexterous robotic hand, are summarized below.
| Parameter | Value | Description |
|---|---|---|
| Optimizer | AdamW | $\beta_1=0.9, \beta_2=0.999$ |
| Initial Learning Rate | $1 \times 10^{-3}$ | With cosine annealing scheduler |
| Batch Size | 64 | With gradient accumulation (steps=4) |
| Sequence Length (T) | 10 | Corresponding to 0.1s at 100 Hz |
| Early Stopping Patience | 15 epochs | Based on validation loss |
| KL Weight ($\lambda_{kl}$) | Annealed | From $10^{-5}$ to $10^{-2}$ |
| Var Reg Weight ($\lambda_{var}$) | 0.01 |
2. Bayesian Adaptive Threshold for Fault Detection
With a trained CVAE providing predictions $\hat{\mu}_m$ and uncertainties $\hat{\sigma}^2_m$, the next step is to detect anomalies in the real sensor stream $j_t$. The core idea is to monitor the reconstruction error $e_t = j_t – \hat{\mu}_{m,t}$. A fixed threshold $[-\tau, \tau]$ is inadequate for space applications due to changing conditions. Even an Exponentially Weighted Moving Average (EWMA) threshold, which updates $\mu_t$ and $\sigma_t$ via:
$$\mu_t = (1-\alpha)\mu_{t-1} + \alpha e_t, \quad \sigma^2_t = (1-\alpha)\sigma^2_{t-1} + \alpha (e_t – \mu_{t-1})^2$$
lacks a principled way to incorporate the model’s confidence. Our Bayesian Adaptive Threshold (BAT) method directly addresses this.
Bayesian Update Rule: We treat the true error mean $\mu_t$ as a random variable to be estimated. At each timestep, we have a prior belief from the previous step: $p(\mu_{t-1}) \sim \mathcal{N}(\hat{\mu}_{t-1}, \hat{\sigma}^2_{t-1})$. We receive a new observation $e_t$ with a likelihood informed by the CVAE’s uncertainty: $p(e_t | \mu_t) \sim \mathcal{N}(\mu_t, \hat{\sigma}^2_{m,t})$. Applying Bayes’ theorem yields a Gaussian posterior:
$$p(\mu_t | e_t) \propto p(e_t | \mu_t) \cdot p(\mu_{t-1}) \sim \mathcal{N}(\hat{\mu}_t, \hat{\sigma}^2_t)$$
The posterior parameters are computed as:
$$\hat{\mu}_t = \frac{ \hat{\sigma}^2_{m,t} \cdot \hat{\mu}_{t-1} + \hat{\sigma}^2_{t-1} \cdot e_t }{ \hat{\sigma}^2_{m,t} + \hat{\sigma}^2_{t-1} }, \quad \hat{\sigma}^2_t = \frac{ \hat{\sigma}^2_{m,t} \cdot \hat{\sigma}^2_{t-1} }{ \hat{\sigma}^2_{m,t} + \hat{\sigma}^2_{t-1} }$$
To prevent the variance from collapsing to zero over time and to allow adaptation, we introduce a forgetting factor $\beta$ (e.g., 0.95) by artificially inflating the prior variance before the update: $\hat{\sigma}^2_{t-1} \leftarrow \hat{\sigma}^2_{t-1} / \beta$.
Adaptive Threshold Calculation: The dynamic fault detection threshold at time $t$ is then defined as:
$$\text{Threshold}_t = \hat{\mu}_t \pm k_t \cdot \hat{\sigma}_t$$
The key innovation is the adaptive scaling factor $k_t$, which modulates based on the CVAE’s confidence (inverse variance $w_t = 1/\hat{\sigma}^2_{m,t}$):
$$k_t = k_0 + \Delta k \cdot \tanh(1 – w_t / \bar{w})$$
where $k_0$ is a baseline coefficient (e.g., 3), $\Delta k$ is a maximum expansion factor, and $\bar{w}$ is a normalization constant. When the CVAE is confident ($w_t$ is high), $k_t \approx k_0$, maintaining a sensitive, narrow threshold. When the CVAE is uncertain ($w_t$ is low), $k_t$ increases, widening the threshold to reduce false alarms. This mechanism makes the fault detection for the dexterous robotic hand inherently robust to model uncertainty.
3. Online Incremental Learning with EWC
The kinematics of a dexterous robotic hand may gradually change on-orbit due to factors like lubricant degradation, tendon wear, or thermal deformation. A static model trained on ground data will inevitably experience performance drift. To maintain accuracy, the CVAE model must be updated online using newly acquired, fault-free data from the mission. However, naive fine-tuning leads to catastrophic forgetting of previously learned knowledge.
We implement an Online-CVAE framework using Elastic Weight Consolidation (EWC). When the BAT mechanism confirms a data point $(m_t, j_t)$ is fault-free and its information content (measured by the entropy of the prediction) is high, it is stored in a Dynamic Incremental Buffer (DIB). Once the DIB reaches capacity $N_{buf}$, an incremental learning step is triggered.
EWC Regularization: EWC adds a quadratic penalty to the loss function during incremental training, preventing important parameters for previous tasks from changing drastically. The importance of each parameter $\theta_i$ is quantified by the diagonal of the Fisher Information Matrix $F_{ii}$, estimated on the original training data. The combined loss for online update is:
$$\mathcal{L}_{online} = \mathcal{L}_{new} + \lambda_{ewc} \sum_i F_{ii} (\theta_i – \theta_i^*)^2$$
where $\mathcal{L}_{new}$ is the standard CVAE loss on the new DIB data, $\theta_i^*$ are the parameter values from the pre-trained model, and $\lambda_{ewc}$ controls the strength of consolidation. This allows the dexterous robotic hand model to adapt to new dynamics while preserving essential knowledge of its fundamental kinematics, striking a balance between stability and plasticity.
Experimental Validation and Results
The proposed framework was validated using a tendon-driven dexterous robotic hand prototype. Ground truth data comprising synchronized tendon displacement and joint angle measurements were collected during standard motion sequences. The CVAE was pre-trained on this nominal dataset. To evaluate performance, various fault scenarios were injected into the joint sensor data stream post-collection, simulating on-orbit anomalies.
1. Fault Detection Performance under Simple Faults
We first evaluated the BAT detector against traditional fixed-threshold and EWMA-based methods under two basic fault types: Data Distortion (simulating EMI via added Gaussian noise) and Complete Data Loss (signal drops to zero). The results clearly demonstrate the superiority of the BAT approach.
| Method | Class | Precision | Recall | F1-Score | Accuracy |
|---|---|---|---|---|---|
| Fixed Threshold | Normal | 0.86 | 1.00 | 0.92 | 0.87 |
| Fault | 1.00 | 0.50 | 0.66 | ||
| EWMA Threshold | Normal | 0.81 | 0.68 | 0.74 | 0.64 |
| Fault | 0.35 | 0.53 | 0.43 | ||
| BAT (Proposed) | Normal | 0.89 | 1.00 | 0.94 | 0.91 |
| Fault | 0.99 | 0.64 | 0.77 |
| Method | Class | Precision | Recall | F1-Score | Accuracy |
|---|---|---|---|---|---|
| Fixed Threshold | Normal | 1.00 | 1.00 | 1.00 | 1.00 |
| Fault | 1.00 | 0.98 | 0.99 | ||
| EWMA Threshold | Normal | 1.00 | 0.41 | 0.58 | 0.51 |
| Fault | 0.25 | 1.00 | 0.41 | ||
| BAT (Proposed) | Normal | 1.00 | 1.00 | 1.00 | 1.00 |
| Fault | 1.00 | 1.00 | 1.00 |
The BAT method consistently achieved the highest or near-highest accuracy and F1-scores. Its ability to integrate predictive uncertainty ($\hat{\sigma}^2_m$) allowed it to maintain high precision (low false alarms) while achieving robust recall (few missed faults), even when the EWMA method became unstable.
2. Robustness in Compound Fault Scenarios
A more rigorous test involved a sequence of consecutive, heterogeneous faults: bias, distortion, bit-flips, communication delay, and loss. We compared our CVAE-based system against a strong baseline—a deterministic Long Short-Term Memory (LSTM) network with identical architecture (minus the probabilistic outputs) trained on the same data. The LSTM relied on a fixed threshold for detection. The results underscore the value of probabilistic modeling.
| Model | Class | Precision | Recall | F1-Score | Accuracy |
|---|---|---|---|---|---|
| Deterministic LSTM | Normal | 0.80 | 1.00 | 0.89 | 0.84 |
| Fault | 1.00 | 0.53 | 0.69 | ||
| Online-CVAE + BAT | Normal | 0.96 | 1.00 | 0.98 | 0.97 |
| Fault | 1.00 | 0.92 | 0.96 |
The CVAE+BAT framework significantly outperformed the LSTM baseline, with a 13-point increase in overall accuracy and a 27-point increase in the F1-score for fault detection. The LSTM suffered from higher missed detections, as its fixed threshold could not adapt to the varying nature of the reconstruction errors under different fault types. The BAT mechanism’s dynamic adjustment, guided by CVAE’s uncertainty, proved essential for handling such complex, real-world scenarios for the dexterous robotic hand.
3. Efficacy of Online Incremental Learning
To simulate on-orbit kinematic drift, we modified the ground data by applying a scale factor (1.1) and a small bias, creating a “space-simulated” dataset. The pre-trained CVAE model, when tested on this shifted data, showed a systematic prediction error. The Online-CVAE framework was then engaged: fault-free data from the new distribution was identified by the BAT detector and used for incremental learning via EWC ($\lambda_{ewc}=10^4$, $N_{buf}=512$).
| Metric | Pre-trained CVAE (Static) | Online-CVAE (After Adaptation) | Improvement |
|---|---|---|---|
| Root Mean Square Error (RMSE) | 0.099 rad | 0.016 rad | 83.8% reduction |
| Error Mean ($\bar{e}$) | -0.080 rad | 0.017 rad | Bias eliminated |
| Error Std Dev ($\sigma_e$) | 0.049 rad | 0.037 rad | 24.5% reduction |
The results are conclusive. The online learning mechanism successfully adapted the model to the new operating regime, drastically reducing the prediction error and eliminating the systematic bias. The error standard deviation also decreased, indicating more stable and confident predictions. This demonstrates that the Online-CVAE framework can effectively maintain the accuracy of the dexterous robotic hand‘s joint reconstruction model throughout its mission lifecycle, compensating for performance degradation and environmental changes.
Conclusion
This paper presented a comprehensive, data-driven solution for achieving robust fault tolerance in space dexterous robotic hands. The integration of a probabilistic Conditional Variational Autoencoder (CVAE) with a Bayesian Adaptive Threshold (BAT) detection mechanism forms a core innovation that addresses the limitations of traditional model-based and static data-driven methods. The CVAE provides a reliable, uncertainty-aware reconstruction of joint angles from redundant sensory inputs, while the BAT intelligently leverages this uncertainty to create a dynamic, self-adjusting fault detection boundary. This combination proved highly effective, achieving over 91% accuracy and 94% F1-scores even in challenging compound fault scenarios, significantly outperforming fixed-threshold and EWMA-based detectors.
Furthermore, the incorporation of an online incremental learning framework based on Elastic Weight Consolidation (EWC) tackles the critical issue of long-term performance drift. The Online-CVAE model demonstrated the ability to adapt to simulated changes in the dexterous robotic hand‘s kinematics, reducing prediction errors by over 80% and stabilizing output variance without catastrophically forgetting its originally trained behavior. This ensures the system’s longevity and reliability over extended missions.
The proposed methodology provides a foundational architecture for resilient perception in autonomous space robotics. Future work will focus on enhancing the framework’s generality through advanced data augmentation techniques, investigating federated learning for multi-agent systems, and optimizing the algorithm for deployment on embedded space-grade computing hardware to meet strict power and latency constraints. By advancing the reliability of critical components like the dexterous robotic hand, this research contributes directly to enabling more ambitious, safe, and sustainable on-orbit servicing and assembly missions.