In modern agriculture, the demand for robotic systems to perform delicate tasks such as fruit picking has grown significantly due to labor shortages and the need for precision. Traditional end-effectors often lack adaptability, leading to high damage rates and poor versatility when handling fragile spherical fruits like peaches. As a researcher in agricultural robotics, I have focused on developing a method to optimize contact forces for a dexterous robotic hand to achieve non-destructive and stable picking. This study addresses the critical issue of matching picking forces with fruit characteristics to minimize damage while ensuring successful harvests. By leveraging advanced modeling and optimization techniques, I aim to enhance the performance of dexterous robotic hands in unstructured environments, ultimately contributing to smarter and more efficient farming practices.
The core of this work lies in analyzing the separation mechanics of spherical fruits and establishing a frictional non-destructive point contact model. I selected peaches during the picking season as the primary研究对象 due to their susceptibility to mechanical damage. Through experimental measurements, I determined key parameters such as the static friction coefficient and non-destructive contact forces between the fingertip material and peaches. These parameters are essential for modeling and optimization. Furthermore, I developed a comprehensive grasp model for the dexterous robotic hand using screw theory, which includes kinematic mapping and force-position relationships from joints to contact points and then to the fruit. A multi-objective optimization approach based on the minimum force principle was proposed, and solved using the Non-dominated Sorting Genetic Algorithm II (NSGA-II) to derive optimal contact forces. Field trials with peaches validated the method, demonstrating high success rates, low damage, and efficient picking times. This article details the entire process, emphasizing the role of the dexterous robotic hand in achieving precise and gentle manipulation.
To begin, I analyzed the agricultural model for fruit picking. Spherical fruits like peaches grow in dense,无序 environments with intertwined branches and leaves, requiring versatile picking gestures. Based on manual harvesting techniques, I proposed a combined twist-and-pull separation method, which minimizes the required separation force. This method involves applying a拉力 along the fruit stem direction at an倾斜 angle while simultaneously imparting a torque around the拉力 axis. The separation force can be represented as a wrench $F_p \in \mathbb{R}^{6 \times 1}$, encapsulating both linear and angular components. This approach mimics human actions and enhances adaptability in complex scenarios.
Next, I established a non-destructive contact model for the dexterous robotic hand. Assuming frictional point contact between the fingertips and fruit surface—where sliding or rolling is negligible—the contact force at each fingertip is denoted as $f_{tip} = [f_x, f_y, f_z]^T$. Here, $f_x$ and $f_y$ are tangential components, and $f_z$ is the normal component. The force must lie within the friction cone to prevent slipping, defined by $\sqrt{f_x^2 + f_y^2} \leq \mu f_z$, where $\mu$ is the static friction coefficient. To avoid damage, I decomposed the maximum allowable contact force into a non-destructive grasping force $f_g$ and a non-destructive manipulation force $f_m$. Thus, the constraints for the frictional non-destructive point contact model are:
$$
P = \left\{ \sqrt{f_x^2 + f_y^2} \leq \mu f_z, \quad \sqrt{f_x^2 + f_y^2} \leq f_m, \quad 0 \leq f_z \leq f_g \right\}
$$
This model ensures that the dexterous robotic hand applies forces within safe thresholds, reducing the risk of bruising or表皮 damage while maintaining stability.
Experimental determination of mechanical parameters was crucial for model accuracy. Using peaches of the ‘Xiahui 5’ variety, I conducted tests to measure the static friction coefficient between polyurethane rubber (Shore hardness 40 A) and peach skin. A friction coefficient tester (MXD-02) was employed, with the peach placed on the rubber surface and pulled via a force sensor. The average static friction coefficient from multiple trials was found to be 0.51. For non-destructive forces, I used a customized setup with the dexterous robotic hand equipped with FSR402 thin-film pressure sensors. By applying incremental forces and observing bruising after 48 hours at 25°C, I determined the average non-destructive grasping force $f_g = 7.5 \, \text{N}$ and non-destructive manipulation force $f_m = 3.5 \, \text{N}$. These values are summarized in Table 1.
| Parameter | Symbol | Average Value | Measurement Method |
|---|---|---|---|
| Static Friction Coefficient | $\mu$ | 0.51 | Friction coefficient tester |
| Non-destructive Grasping Force | $f_g$ | 7.5 N | Pressure sensor and bruising observation |
| Non-destructive Manipulation Force | $f_m$ | 3.5 N | 拉力计 and bruising observation |
The grasp model for the dexterous robotic hand involves multi-layer mappings from joints to contact points and then to the fruit. Consider a dexterous robotic hand with $k$ fingers grasping a spherical fruit. Let $\{C_k\}$ be the contact point coordinate frames and $\{O\}$ be the fruit’s center-of-mass frame. The mappings can be divided into two parts: joint-to-contact point and contact-point-to-fruit.
For the joint-to-contact point mapping, I used the Product of Exponentials (POE) method based on screw theory to derive the kinematics of each finger. Taking the thumb as an example, I established a base frame $\{P_1\}$ and a fingertip frame $\{H\}$. The POE model for an $n$-degree-of-freedom spatial manipulator is given by:
$$
T(\theta) = \prod_{i=1}^{n} e^{\hat{\xi}_i \theta_i} M
$$
where $\hat{\xi}_i = \begin{bmatrix} \hat{\omega}_i & \nu_i \\ 0 & 0 \end{bmatrix}$, $\hat{\omega}_i$ is the skew-symmetric matrix of the joint axis direction $\omega_i \in \mathbb{R}^3$, $\nu_i \in \mathbb{R}^3$ is the joint axis position vector, $\theta_i$ is the joint variable, and $M$ is the transformation matrix from the base to the fingertip at the zero position. For the thumb with four joints, the parameters are listed in Table 2.
| Joint $i$ | $\omega_i \in \mathbb{R}^3$ | $\nu_i \in \mathbb{R}^3$ |
|---|---|---|
| 1 | $[0, 0, 1]^T$ | $[0, 0, 1]^T$ |
| 2 | $[0, -1, 0]^T$ | $[0, 0, -L_1]^T$ |
| 3 | $[0, -1, 0]^T$ | $[0, 0, -(L_1 + L_2)]^T$ |
| 4 | $[0, -1, 0]^T$ | $[0, 0, -(L_1 + L_2 + L_3)]^T$ |
Here, $L_1$, $L_2$, $L_3$, and $L_4$ are link lengths. The resulting transformation matrix $T_1$ from the thumb base to the fingertip is:
$$
T_1 = \begin{bmatrix}
c_1 c_{234} & -c_1 s_{234} & s_1 & P_x \\
s_1 c_{234} & -s_1 s_{234} & -c_1 & P_y \\
s_{234} & c_{234} & 0 & P_z \\
0 & 0 & 0 & 1
\end{bmatrix}
$$
where $c_i = \cos \theta_i$, $s_i = \sin \theta_i$, $c_{ij} = \cos(\theta_i + \theta_j)$, $s_{ij} = \sin(\theta_i + \theta_j)$, and $P_x$, $P_y$, $P_z$ are functions of joint angles and link lengths. The spatial Jacobian matrix $J(\theta)$ for the thumb is derived using the adjoint transformation $Ad(T)$:
$$
Ad(T) = \begin{bmatrix} R & 0 \\ [p] R & R \end{bmatrix}, \quad J(\theta) = [\xi_1′, \xi_2′, \dots, \xi_n’]
$$
with $\xi_i’ = Ad\left( \prod_{k=1}^{i-1} e^{\hat{\xi}_k \theta_k} \right) \xi_i$. The relationship between joint torques $\tau$ and contact forces $f_{tip}$ is given by $\tau = J^T f_{tip}$. This mapping ensures that the dexterous robotic hand can precisely control forces at the fingertips.
For the contact-point-to-fruit mapping, I transformed contact forces to the fruit’s coordinate frame. The grasp matrix $G_i$ for the $i$-th contact point is computed as $G_i = Ad(T_{O}^{C_i}) B$, where $B = [I_{3 \times 3}, 0]^T$ and $T_{O}^{C_i}$ is the transformation from $\{C_i\}$ to $\{O\}$. The overall grasp matrix $G = [G_1, G_2, \dots, G_k]$ relates the contact forces $f_{tip} = [f_{tip1}^T, f_{tip2}^T, \dots, f_{tipk}^T]^T$ to the external wrench $F_e$ acting on the fruit:
$$
G f_{tip} = F_e
$$
This equation represents the force balance required for stable grasping with the dexterous robotic hand.
To optimize contact forces for minimal damage, I formulated a multi-objective optimization problem. The goals are to minimize the grasping force and the manipulation force, as these directly influence fruit damage and stability. Define the grasping force vector $f_z = [f_{z1}, f_{z2}, \dots, f_{zk}]$ and a weight matrix $Q$. The first objective function $\Psi(f_{tip})$ aims to minimize the total grasping force:
$$
\Psi(f_{tip}) = f_z Q f_z^T
$$
The second objective function $\Phi(f_{tip})$ focuses on stability by minimizing the angle between the contact force and the normal direction. Let $\alpha = \arctan \mu$ be the friction cone angle and $\beta_i = \arctan\left( \sqrt{f_{xi}^2 + f_{yi}^2} / f_{zi} \right)$ be the angle for the $i$-th contact point. Then:
$$
\Phi(f_{tip}) = \frac{1}{k} \sum_{i=1}^{k} \frac{\beta_i – \alpha}{\beta_i + \alpha}
$$
Here, $\Phi \in (-1, 0]$, where values closer to 0 indicate forces aligned with the normal, enhancing stability. The overall optimization model is:
$$
\begin{aligned}
\min_{f_{tip}} \quad & \left\{ \Psi(f_{tip}), \Phi(f_{tip}) \right\} \\
\text{s.t.} \quad & F_p – F_e \leq 0 \\
& J^T f_{tip} – \tau_{\text{max}} \leq 0 \\
& P
\end{aligned}
$$
where $F_p$ is the separation wrench, $\tau_{\text{max}}$ is the maximum joint torque, and $P$ represents the non-destructive contact constraints. This formulation ensures that the dexterous robotic hand applies forces that are both safe and effective.
I solved this optimization problem using the NSGA-II algorithm, a popular multi-objective evolutionary algorithm. For a case study with a peach of radius 0.062 m and mass 112 g, I considered a three-fingered grasp by the dexterous robotic hand. The grasp matrix $G$ was computed based on contact points on the fruit’s surface. The external wrenches $F_p$ were defined for three stages: pre-picking ($F_{p1} = \mathbf{0}$), during picking ($F_{p2} = [0, -3, -5.2, 0, -0.1, -0.17]^T$), and post-picking ($F_{p3} = [0, 0, 1.12, 0, 0, 0]^T$). NSGA-II parameters included a population size of 120, crossover probability of 0.8, mutation probability of 0.5, and 2000 generations. The Pareto front obtained from optimization revealed trade-offs between minimizing grasping force (for non-destructiveness) and minimizing manipulation force (for stability). For instance, during picking, optimal solutions had $\Psi$ values between 150 and 165, indicating a balance that achieves high success rates with low damage. The dexterous robotic hand’s ability to adapt forces based on this optimization is key to its performance.
To validate the method, I conducted field trials in a peach orchard. The experimental setup included a dexterous robotic hand (model RH56DFX) mounted on a robotic arm (RM65 BI SE), a depth camera (D435) for vision, a工控机, and a mobile chassis. The system operated on Ubuntu 20.04 with ROS Noetic. Peaches with radii ranging from 2.6 to 8 cm were harvested using the optimized contact forces. The picking process involved three phases: approach, grasp and twist-pull separation, and placement. Contact forces were monitored using FSR402 sensors on the fingertips. Figure 1 shows the dexterous robotic hand in action, highlighting its versatility in handling spherical fruits.
The results from 50 picking trials with force optimization are summarized in Table 3. For comparison, I also tested constant-force mode (6 N) and position-control mode (1 mm indentation after contact). The optimized method achieved an average peak grasping force of 6.10 N, an average picking time of 10.3 s, a non-destructive rate of 97.8%, and a success rate of 92.0%. In contrast, constant-force mode had a higher peak force (7.72 N) and lower non-destructive rate (70.4%), while position-control mode had a lower success rate (83.3%). The dexterous robotic hand outperformed both alternatives, demonstrating the effectiveness of the optimization.
| Group | Number of Trials | Average Peak Grasping Force (N) | Average Picking Time (s) | Non-destructive Rate (%) | Success Rate (%) |
|---|---|---|---|---|---|
| Force Optimization | 50 | 6.10 | 10.3 | 97.8 | 92.0 |
| Constant-Force Mode | 30 | 7.72 | 9.8 | 70.4 | 90.0 |
| Position-Control Mode | 30 | 6.37 | 10.7 | 96.0 | 83.3 |
Further analysis of contact force errors showed that the average absolute error between theoretical optimized values and measured values was 0.39 N, indicating good agreement. The dexterous robotic hand successfully handled peaches of various sizes, with failures primarily occurring for fruits smaller than 2.6 cm radius due to localization errors relative to fruit size. This underscores the importance of precise control in the dexterous robotic hand for small objects.
The optimization process also highlighted the role of the dexterous robotic hand in achieving force closure—a condition where the grasp can resist任意 external wrenches. By satisfying the constraints $G f_{tip} = F_e$ and $f_{tip} \in P$, the dexterous robotic hand ensures that the fruit remains stable during picking motions. The NSGA-II algorithm efficiently explored the solution space, providing a set of Pareto-optimal contact forces that balance non-destructiveness and stability. For example, the optimal contact forces for a three-fingered grasp can be expressed as:
$$
f_{tip}^* = \arg \min \left\{ \Psi, \Phi \right\}
$$
These forces are dynamically adjusted during picking, from initial contact to separation and transport. The dexterous robotic hand’s multi-fingered design allows for distributed forces, reducing stress on any single point and minimizing damage.
In terms of mechanical design, the dexterous robotic hand used in this study features five fingers with multiple joints each, enabling human-like dexterity. The fingertips are coated with polyurethane rubber to enhance friction and cushioning. The hand is controlled via torque commands based on the optimized forces, and its kinematics are calibrated using the POE model. This integration of hardware and software makes the dexterous robotic hand a powerful tool for agricultural robotics.
Looking ahead, this method can be extended to other spherical fruits like apples, tomatoes, or citrus, by adjusting the mechanical parameters. The dexterous robotic hand’s adaptability stems from its ability to modulate forces based on real-time sensor feedback. Future work could incorporate machine learning to refine the optimization model or use vision systems for better fruit detection and grasp planning. Additionally, the dexterous robotic hand could be employed in other farm tasks such as pruning or sorting, further increasing its utility.
In conclusion, I have presented a comprehensive approach for optimizing contact forces in a dexterous robotic hand for spherical fruit picking. By combining agricultural modeling, screw theory kinematics, and multi-objective optimization, this method achieves non-destructive and stable grasps. The dexterous robotic hand demonstrates high performance in field trials, with success rates over 90% and minimal damage. This research contributes to the advancement of robotic harvesting, offering a scalable solution for modern agriculture. The dexterous robotic hand, with its precise force control, represents a significant step toward autonomous and efficient fruit production systems.
Throughout this study, the dexterous robotic hand has proven to be a versatile and effective tool. Its ability to apply minimal forces while maintaining stability is crucial for handling delicate produce. As robotics continues to evolve, the dexterous robotic hand will play a pivotal role in transforming agricultural practices, reducing labor dependency, and improving food quality. I am confident that the methods described here will inspire further innovations in the field, ultimately leading to smarter and more sustainable farming.