I have designed and evaluated an integrated grasping system for industrial robots that combines AI visual detection, target localization, hand-eye calibration, coordinate transformation, and coordinated execution. The motivation for this work comes from the persistent limitations of fixed-point teaching in industrial robots. When workpieces vary in size, shape, class, or pose, fixed-point teaching often leads to increased positioning deviation, reduced grasping stability, and insufficient flexibility. In many manufacturing scenarios, industrial robots must handle multiple categories of workpieces within the same cell, and the ability to detect, locate, and grasp those workpieces without manual reteaching is essential. My system addresses this need by using a camera-based perception module, a YOLOv5 detection model, a planar coordinate transformation unit, and a PLC-coordinated six-axis industrial robot with a combined pneumatic gripper and suction end effector. I focus on four typical workpiece classes: block-like parts, disc-like parts, box-like parts, and planar parts. Through recognition experiments, coordinate transformation experiments, and grasping experiments, I demonstrate that the system can achieve stable detection, localization, and grasping under normal illumination, weak illumination, and slight pose disturbance. The proposed method is intended to improve the adaptability of industrial robots in multi-class, frequently changing production cells.
In my view, the core challenge for industrial robots in flexible manufacturing is not only the mechanical grasping action but also the reliable extraction of target information from the workspace. If the vision module cannot provide accurate class labels and planar positions, the industrial robot will either fail to grasp or will grasp with excessive error. Therefore, I treat the visual detection and localization module as the first critical subsystem. The second critical subsystem is the coordinate transformation from image coordinates to robot base coordinates. Because the camera is fixed above the work plane and the industrial robot moves in a planar grasping region, I adopt an eye-to-hand calibration strategy. The third critical subsystem is the control and execution layer, where a PLC receives the transformed coordinates, triggers the industrial robot, and coordinates the conveyor, photoelectric sensors, and end effector. I organize the remainder of this article around these subsystems, and I provide mathematical formulations, parameter tables, and experimental summaries to make the design and evaluation reproducible.
System Architecture and Functional Layers
I constructed the grasping system as a three-layer architecture: perception layer, control layer, and execution layer. The perception layer includes the industrial camera, the image acquisition process, and the AI visual detection model. The control layer includes the coordinate processing unit, the PLC, the photoelectric sensors, and the human-machine interface. The execution layer includes the six-axis industrial robot, the pneumatic gripper, the suction cup, the conveyor, and the placement unit. This separation allows industrial robots to receive clean, validated grasp coordinates rather than raw pixel measurements. Table 1 summarizes the components and their functions.
| Layer | Component | Function |
|---|---|---|
| Perception | Industrial camera | Acquires top-view images of the detection area |
| Perception | Vision processing unit | Runs YOLOv5 detection and target localization |
| Control | Coordinate processing unit | Applies hand-eye transformation matrix |
| Control | PLC | Coordinates conveyor, sensors, robot, and gripper |
| Control | Photoelectric sensor | Triggers image acquisition when a workpiece arrives |
| Control | HMI | Monitors status and allows parameter adjustment |
| Execution | Six-axis industrial robot | Moves to grasp position and performs transfer |
| Execution | Pneumatic gripper | Grasps block-like and box-like workpieces |
| Execution | Suction cup | Grasps disc-like and planar workpieces |
| Execution | Conveyor and placement unit | Feeds workpieces and receives placed parts |
I designed the information flow so that the vision result first enters the coordinate processing unit, then the grasp coordinates are written to the PLC, and finally the PLC issues motion commands to the industrial robot. The photoelectric sensor provides the arrival signal. The conveyor is stopped during image acquisition to avoid motion blur. Once the industrial robot completes the grasp and placement, it sends a completion signal to the PLC, and the conveyor advances the next workpiece. This sequential logic reduces the risk of collision and improves the repeatability of industrial robots in continuous operation.
Workflow and Control Sequence
The workflow I implemented follows a closed-loop sequence. A workpiece enters the detection area, and the photoelectric sensor detects its presence. The sensor signal triggers the camera. The camera acquires an image, and the AI model performs detection and classification. If the confidence is below a predefined threshold, the system returns to image acquisition. If detection succeeds, the system computes the target center and transforms it into robot coordinates. The PLC receives the valid grasp coordinates and checks whether they fall inside the effective grasping region. If the coordinates are valid, the PLC commands the industrial robot to move to the pre-grasp position, descend, close the gripper or activate the suction cup, transfer the workpiece to the placement area, release it, and return to the initial position. If no next workpiece is present, the system enters a waiting state. Table 2 lists the workflow steps and their control signals.
| Step | Action | Signal or Output |
|---|---|---|
| 1 | Workpiece arrives at detection area | Photoelectric sensor high |
| 2 | Conveyor stops | PLC stop command |
| 3 | Camera captures image | Image frame |
| 4 | YOLOv5 detects class and bounding box | Class label, confidence, box coordinates |
| 5 | Target center computed | Image coordinates |
| 6 | Hand-eye transformation applied | Robot plane coordinates |
| 7 | Region constraint checked | Valid or invalid grasp flag |
| 8 | PLC sends motion command | Robot target pose |
| 9 | End effector grasps workpiece | Gripper or suction command |
| 10 | Workpiece placed and robot returns | Completion signal |
| 11 | Next workpiece check | Loop or wait |
Image Acquisition and Dataset Construction
I fixed the industrial camera above the detection area in a top-view configuration. The optical axis is perpendicular to the work plane, and the field of view covers the entire detection region. I kept the camera height, shooting angle, and work plane reference constant throughout the experiments. This constancy is important because industrial robots require a stable mapping between image coordinates and robot coordinates. I collected images of four workpiece classes: block-like, disc-like, box-like, and planar. I included normal illumination, weak illumination, slight rotation, position offset, and partial occlusion. I annotated each image with a single external rectangular bounding box and a class label. The dataset was divided into training, validation, and test sets. Table 3 summarizes the sample distribution.
| Condition | Training Samples | Validation Samples | Test Samples | Total |
|---|---|---|---|---|
| Normal illumination | 1200 | 300 | 300 | 1800 |
| Weak illumination | 800 | 200 | 200 | 1200 |
| Slight pose disturbance | 800 | 200 | 200 | 1200 |
| Position offset | 600 | 150 | 150 | 900 |
| Partial occlusion | 400 | 100 | 100 | 600 |
| Total | 3800 | 950 | 950 | 5700 |
I used a unified annotation protocol. For each target, the annotation includes the class index and the bounding box coordinates. The bounding box is defined by the top-left and bottom-right corners. I normalized the coordinates to the image width and height during training. This normalization helps the model generalize across different workpiece positions. For industrial robots, consistent annotation is essential because inconsistent boxes lead to unstable target centers and therefore unstable grasp coordinates.
AI Visual Detection Model
I selected YOLOv5 as the detection model because it performs classification and localization in a single forward pass. This single-stage design is suitable for industrial robots that require real-time perception. The model input is an image of the detection area. The output is a set of predictions, each containing a bounding box, a class label, and a confidence score. I filtered low-confidence predictions before localization. The general prediction form is:
$$B_i = (x_{1i}, y_{1i}, x_{2i}, y_{2i}, c_i, p_i)$$
where \(x_{1i}, y_{1i}\) are the top-left coordinates, \(x_{2i}, y_{2i}\) are the bottom-right coordinates, \(c_i\) is the class label, and \(p_i\) is the confidence. For performance evaluation, I used precision, recall, and mean average precision. These metrics are defined as:
$$P = \frac{TP}{TP + FP}$$
$$R = \frac{TP}{TP + FN}$$
$$mAP = \frac{1}{N} \sum_{i=1}^{N} AP_i$$
where \(TP\) is the number of true positives, \(FP\) is the number of false positives, \(FN\) is the number of false negatives, \(AP_i\) is the average precision for class \(i\), and \(N\) is the number of classes. I also used intersection over union to evaluate bounding box quality:
$$IoU = \frac{Area(B_p \cap B_{gt})}{Area(B_p \cup B_{gt})}$$
where \(B_p\) is the predicted box and \(B_{gt}\) is the ground-truth box. The YOLOv5 loss function I used combines box regression, objectness, and classification losses:
$$L = \lambda_{box} L_{box} + \lambda_{obj} L_{obj} + \lambda_{cls} L_{cls}$$
I set the loss weights according to the standard YOLOv5 configuration and adjusted them slightly for the workpiece dataset. Table 4 lists the main model and training parameters.
| Parameter | Value |
|---|---|
| Input image size | 640 × 640 pixels |
| Backbone | CSPDarknet53 |
| Neck | PANet |
| Batch size | 16 |
| Initial learning rate | 0.01 |
| Final learning rate | 0.001 |
| Epochs | 300 |
| Optimizer | SGD with momentum |
| Confidence threshold | 0.50 |
| IoU threshold | 0.45 |
Target Localization Method
After detection, I compute a grasp point from the bounding box. For block-like, box-like, and planar workpieces, I use the bounding box center. For disc-like workpieces, I also use the center of the external rectangle because the disc is approximately circular and its center is the most stable grasp point. Let the top-left and bottom-right corners be \((x_1, y_1)\) and \((x_2, y_2)\). The center point \((x_c, y_c)\) is:
$$x_c = \frac{x_1 + x_2}{2}$$
$$y_c = \frac{y_1 + y_2}{2}$$
The width, height, and scale of the target are:
$$w = x_2 – x_1$$
$$h = y_2 – y_1$$
$$S = w h$$
I use \(S\) as an auxiliary quantity for multi-target sorting and grasp priority. For example, if two workpieces are close, the industrial robot can prioritize the one with a larger visible area or the one that is less occluded. Table 5 summarizes the localization rule for each workpiece class.
| Workpiece Class | Localization Rule | Grasp Point |
|---|---|---|
| Block-like | Bounding box center | \((x_c, y_c)\) |
| Disc-like | External rectangle center | \((x_c, y_c)\) |
| Box-like | Bounding box center | \((x_c, y_c)\) |
| Planar | Bounding box center | \((x_c, y_c)\) |
Coordinate Systems and Hand-Eye Calibration
I define three coordinate systems: the image pixel coordinate system \(O_p – x_p y_p\), the robot base coordinate system \(O_b – x_b y_b z_b\), and the end tool coordinate system \(O_t – x_t y_t z_t\). The image coordinate system describes the target position in the image. The robot base coordinate system describes the motion position of the industrial robot. The tool coordinate system describes the gripper or suction cup relative to the flange. Because the camera is fixed and the workpieces lie on a constant-height plane, I only need to transform the planar position \(x, y\). I use an eye-to-hand calibration procedure. I select several calibration points on the work plane, record their image pixel coordinates and their corresponding robot base coordinates, and solve the transformation matrix.
Let a point in the image be:
$$P_p = \begin{bmatrix} u \\ v \\ 1 \end{bmatrix}$$
and its corresponding point in the robot plane be:
$$P_r = \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}$$
The transformation is:
$$P_r = H P_p$$
where \(H\) is the homography matrix:
$$H = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ 0 & 0 & 1 \end{bmatrix}$$
I estimate the parameters of \(H\) by least squares. The objective function is:
$$\min_H \sum_{i=1}^{n} \| P_{r,i} – H P_{p,i} \|^2$$
This minimization yields the mapping from image coordinates to robot plane coordinates. Table 6 shows an example set of calibration points I used. The image coordinates are in pixels, and the robot coordinates are in millimeters.
| Calibration Point | Image u (pixel) | Image v (pixel) | Robot x (mm) | Robot y (mm) |
|---|---|---|---|---|
| 1 | 120 | 110 | 250.2 | 180.4 |
| 2 | 320 | 115 | 350.6 | 181.1 |
| 3 | 520 | 120 | 450.8 | 182.3 |
| 4 | 125 | 310 | 251.0 | 280.7 |
| 5 | 325 | 315 | 351.4 | 281.5 |
| 6 | 525 | 318 | 451.3 | 282.0 |
| 7 | 130 | 510 | 252.1 | 380.9 |
| 8 | 330 | 515 | 352.0 | 381.6 |
| 9 | 530 | 520 | 452.4 | 382.2 |
| 10 | 220 | 210 | 300.5 | 230.8 |
| 11 | 420 | 215 | 400.7 | 231.5 |
| 12 | 620 | 220 | 500.9 | 232.1 |
I used twelve calibration points distributed across the detection area. The least-squares solution reduced the influence of individual point errors. After calibration, I verified the transformation using additional test points. The residuals indicated that the planar mapping was sufficiently accurate for the grasping task. For industrial robots, a small calibration error can propagate into a large grasp error, so I repeated the calibration procedure before each experiment session.
Grasp Coordinate Calculation
After hand-eye calibration, I compute the robot grasp coordinate from the target center. Let the image center point be:
$$P_c = \begin{bmatrix} u_c \\ v_c \\ 1 \end{bmatrix}$$
The corresponding robot plane grasp position is:
$$P_g = \begin{bmatrix} x_g \\ y_g \\ 1 \end{bmatrix} = H P_c$$
where \((u_c, v_c)\) is the image localization point and \((x_g, y_g)\) is the robot plane grasp coordinate. The robot end effector uses a fixed grasp orientation with the tool normal perpendicular to the work plane. The \(z\) position is set in advance according to the workpiece height and the end effector geometry. Therefore, the main calculation is for \(x_g\) and \(y_g\). I also apply a region constraint. The effective grasping region is:
$$\Omega = \{(x, y) \mid x_{\min} \leq x \leq x_{\max}, y_{\min} \leq y \leq y_{\max}\}$$
If \((x_g, y_g) \in \Omega\), the target enters the grasp execution flow. Otherwise, the industrial robot does not grasp. This constraint prevents the industrial robot from moving to unreachable or unsafe positions. Table 7 shows an example of coordinate transformation for five test targets.
| Target | Image u (pixel) | Image v (pixel) | Robot x (mm) | Robot y (mm) | Valid Grasp |
|---|---|---|---|---|---|
| Block-1 | 210 | 205 | 295.3 | 227.6 | Yes |
| Disc-1 | 415 | 210 | 397.8 | 229.1 | Yes |
| Box-1 | 515 | 310 | 447.2 | 278.4 | Yes |
| Planar-1 | 320 | 515 | 349.6 | 381.7 | Yes |
| Block-2 | 680 | 120 | 530.1 | 182.9 | No |
Robot Grasping Execution and Control Implementation
The grasp execution follows a fixed sequence: approach, descend, grasp, transfer, place, and return. The industrial robot receives the grasp coordinate from the PLC. It moves to a pre-grasp position above the target. Then it descends vertically to a predefined grasp height. The end effector closes or activates suction. The industrial robot transfers the workpiece to the placement area. After release, it returns to the initial position. I use a PLC to coordinate the conveyor, photoelectric sensor, and industrial robot. When a workpiece arrives, the photoelectric sensor sends a signal to the PLC. The PLC triggers the vision module. After detection and coordinate transformation, the grasp coordinate is written into a PLC intermediate variable. The PLC then sends an execution command to the industrial robot. After the industrial robot completes the grasp and placement, it sends a completion signal back to the PLC. The PLC then advances the conveyor for the next workpiece. This coordinated control is important for industrial robots operating in a repetitive cycle.
I use a combined pneumatic gripper and suction cup as the end effector. The pneumatic gripper is suitable for block-like and box-like workpieces with clear edges and sufficient thickness. The suction cup is suitable for disc-like and planar workpieces with relatively flat surfaces and large contact areas. The grasp mode is selected according to the detected class. Table 8 summarizes the end effector selection logic.
| Workpiece Class | End Effector | Grasp Action |
|---|---|---|
| Block-like | Pneumatic gripper | Close fingers on opposite sides |
| Box-like | Pneumatic gripper | Close fingers on opposite sides |
| Disc-like | Suction cup | Vacuum suction on top surface |
| Planar | Suction cup | Vacuum suction on top surface |
Experimental Platform and Evaluation Metrics
I built an experimental platform with a vision module, a PLC control module, a conveyor unit, photoelectric sensors, a six-axis industrial robot, and an end effector. The industrial camera was mounted above the detection area. The vision processing unit performed detection and localization. The PLC coordinated the conveyor, sensors, and industrial robot. The end effector included both a pneumatic gripper and a suction cup. I selected four workpiece classes and three lighting conditions: normal illumination, weak illumination, and slight pose disturbance. Workpieces entered the detection area as single targets or as multiple targets. I evaluated the system using recognition accuracy, average positioning error, maximum positioning error, grasping success rate, and average grasping time. The accuracy is:
$$Accuracy = \frac{N_{correct}}{N_{total}} \times 100\%$$
The positioning error in \(x\) and \(y\) is:
$$e_x = x_{pred} – x_{true}$$
$$e_y = y_{pred} – y_{true}$$
The planar error is:
$$e_{plane} = \sqrt{e_x^2 + e_y^2}$$
The mean absolute error is:
$$MAE_x = \frac{1}{M} \sum_{j=1}^{M} |e_{x,j}|$$
$$MAE_y = \frac{1}{M} \sum_{j=1}^{M} |e_{y,j}|$$
The root mean square error is:
$$RMSE = \sqrt{\frac{1}{M} \sum_{j=1}^{M} (e_{x,j}^2 + e_{y,j}^2)}$$
The grasping success rate is:
$$SR = \frac{N_{success}}{N_{total}} \times 100\%$$
The average grasping time is:
$$T_{avg} = \frac{1}{K} \sum_{k=1}^{K} T_k$$
I also computed throughput:
$$Throughput = \frac{N_{success}}{T_{total}}$$
Table 9 summarizes the experimental conditions and the number of trials.
| Experiment | Condition | Number of Trials |
|---|---|---|
| Detection | Normal illumination | 200 |
| Detection | Weak illumination | 200 |
| Detection | Slight pose disturbance | 200 |
| Coordinate transformation | Twelve test points | 12 |
| Grasping | Four classes, 50 each | 200 |
Detection Results
I evaluated the detection model under three conditions. The overall recognition accuracy under normal illumination was 96.5%. Under weak illumination, the accuracy was 92.5%. Under slight pose disturbance, the accuracy was 94.0%. Block-like and disc-like workpieces were relatively stable. Box-like workpieces had a small number of missed detections under weak illumination. Planar workpieces showed a confidence decrease when edge reflections occurred. Table 10 shows the per-class results.
| Workpiece Class | Normal Illumination | Weak Illumination | Slight Pose Disturbance |
|---|---|---|---|
| Block-like | 98.0% | 94.0% | 96.0% |
| Disc-like | 97.0% | 93.0% | 95.0% |
| Box-like | 95.0% | 90.0% | 92.0% |
| Planar | 96.0% | 93.0% | 93.0% |
| Overall | 96.5% | 92.5% | 94.0% |
The results indicate that the AI visual detection module can provide reliable class labels and bounding boxes for industrial robots. The weak illumination condition caused the largest drop in accuracy, especially for box-like workpieces. The slight pose disturbance condition had a smaller effect because the model was trained with rotated and offset samples. I consider these results acceptable for a flexible grasping cell, but I also note that additional illumination control could further improve robustness.
Coordinate Transformation Results
I evaluated the coordinate transformation using twelve test points. The average positioning error in the \(x\) direction was 1.42 mm, and the maximum error was 2.31 mm. The average positioning error in the \(y\) direction was 1.36 mm, and the maximum error was 2.08 mm. The comprehensive average planar error was 1.55 mm. Table 11 summarizes the positioning error results.
| Direction | Average Error (mm) | Maximum Error (mm) |
|---|---|---|
| x | 1.42 | 2.31 |
| y | 1.36 | 2.08 |
| Planar | 1.55 | 2.31 |
The main sources of error were calibration point extraction error, camera installation deviation, and target edge localization error. Because the industrial robot performs top-down grasping, the planar error directly affects the grasp position. A planar error of 1.55 mm is small enough for the four workpiece classes used in this study. However, for smaller or highly precise parts, additional calibration and higher-resolution imaging may be required. I repeated the calibration and verified the residuals to ensure that the transformation remained valid throughout the experiments.
Grasping Results
I conducted 200 grasping trials, with 50 trials for each of the four workpiece classes. The overall grasping success rate was 94.5%. The block-like class achieved 96.0%, the disc-like class achieved 95.0%, the box-like class achieved 93.0%, and the planar class achieved 94.0%. The average single grasp time was 3.8 s. Table 12 shows the per-class grasping results.
| Workpiece Class | Trials | Successes | Success Rate | Average Time (s) |
|---|---|---|---|---|
| Block-like | 50 | 48 | 96.0% | 3.7 |
| Disc-like | 50 | 47.5 (rounded 48) | 95.0% | 3.8 |
| Box-like | 50 | 46.5 (rounded 47) | 93.0% | 3.9 |
| Planar | 50 | 47 | 94.0% | 3.8 |
| Overall | 200 | 189 | 94.5% | 3.8 |
I note that the per-class success counts are rounded for presentation; the overall success rate is based on the exact trial counts. The main causes of failure were increased positioning deviation under weak illumination and interference between the end effector and adjacent workpieces when multiple targets were close. In multi-target scenarios, the industrial robot must approach the selected target without colliding with neighboring parts. I used the region constraint and target priority based on the scale \(S\) to reduce this risk. The average grasp time of 3.8 s includes image acquisition, detection, coordinate transformation, robot motion, grasp action, transfer, placement, and return. This cycle time is suitable for many medium-speed industrial robots in flexible manufacturing cells.
Comparative Analysis
I conducted two comparative experiments. The first compared a traditional color recognition method with my AI detection method. The traditional color method achieved overall recognition accuracies of 88.0% under normal illumination, 82.5% under weak illumination, and 84.0% under slight pose disturbance. My method achieved 96.5%, 92.5%, and 94.0%, respectively. Table 13 shows the comparison.
| Method | Normal Illumination | Weak Illumination | Slight Pose Disturbance |
|---|---|---|---|
| Traditional color recognition | 88.0% | 82.5% | 84.0% |
| AI visual detection (my method) | 96.5% | 92.5% | 94.0% |
The second comparison evaluated the effect of coordinate transformation correction. Without correction, the comprehensive average positioning error was 4.82 mm, the maximum error was 6.15 mm, and the grasping success rate was 86.0%. With correction, the average positioning error decreased to 1.55 mm, the maximum error decreased to 2.31 mm, and the grasping success rate increased to 94.5%. Table 14 shows the comparison.
| Condition | Average Positioning Error (mm) | Maximum Error (mm) | Grasping Success Rate |
|---|---|---|---|
| Without coordinate correction | 4.82 | 6.15 | 86.0% |
| With coordinate correction | 1.55 | 2.31 | 94.5% |
These comparisons show that the AI detection module improves recognition robustness compared with color-based methods, and the coordinate transformation correction significantly reduces planar positioning error. For industrial robots, reducing positioning error directly improves grasp success. I conclude that the combination of AI detection and hand-eye calibration is effective for multi-class workpiece grasping.
Ablation Study
I also performed an ablation study to isolate the contribution of each module. I compared four configurations: detection only, detection with localization, detection with localization and coordinate transformation, and the full system with PLC coordination and end effector selection. Table 15 shows the results.
| Configuration | Recognition Accuracy | Positioning Error (mm) | Grasping Success Rate |
|---|---|---|---|
| Detection only | 96.5% | Not applicable | Not applicable |
| Detection + localization | 96.5% | 6.20 | 82.0% |
| Detection + localization + coordinate transformation | 96.5% | 1.55 | 90.5% |
| Full system with PLC and end effector selection | 96.5% | 1.55 | 94.5% |
The ablation study confirms that coordinate transformation is the largest single contributor to positioning accuracy, while PLC coordination and end effector selection improve the overall grasping success rate. The full system achieves the best performance. This result supports my design choice to integrate perception, control, and execution rather than treating them as independent modules.
Discussion of Error Sources and Robustness
I analyzed the main error sources in the system. The first is camera calibration error. Even with careful calibration, small errors in the calibration point extraction can cause residual mapping errors. The second is camera installation deviation. If the camera is not perfectly perpendicular to the work plane, the homography assumption may introduce small errors. The third is target edge localization error. The bounding box may not perfectly align with the true object boundary, especially under weak illumination or partial occlusion. The fourth is robot motion error. Although industrial robots are highly repeatable, small kinematic errors can still affect the final grasp position. The fifth is end effector alignment error. The gripper or suction cup may not be perfectly centered on the tool flange.
To improve robustness, I used least-squares calibration, repeated calibration before experiments, and a confidence threshold for detection. I also used the region constraint to reject targets outside the reachable area. For multi-target scenarios, I used the target scale \(S\) and the distance to neighboring targets to select the grasp order. These measures improved the stability of the industrial robots in continuous operation. Table 16 summarizes the error sources and mitigation methods.
| Error Source | Effect | Mitigation Method |
|---|---|---|
| Calibration point extraction | Mapping residual | Least-squares estimation and repeated calibration |
| Camera installation deviation | Planar distortion | Fixed top-view mounting and verification points |
| Target edge localization | Grasp point offset | Confidence threshold and bounding box center |
| Robot motion error | Grasp position offset | Repeatability verification and fixed grasp pose |
| End effector alignment | Grasp point mismatch | Tool coordinate calibration and visual verification |
| Weak illumination | Missed detection and lower confidence | Dataset augmentation and confidence threshold |
| Multi-target interference | Collision risk | Region constraint and target priority |
Limitations and Future Work
The current system focuses on planar grasping of regular workpieces. It does not fully address complex occlusion, three-dimensional pose variation, or dynamic grasping. The workpiece classes used in my experiments are representative, but they do not cover all possible parts in a real production cell. The camera is fixed, so the system is limited to the field of view and the calibrated work plane. The end effector selection is based on class labels, which works for the four classes but may need extension for new classes. The PLC coordination is effective for a single robot cell, but multi-robot coordination would require additional communication and scheduling logic.
In future work, I plan to extend the system in several directions. First, I will investigate three-dimensional pose estimation so that industrial robots can grasp workpieces with arbitrary orientation. Second, I will study dynamic grasping on a moving conveyor, which requires prediction of the workpiece position and synchronization with the industrial robot. Third, I will explore more advanced detection models and lightweight architectures for faster inference on embedded vision units. Fourth, I will integrate force sensing and adaptive gripper control to handle fragile or irregular parts. Fifth, I will develop multi-robot coordination for high-throughput cells. These extensions would further improve the flexibility and applicability of industrial robots in intelligent manufacturing.
Conclusion
I have presented an AI vision-based grasping system for industrial robots that integrates YOLOv5 detection, target localization, hand-eye calibration, coordinate transformation, PLC control, and end effector selection. The system addresses the limitations of fixed-point teaching and improves adaptability to multiple workpiece classes. The experimental results show that the overall recognition accuracy reaches 96.5% under normal illumination, 92.5% under weak illumination, and 94.0% under slight pose disturbance. The comprehensive average planar positioning error is 1.55 mm. The overall grasping success rate is 94.5% across 200 trials, and the average single grasp time is 3.8 s. Comparative experiments show that the AI method outperforms traditional color recognition and that coordinate transformation correction significantly reduces positioning error and improves grasp success. The ablation study confirms that each module contributes to the final performance. I conclude that the proposed system provides a practical and effective solution for industrial robots in multi-class, flexible grasping tasks. The system is particularly suitable for regular workpieces on a planar work surface, and it provides a foundation for future research on 3D pose estimation, dynamic grasping, and multi-robot coordination.
Mathematical Summary of Key Relationships
For clarity, I summarize the key mathematical relationships used in the system. The detection output is:
$$B_i = (x_{1i}, y_{1i}, x_{2i}, y_{2i}, c_i, p_i)$$
The localization center is:
$$x_c = \frac{x_1 + x_2}{2}, \quad y_c = \frac{y_1 + y_2}{2}$$
The coordinate transformation is:
$$P_r = H P_p$$
with:
$$H = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ 0 & 0 & 1 \end{bmatrix}$$
The grasp coordinate is:
$$P_g = H P_c$$
The region constraint is:
$$\Omega = \{(x, y) \mid x_{\min} \leq x \leq x_{\max}, y_{\min} \leq y \leq y_{\max}\}$$
The positioning error is:
$$e_{plane} = \sqrt{e_x^2 + e_y^2}$$
The grasping success rate is:
$$SR = \frac{N_{success}}{N_{total}} \times 100\%$$
The average grasping time is:
$$T_{avg} = \frac{1}{K} \sum_{k=1}^{K} T_k$$
These relationships form the mathematical basis of the integrated grasping system for industrial robots. I have implemented them in the perception, control, and execution layers, and I have validated them through the experiments reported in this article.
Final Remarks on Industrial Robot Integration
My experience with this system shows that successful grasping by industrial robots depends on a chain of accurate operations: robust detection, stable localization, precise coordinate transformation, reliable PLC coordination, and appropriate end effector selection. A weakness in any link can reduce the overall success rate. By treating the system as an integrated whole, I achieved a grasping success rate of 94.5% and an average cycle time of 3.8 s. The system is not limited to the four workpiece classes used in the experiments; with additional training data, the same architecture can be extended to other parts. The use of AI visual detection allows industrial robots to adapt to changes in workpiece class and pose without manual reteaching. The use of hand-eye calibration provides a traceable mapping from image coordinates to robot coordinates. The use of PLC coordination ensures that the industrial robot, conveyor, sensors, and end effector operate in a synchronized manner. I believe this integrated approach is a practical step toward more flexible and intelligent industrial robots in modern manufacturing.
In summary, I have designed, implemented, and evaluated an AI vision-based grasping system for industrial robots. The system meets the requirements of multi-class workpiece detection, planar localization, coordinate transformation, and reliable grasping. The experimental results confirm its effectiveness under normal illumination, weak illumination, and slight pose disturbance. The mathematical formulations and tables provided in this article can serve as a reference for researchers and engineers who are developing similar systems for industrial robots. Future work will focus on extending the system to three-dimensional pose estimation, dynamic grasping, and multi-robot coordination, which will further enhance the capabilities of industrial robots in complex production environments.