In the modern power grid, the efficient and safe operation of substations is critical for ensuring reliable electricity supply. Traditional substation operations, such as switching and inspection, heavily rely on manual labor, which poses significant challenges including low efficiency, high human resource costs, and safety risks due to live-line work. For instance, a typical switching operation requires two personnel to travel to remote substations, often spending hours on transit for tasks that take only minutes to complete. This inefficiency has become a bottleneck as grid scales expand, necessitating innovative solutions. To address these issues, we have developed an intelligent robot based on machine vision, integrating artificial intelligence and robotics to automate substation operations. This intelligent robot enables remote switching, autonomous inspection, and emergency tripping, thereby enhancing operational efficiency, reducing costs, and minimizing human exposure to hazardous environments. In this paper, we detail the design, implementation, and application of this intelligent robot, focusing on its visual localization methods, system architecture, and functional capabilities.
The core innovation of our intelligent robot lies in its advanced machine vision system, which utilizes target tracking algorithms for precise localization and operation. We employ a Mean Shift-based algorithm for robust object tracking in dynamic substation environments. This algorithm allows the intelligent robot to identify and follow targets such as control knobs, buttons, and indicators on switchgear, ensuring accurate manipulation during switching tasks. The integration of multi-axis robotic arms, floating operation platforms, 3D visual cameras, and 2D lasers further enhances the robot’s dexterity and adaptability. Our intelligent robot has been successfully deployed in 220 kV, 110 kV substations, and 10 kV switch rooms, demonstrating significant improvements in operation speed and safety. For example, switching operations that previously took hours can now be completed in minutes remotely, showcasing the transformative potential of this intelligent robot in modernizing substation management.
To provide context, we first review the global research landscape for substation robots. Internationally, the development of intelligent robots for power systems began early, with countries like the United States and Japan pioneering robotic applications in hazardous environments. For instance, Katrasnik et al. proposed an image vision-based inspection robot for substations, validating its effectiveness through experiments. Their work highlighted the use of visual sensors for autonomous navigation and defect detection. Similarly, John-Young Park et al. introduced a novel inspection robot with wheel-leg mobility, emphasizing its compact design and enhanced insulation properties. They suggested that biomimetic stereo vision could further improve the intelligence of such robots. These advancements underscore the growing trend towards automation in power systems, where intelligent robots are seen as key enablers for reducing human intervention and improving reliability.
In China, research on substation intelligent robots started later but has progressed rapidly, driven by national initiatives and utility demands. Early efforts by researchers like Lu Shouyin in 2005 led to mobile robot-based inspection systems for unmanned substations, focusing on outdoor high-voltage equipment. Subsequent developments, such as the cloud platform control system by Xiao Peng, enabled precise positioning and control for inspection tasks. State Grid Corporation of China has established dedicated laboratories for power robotics, producing various prototypes including inspection robots and live-line work robots. For example, Shandong Electric Power Research Institute and Shandong Luneng Intelligent Technology Co., Ltd. developed the first functional prototype of a substation inspection robot in 2004, which has since been widely adopted. More recently, collaborations between Chongqing Power Company and Chongqing University resulted in a巡检 robot deployed in 500 kV substations, capable of remote monitoring and autonomous operation. These efforts reflect a concerted push towards intelligent robotics in China’s power sector, with our work building upon these foundations to create a more versatile and integrated intelligent robot.
A critical component of our intelligent robot is its visual localization system, which relies on the Mean Shift algorithm for target tracking. Mean Shift is a non-parametric density estimation technique that does not require prior knowledge of probability distributions, making it suitable for complex substation environments with varying lighting and occlusions. The algorithm iteratively shifts a search window toward the direction of maximum density increase, effectively tracking moving objects. Given a set of sample points \( x_i \) in a d-dimensional space \( \mathbb{R}^d \), the Mean Shift vector at a point \( x \) is defined as:
$$ M_h(x) = \frac{1}{k} \sum_{x_i \in S_h(x)} (x_i – x) $$
where \( S_h(x) \) is a high-dimensional sphere region of radius \( h \), defined as \( S_h(x) = \{ y : (y – x)^T (y – x) < h^2 \} \), and \( k \) is the number of sample points falling within \( S_h \). This vector points toward the region of highest data density, guiding the tracking process. In practice, we apply Mean Shift to video frames by modeling the target’s color histogram in HSV space and searching for similar regions in subsequent frames. The similarity is measured using the Bhattacharyya coefficient:
$$ \rho(p, q) = \sum_{u=1}^{m} \sqrt{p_u(y_0) q_u} $$
where \( p_u(y_0) \) represents the candidate target model and \( q_u \) is the reference model. To improve robustness, we incorporate kernel functions and weight coefficients, giving more importance to pixels near the target center. We use the Epanechnikov kernel:
$$ K_E(x) = \begin{cases} c (1 – \|x\|^2), & \text{if } \|x\| \leq 1 \\ 0, & \text{otherwise} \end{cases} $$
This kernel reduces the influence of outliers, enhancing the intelligent robot’s ability to track targets under partial occlusion or illumination changes. The Mean Shift procedure involves initializing the target location, computing histograms, and iteratively updating the position until convergence. We have implemented this algorithm in real-time on the robot’s onboard processors, enabling seamless tracking during operations such as knob rotation or button pressing. The following table summarizes the key parameters and their values used in our Mean Shift implementation for the intelligent robot:
| Parameter | Description | Value |
|---|---|---|
| \( h \) | Bandwidth for search window | 15 pixels |
| Kernel | Epanechnikov function | \( c = 0.75 \) |
| Histogram bins | Number of bins for H channel | 16 |
| Iterations | Maximum iterations for convergence | 20 |
| Similarity threshold | Bhattacharyya coefficient minimum | 0.7 |
The system architecture of our intelligent robot is designed for scalability and remote operability. It consists of two main components: the robot本体 and the centralized control center & district server. These components communicate via wireless and wired networks, ensuring reliable data transmission even in electromagnetically noisy substation environments. The robot本体 serves as the physical executor of tasks, equipped with multiple sensors and actuators. Its architecture includes ROS nodes for hardware interaction, core services for task management, database services for state storage, file services for result logging, and real-time video services for environmental monitoring. The ROS nodes interface with devices such as robotic arms, RGB-D cameras, visible light cameras, laser rangefinders, and partial discharge sensors, enabling coordinated control. For example, the robotic arm’s kinematics are modeled using Denavit-Hartenberg parameters, with forward kinematics given by:
$$ T_i^{i-1} = \begin{bmatrix} \cos\theta_i & -\sin\theta_i \cos\alpha_i & \sin\theta_i \sin\alpha_i & a_i \cos\theta_i \\ \sin\theta_i & \cos\theta_i \cos\alpha_i & -\cos\theta_i \sin\alpha_i & a_i \sin\theta_i \\ 0 & \sin\alpha_i & \cos\alpha_i & d_i \\ 0 & 0 & 0 & 1 \end{bmatrix} $$
where \( \theta_i \), \( a_i \), \( d_i \), and \( \alpha_i \) are joint parameters. This allows precise positioning of the end-effector during operations. The core service acts as a gateway, receiving commands from the control center and orchestrating task execution. It also handles alarm generation and data synchronization. The database service uses SQLite for local storage, while file services upload images and logs to district servers via FTP. Real-time video services stitch feeds from panoramic ball cameras, providing operators with a comprehensive view of the robot’s surroundings. This architecture ensures that the intelligent robot can operate autonomously or under remote guidance, adapting to various substation layouts.
On the other side, the centralized control center & district server provides a unified management platform for multiple intelligent robots across different substations. It comprises video servers for storage and playback, district databases for aggregated data, file servers for archival, and service interfaces for user interaction. Operators access a graphical user interface to monitor robot status, view live video, and issue commands. The system supports simultaneous control of up to 10 intelligent robots, with task scheduling optimized to minimize conflicts. For instance, inspection routes are planned using A* algorithms to avoid obstacles, with path cost defined as:
$$ f(n) = g(n) + h(n) $$
where \( g(n) \) is the cost from start to node \( n \), and \( h(n) \) is a heuristic estimate to the goal. This enables efficient navigation in cluttered switch rooms. The control center also integrates predictive maintenance analytics, using data from robot sensors to forecast equipment failures. We have developed a failure prediction model based on support vector machines (SVM), with decision function:
$$ f(x) = \text{sign} \left( \sum_{i=1}^{n} \alpha_i y_i K(x_i, x) + b \right) $$
where \( \alpha_i \) are Lagrange multipliers, \( y_i \) are labels, and \( K \) is a radial basis function kernel. This enhances the proactive capabilities of the intelligent robot, shifting from reactive inspections to condition-based maintenance.
The hardware design of our intelligent robot is tailored for robustness and versatility in harsh substation conditions. As shown in the earlier image, the robot features a three-part structure: a top-mounted robotic arm, a middle floating platform for switchgear operation, and a bottom chassis for mobility. The robotic arm has six degrees of freedom, allowing dexterous manipulation of various controls. Its end-effector includes a customizable gripper for handling knobs, keys, and buttons, along with integrated visible light and infrared cameras for visual inspection. The floating platform utilizes linear actuators to adjust horizontally and vertically, compensating for misalignments when engaging with switchgear drawers or grounding switches. This platform is essential for operations like racking circuit breakers, where precision is critical to avoid damage. The chassis houses lithium-ion batteries, drive wheels, motor controllers, a lidar for SLAM (Simultaneous Localization and Mapping), and an industrial PC for processing. The lidar enables 2D mapping with scan matching via iterative closest point (ICP) algorithms, optimizing pose estimation. The robot’s power system is designed for extended operation, with battery capacity supporting up to 8 hours of continuous work. We have conducted extensive testing in environmental chambers to ensure reliability across temperature ranges from -20°C to 50°C, meeting IEC standards for substation equipment.
Functionally, the intelligent robot performs three primary tasks: remote switching operations, autonomous inspections, and emergency tripping. For switching, the robot navigates to designated switchgear, identifies target controls using vision algorithms, and executes sequences such as changing modes from “remote” to “local”. During inspections, it follows predefined routes, capturing thermal images with infrared cameras to detect overheating and using ultrasonic sensors for partial discharge monitoring. Data is analyzed onboard with convolutional neural networks (CNNs) for anomaly detection, where feature extraction is expressed as:
$$ z^{(l)} = W^{(l)} * a^{(l-1)} + b^{(l)} $$
followed by activation \( a^{(l)} = \sigma(z^{(l)}) \). This allows real-time identification of defects like corrosion or loose connections. In emergency scenarios, the robot can be dispatched to trip breakers manually, reducing response times from hours to minutes. We have deployed this intelligent robot in over 10 substations, achieving an average switching time of 13 minutes per interval, compared to 2 hours manually. The following table compares key performance metrics before and after deploying the intelligent robot:
| Metric | Manual Operation | Intelligent Robot Operation |
|---|---|---|
| Switching time per interval | 120 minutes | 13 minutes |
| Inspection coverage | 70% (spot checks) | 100% (full automation) |
| Safety incidents per year | 5 (estimated) | 0 (to date) |
| Operational cost reduction | Baseline | 60% |
| Data accuracy for diagnostics | Subjective | 95% (quantified) |
In conclusion, our development of an intelligent robot based on machine vision represents a significant leap forward in substation automation. By integrating advanced algorithms like Mean Shift for visual tracking, robust system architectures, and versatile hardware, we have created a solution that addresses the inefficiencies and risks of traditional operations. This intelligent robot not only enhances operational efficiency but also paves the way for fully unmanned substations, aligning with global trends toward smart grids. Future work will focus on enhancing AI capabilities, such as deep reinforcement learning for adaptive control, and expanding deployment to higher voltage levels. We believe that intelligent robots will become indispensable in power systems, driving safer, cheaper, and more reliable electricity distribution worldwide.