A research team at Shanghai University has proposed a task-transferable system architecture for embodied intelligence that inserts a stable functional layer between task requirements and concrete algorithm implementations, converting task variation into a problem of functional composition and configuration. The work, reported in the Journal of Mechanical Engineering under DOI 10.3901/JME.260754, argues that the persistent difficulty of reusing software across tasks and robot platforms in embodied intelligence can be addressed structurally rather than algorithmically.
The authors are Yangmin Xie, Qiaoni Yang, Yichuan Jiang, Wenshuai Shi, Hao Zhang, Yiwei Xu, Kai Liu, Jinghan Zhang, Yujie Shi and Yusheng Yang, affiliated with the Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, the Bionic Intelligent Robot Dreamworks for Special-purpose at Shanghai University, the School of Mechatronic Engineering and Automation, and the School of Future Technology. The study was supported by the National Natural Science Foundation of China under grants 62173220 and 62303294. The initial manuscript was received on 30 December 2025, a revised version on 27 April 2026, and the paper was published online on 26 August 2026.
1. Embodied Intelligence Moves From Laboratory Validation to Real-World Deployment
Embodied intelligence refers to robotic systems that achieve intelligent evolution through physical interaction with their environments. Unlike conventional robots driven by predefined programs, embodied intelligence systems are expected to learn and adapt while interacting with surroundings and with humans, delivering higher levels of flexibility and robustness in complex, dynamic settings. Applications are already being explored in intelligent manufacturing, service robotics and special-purpose operations.
The field is currently at a critical transition point between laboratory validation and real-world application. Software developed for embodied intelligence systems largely inherits the practices of the laboratory stage, where each deployment is built around a specific task, a specific robot platform and a specific application scenario. This highly customized pattern, described in the paper as a “one task, one software stack” approach, has begun to expose serious weaknesses as multi-task and multi-scenario requirements emerge.
Those weaknesses include software architectures that are difficult to reuse, high costs when switching tasks, and growing complexity in system extension and maintenance. Taken together, they amount to insufficient task transferability of embodied intelligence systems. The authors are careful to define the term precisely: task transferability here does not refer to the generalization capability of a single algorithm or model across many scenarios. It refers instead to the systemic capacity of a software architecture to support rapid construction, functional configuration, code reuse and continuous evolution when faced with different combinations of task requirements, robot platforms and application scenarios.
2. Two Dominant Paradigms and Their Structural Limits
Existing architectures for embodied intelligence fall broadly into two categories. The first is the end-to-end architecture built around holistic policy learning. The second is the hierarchical architecture organized around functional modules.
End-to-end architectures typically integrate perception, decision-making and manipulation into a single learned framework, allowing the system to generate low-level control actions directly from raw multimodal sensor input. The rise of large language models and vision-language models has strengthened this paradigm considerably, particularly in high-level semantic understanding, task description and instruction generation.
Representative examples cited in the paper include OpenVLA, which builds a unified vision-language-action model framework enabling a robot to output joint-level motion commands from multimodal inputs such as language text and images through a single policy. PaLM-E embeds visual information and robot state into a large language model through multimodal tokenization, generating executable action policies within a unified model. RT-2 introduces large-scale web knowledge during pretraining, allowing the model to transfer internet semantic knowledge to concrete robotic manipulation tasks such as object placement and tabletop organization.
Other efforts attempt to introduce intermediate structures inside end-to-end frameworks to strengthen hierarchical task representation. RT-H adds language-based hierarchical action representation on top of RT-2 to compensate for the shortcomings of purely end-to-end policies in complex task planning and execution. ChatGPT for Robotics uses code as an intermediary between semantic instructions and low-level control, improving interpretability and controllability. The authors note, however, that although these methods introduce intermediate representations such as language or code, their training and inference still rely on a unified model performing the full mapping from perception to action, making them extensions of the end-to-end paradigm rather than departures from it.
From a system architecture perspective, end-to-end methods implicitly couple task structure, environmental assumptions, robot morphology and control interfaces into a unified policy representation. Generalization is therefore typically confined to scenarios where task distributions and platform conditions are relatively homogeneous. This coupling makes it difficult for embodied intelligence systems to achieve effective reuse across tasks and platforms, with transfer depending more on parameter fine-tuning or retraining than on system-level capability reorganization.
On the theoretical side, mainstream end-to-end policies often fail to account adequately for behavior modeling in out-of-distribution environments, safety in long-horizon physical interaction, and continuous self-adaptation to new tasks. End-to-end large models also struggle to explicitly incorporate predictive planning and adaptive behavior representations such as world models, and find it difficult to impose explicit robot and environmental parameter constraints, which fundamentally limits their generality. In engineering practice, robustness and reliability of end-to-end policies degrade markedly under real conditions such as illumination variation and dynamic environments. Large-model-driven embodied intelligence also faces high costs of acquiring physical interaction data, demanding compute and real-time requirements, and difficulties in behavior explanation and safety verification. These are widely regarded as key factors constraining efficient multi-task, multi-scenario transfer and industrial deployment.
Hierarchical architectures organized around functional modules have a deeper theoretical and engineering foundation in robotics and have long been viewed as an important technical path toward complex robot behavior. Since the introduction of classic robot architectures such as the three-layer perception-planning-execution structure, hierarchical behavioral control, and the separation of task planning from motion planning, modular and hierarchical design has been widely used to meet the construction needs of complex robotic systems.
Research generally holds that explicitly decoupling perception, cognition, decision-making and execution improves maintainability, interpretability and debuggability, and is an important prerequisite for engineering deployment of complex robotic systems. In embodied intelligence research, this idea has evolved into system architecture design centered on functional hierarchy. One example cited is a framework built around three dimensions—industrial eyes, industrial hands and industrial brain—which models multimodal perception, compliant manipulation and process-level decision-making separately to support fine-grained, adaptive manipulation in complex manufacturing scenarios. Such layered modeling provides a feasible path for process modeling, state monitoring and online adjustment under limited perception conditions, and reflects the stability and controllability advantages of hierarchical architectures in industrial settings.
From the perspective of safety and system trustworthiness, modular hierarchical architectures are also more conducive to graded risk analysis and responsibility allocation. One cited study constructs an L1–L5 hierarchical framework that systematically defines and safeguards perception reliability, decision safety and execution controllability. Such work emphasizes that explicit hierarchical structure provides the necessary structural basis for safety verification, fault isolation and system auditing, and is an indispensable design principle for real-world deployment.
With the development of foundation models, hierarchical architectures are also viewed as one of the realistic paths for integrating large model capabilities. Large language models or vision-language models can be embedded as high-level cognition and reasoning modules within a hierarchical system for task understanding, symbolic reasoning or high-level planning, while low-level perception and control remain in dedicated modules to satisfy real-time, stability and safety requirements. This “model as module” integration approach alleviates to some extent the engineering pressure of end-to-end solutions in computing power, data and reliability, and provides architectural support for controlled use of large model capabilities.
Yet existing hierarchical embodied intelligence systems based on module partitioning still face significant challenges. Most systems depend heavily during the design phase on manual modeling of task flows and functional boundaries by researchers, and are typically constrained by the hardware interfaces and middleware of a specific robot platform. Reuse across different tasks, platforms and application scenarios therefore still requires considerable engineering effort. How to introduce a more general representation mechanism and automated configuration capability while preserving the clarity and controllability of hierarchical structure remains a key open problem for hierarchical embodied intelligence architectures.
3. The Structural Gap That Algorithmic Progress Alone Cannot Close
Viewed from the system-level perspective of task transferability, neither mainstream paradigm effectively controls how task change propagates through the software system. End-to-end learning methods implicitly couple task objectives, environmental assumptions and control policies within a unified model, so when task requirements or robot platforms change, the system cannot achieve rapid adaptation by recombining existing capabilities. Existing hierarchical modular methods achieve a degree of functional decoupling, but their module partitioning and system organization often implicitly embed particular task flows and platform assumptions, so task change still requires manual adjustment of system structure or interface logic.
This indicates that embodied intelligence systems currently lack a software architecture mechanism capable of systematically converging task change into a problem of functional composition and configuration at the architectural level. Effective isolation between task requirements, platform differences and algorithm implementations is not yet achieved. This structural gap means that algorithmic capability improvement or module-level refinement alone cannot fundamentally support reusable construction and long-term evolution of embodied intelligence systems across multiple tasks and scenarios. A higher-level mechanism for systematic organization and mapping is required.
4. Functional Abstraction as an Architectural Principle
To address this problem, the authors propose a systematic technical framework that starts from task requirements and maps them level by level down to functional structure and technical implementation. Unlike conventional Robot Operating System development, where task requirements are organized as communication between concrete algorithm nodes, this framework builds an explicit three-tier system model comprising a task layer, a functional layer and an algorithm layer.
Functional abstraction is introduced as an intermediate structural layer, decoupling task requirements from algorithm implementations at the architectural level. Although the Robot Operating System provides general communication and runtime support for developing embodied intelligence systems, it cannot itself decompose task requirements, nor can it guarantee that different algorithms are substitutable at the functional semantic level. By contrast, the proposed framework shifts the construction of embodied intelligence systems from combining concrete algorithm modules to combining and configuring functional capabilities, so that task change affects only functional composition and configuration and no longer triggers reconstruction of the underlying algorithm system.
The authors emphasize that the framework does not aim to improve the internal mechanisms or performance of specific perception, planning or control algorithms, nor does it introduce new learning models. It treats existing algorithms and models as reusable functional components and focuses on their system-level organization and architectural design principles under task-driven conditions.
5. Design Goal: Architectural Reusability as the Answer to Task Diversity
In real application scenarios, the application scenario, task flow and robot platform together determine a concrete task, and these three elements often differ significantly across applications. Existing construction methods typically bind task logic, algorithm implementation and software architecture tightly together, so that a change in any one element requires extensive modification of the system, making continuous iteration and cross-task reuse difficult.
Under the realistic constraint that a general-purpose agent covering all tasks is not yet an engineering possibility, designing a reusable software architecture for embodied intelligence becomes a key issue for translating related technologies into deployed applications. In conventional embodied intelligence software development, developers must build task-level information flow directly on top of algorithm modules. As task types and the number of algorithm modules grow, the algorithm pool expands continuously and the design, debugging and maintenance of task information flow becomes a highly complex manual process.
Developers must repeatedly handle interface adaptation between algorithms and between algorithms and tasks. Software development cost therefore grows approximately linearly with the number of tasks and exhibits pronounced combinatorial complexity as the algorithm pool expands, making long-term evolution and reuse difficult.
In response, the authors treat “function” as a relatively stable abstract unit in the embodied intelligence system and introduce a functional layer as an intermediate structure between the task layer and the algorithm layer. This design explicitly distinguishes task requirements, functional capabilities and algorithm implementations, isolating task-side uncertainty from continuous algorithm-side evolution. Under this architecture, software developers focus primarily on the organizational relationship between task requirements and functional capabilities, while framework developers maintain the mapping between functions and algorithms and the underlying supporting tools.
During system construction and evolution, software developers configure the information flow of the embodied agent based on function nodes, and the correspondence between the functional layer and the algorithm layer remains transparent and stable to them. When task requirements change, system adaptation is achieved by adjusting the combination of function nodes without modifying concrete algorithm implementations. When algorithm capabilities are updated or extended, new algorithms are connected to the system according to existing functional interfaces, again without task-specific architectural adjustment.
From an architectural standpoint, this design confines task change to the level of functional composition and confines algorithm change to the level of functional implementation, preventing changes in task requirements from acting directly on underlying algorithms and software structure. It thereby provides a stable foundation for system reuse and extension across multiple tasks and platforms.
6. The Functional Layer: Stable Abstractions of Task-Oriented Capability
The functional layer occupies the core position in the embodied intelligence architecture. Its goal is to stably abstract the fundamental capabilities that an embodied intelligence system requires when executing tasks. The layer is concerned with what types of capability the system can provide, not with how those capabilities are concretely implemented.
Organized around the typical information flow and capability loop of an embodied agent, the functional layer is divided into three major subsystems. The edge perception system is responsible for real-time acquisition of local environment and body state, emphasizing high frequency and low latency. The brain system is responsible for scene understanding, task decision-making and plan generation, emphasizing semantic abstraction and task reasoning. The cerebellum system is responsible for motion execution, stability control and human-robot interaction, emphasizing continuous control and stability assurance. According to specific task requirements, these are concretized within the three subsystems into capabilities such as environment perception and modeling, localization and navigation, target interaction, motion planning and behavior execution, each corresponding to a class of robot capability unit that can be invoked by a task.
The definition of a function emphasizes four characteristics. First, task-oriented semantics: functional descriptions serve task objectives and execution logic directly, avoiding the introduction of concrete algorithms or implementation details. Second, composability: different functional modules can be flexibly combined according to task requirements to form diverse task flows. Third, cross-platform stability: functional definitions remain consistent across different robot morphologies, sensor configurations and application scenarios. Fourth, interface clarity: functions provide services externally through clear input-output relationships, serving as standard invocation units within the system.
Through the introduction of the functional layer, task requirements no longer act directly on algorithms and software implementation. They are first mapped to capability requirements at the functional level, providing a unified structural entry point for system construction.
7. The Algorithm Layer: A Substitutable Implementation Mechanism
The algorithm layer implements the various system capabilities defined in the functional layer. Its core concern is the selection, combination and continuous optimization of concrete technical implementation approaches. Unlike the stable capability abstraction role assumed by the functional layer, the algorithm layer is explicitly designed as an evolvable, substitutable technical implementation layer. Algorithms are treated as technical means of realizing functions rather than as the organizational core of the system architecture, avoiding frequent adjustment of system structure whenever algorithms are updated or technical routes change.
In the proposed architecture, the functional layer and the algorithm layer are explicitly decoupled. The same functional capability is no longer bound to a single algorithm implementation but may correspond to multiple technical implementation paths, such as model-based methods, data-driven methods, or hybrid strategies combining both. The algorithm layer allows the system to flexibly adjust underlying implementation approaches according to task precision requirements, computing resource constraints or changes in operating environment, while keeping the functional structure stable.
The algorithm layer also serves as the primary carrier for technical evolution and capability upgrading. As perception algorithms, learning models and optimization methods continue to advance, new algorithms can be introduced gradually without breaking existing functional definitions, participating in system operation as substitutes or enhancements for functional implementations. This allows the system to absorb new technical results continuously during long-term operation and multi-task deployment, achieving gradual capability evolution without systematic reconstruction of functional layer structure or task organization.
Because the functional layer and the algorithm layer are explicitly decoupled, the substitutability of different algorithm modules for the same functional task in terms of program interfaces, data semantics and invocation methods is guaranteed in advance by the architecture. In actual deployment, the system can therefore rapidly improve the overall capability of an existing task software system by directly replacing or upgrading algorithm modules, without modifying the original task flow, functional organization or system integration structure.
The algorithm layer follows four design principles. The first is a one-to-many mapping relationship: functions and algorithms are not in one-to-one correspondence, and the same function can be implemented by multiple different algorithms. The second is interface consistency: algorithm modules implementing the same function maintain consistent input and output data forms, ensuring that algorithms can be replaced without introducing additional adaptation costs. The third is substitutability and extensibility: algorithm implementations can be dynamically replaced or extended according to task requirements and system resource conditions without affecting functional layer definitions. The fourth is evolution friendliness: the introduction of a new algorithm serves only as an additional implementation option for an existing function and does not trigger changes to system functional structure or task organization.
8. An Instantiated Architecture Built on Functional Abstraction
To verify the operability of the architecture, the authors present a system design instance based on functional abstraction that focuses on how the architecture can be implemented in practice. The instance does not pursue optimal system performance on a single task, nor does it exhaustively compare existing algorithms. It focuses on verifying how mature algorithms can be effectively organized and reused through the architecture to flexibly support diverse tasks.
The instantiation follows two principles. In functional layer design, the authors build around the capability loop required for real task execution and abstract related capabilities into a set of functional modules with clear semantics and unified interfaces. Functional definitions correspond directly to task requirements and system behavior without binding to concrete algorithm implementations, allowing task flows to be described through combinations of functional modules and providing a stable foundation for system reconstruction and reuse across tasks. In algorithm layer design, priority is given to solutions with high engineering maturity, such as open-source implementations that have been experimentally validated or key technologies with an engineering application base. This choice does not exclude frontier methods; rather, it reflects the goal of system instantiation in this chapter, positioning algorithms as concrete implementation forms of functions. Under this design, algorithm replacement or updating does not affect functional definitions and task organization structure.
Following the principles of input-output closure, substitutability and verifiability, the authors abstract and summarize 25 core functional modules covering perception, cognition, planning and execution, organized across the three functional systems. Each function corresponds to a class of capability unit that can be directly invoked by a task, describing what the system can accomplish without limiting its concrete implementation.
| Subsystem | Functional modules | Functional emphasis |
|---|---|---|
| Edge perception system | SLAM, scene recognition, environment reconstruction, object detection, object localization, object tracking, terrain classification, semantic map construction, tactile recognition, hand pose recognition | Real-time acquisition of local environment and body state; high frequency and low latency |
| Brain system | Behavior recognition and intention inference, natural language understanding, task planning, grasp strategy generation, human pose perception, human-robot task allocation | Scene understanding, task decision-making and plan generation; semantic abstraction and task reasoning |
| Cerebellum system | Manipulation control, gait coordination, balance control, path planning, trajectory planning, collision detection, trajectory tracking, teleoperation enhancement, pose mapping | Motion execution, stability control and human-robot interaction; continuous control and stability assurance |
9. Inside the Edge Perception System
The edge perception system handles real-time perception of the external environment and the agent’s own state on the robot side, and is the functional layer that interacts most directly with sensor data. Its design goal is not complete environmental understanding or task reasoning, but the provision of stable, continuous and spatially consistent perception input for subsequent brain decision-making and cerebellum control, ensuring operability and real-time responsiveness in real environments.
SLAM provides a unified spatial reference frame for all perception results and is the most fundamental capability for localization and consistency maintenance in the edge perception system. In engineering practice, LIO-SAM achieves strong robustness through tightly coupled optimization of lidar and inertial measurement units in structurally stable environments with complex motion. VINS emphasizes visual-inertial fusion and suits mobile platforms where cameras are the primary sensors. ORB-SLAM provides a modular visual SLAM framework and is a classic baseline for three-dimensional visual localization and sparse mapping. FAST-LIO2, based on tightly coupled iterated Kalman filtering, achieves high-frequency point cloud-to-map matching and efficient incremental mapping, suitable for real-time localization under high-speed motion.
Scene recognition performs place re-identification and historical environment matching during long-term operation, supporting global consistency maintenance and drift correction. Scan Context encodes three-dimensional laser scans into a two-dimensional description matrix insensitive to viewpoint changes, enabling robust scene matching. DBoW2 performs efficient loop closure detection based on a visual bag-of-words model. iBoW-LCD adds geometric consistency verification on this basis, improving recognition reliability in structurally similar scenes.
Environment reconstruction recovers three-dimensional scene structure from multi-view or multimodal observations, providing a geometric basis for object perception and spatial understanding. SFM jointly estimates camera poses and sparse point clouds for basic structure recovery. Three-dimensional Gaussian splatting adopts a learnable Gaussian representation for continuous field modeling, supporting high-quality reconstruction and real-time rendering. Multi-view stereo performs dense matching under known camera poses and suits scenarios with high geometric accuracy requirements. ICP minimizes geometric error between point clouds to achieve precise data registration and, combined with multi-view point cloud data, high-quality environment reconstruction.
Object detection identifies targets of interest in perception data and outputs their category and two-dimensional spatial extent, serving as a precondition for subsequent localization, tracking and interaction. YOLO, as a single-stage detector, balances speed and accuracy and suits real-time systems. Mask R-CNN provides fine boundary information through instance segmentation. DETR, based on a Transformer set prediction mechanism, simplifies the detection pipeline and performs stably in structured scenes.
Object localization estimates the three-dimensional position or pose of an object based on detection results, providing key support for grasping and interaction tasks. Frustum-PointNet maps two-dimensional detection results into a point cloud frustum for three-dimensional localization. CenterNet3D performs monocular three-dimensional object localization by regressing three-dimensional center points. SMM-POD fuses appearance and geometric information to enhance robustness in cluttered scenes. Some researchers also combine object detection algorithms such as YOLO with depth sensing to achieve efficient object localization.
Object tracking performs continuous state estimation and identity maintenance for dynamic targets, an important foundation for multi-target collaboration and safety planning. JRMOT learns target embedding features alongside detection to strengthen cross-frame association. GlobalTrack handles re-identification after long-term occlusion through global retrieval. ODTrack provides a lightweight integrated detection and tracking implementation suited to compute-constrained platforms.
Terrain classification evaluates ground traversability by analyzing visual or geometric features, providing a basis for mobility planning and risk avoidance. Random Forest relies on handcrafted features for terrain discrimination. DeepTerramechanics models wheel-terrain or foot-terrain interaction mechanisms through deep networks to improve generalization in complex terrain. LB-ASN builds boundary-sensitive feature representations based on deep learning and finely models transition regions between different terrains, suitable for road edge recognition in autonomous driving scenarios.
Semantic map construction associates semantic information with geometric structure to form an environment representation directly usable by task planning and interaction modules. Panoptic-Mapper fuses visual perception results online to build dense maps containing instance semantics, suitable for long-term operation in dynamic environments. SA-LOAM assigns semantic labels to point clouds during lidar SLAM, enhancing understanding of structured scenes such as roads.
Tactile recognition acquires contact state, force perception and surface property information through tactile sensors, providing supplementary perception for fine manipulation and stable grasping. Transferable tactile transformers use a Transformer structure to encode time-series tactile signals and obtain transferable tactile representations. Deep tactile CNN extracts structural features from tactile array images for shape and contact state recognition. GelSight-based CNN relies on high-resolution tactile imaging to achieve micro-texture and fine surface analysis, suitable for near-contact manipulation scenarios such as assembly and inspection.
Hand pose recognition estimates human hand keypoints and pose information, providing structured input for teleoperation, teaching and interactive control. MediaPipe emphasizes lightweight operation and real-time performance, suitable for edge devices and mobile platforms. DeepPose achieves high-precision pose regression through deep networks, suitable for manipulation and analysis scenarios with high accuracy requirements.
10. Inside the Brain System
The brain system handles cognition and decision-making for the embodied agent. Its core role is to transform environment, target and state information output by the edge perception system into structured task decisions and behavior strategies, providing clear instructions for subsequent execution. The system focuses on task understanding, behavior organization and collaborative decision-making, and does not directly participate in low-level motion control.
Behavior recognition and intention inference determines the current action type from behavior state or trajectory information and infers subsequent behavior trends, an important precondition for human-robot collaboration and safety decisions. Social LSTM models multi-agent trajectories and characterizes mutual influence between individuals, suitable for crowd behavior prediction and collaborative motion scenarios. ST-GCN takes a human skeleton as graph-structured input and jointly models joint topology and temporal evolution relationships, offering strong robustness to occlusion and appearance variation and suitable for fine-grained action recognition and interaction intention judgment.
Natural language understanding parses human language instructions into structured task semantics or intention representations, serving as the entry point for task planning and human-robot interaction. TinyBERT suits rapid parsing of short control instructions on resource-constrained platforms. Llama3-8B provides strong general semantic understanding and multi-turn dialogue capability, suitable for complex interaction scenarios. Qwen3 is optimized for Chinese language and industrial terminology, suitable for Chinese-dominant human-robot interaction environments. AV2Vec further introduces cross-modal semantic representation to support semantic alignment and task understanding under multi-source input.
Task planning transforms high-level task objectives into task execution flows with temporal structure and is the core decision unit in the brain system. Behavior trees organize decision logic through node structures with good interpretability and maintainability. PDDL performs state-space search based on symbolic descriptions, suitable for discrete task planning. OpenVLA uses joint vision-language-action modeling to improve generalization in unstructured environments. RT-H adopts a hierarchical planning structure with language as intermediate guidance, supporting decomposition and execution of complex long-horizon tasks.
Grasp strategy generation provides feasible grasping approaches and execution strategies based on object perception results, a key component of manipulation tasks. Dex-Net is trained on large-scale synthetic data and offers good stability and zero-shot generalization in industrial grasping scenarios. Diffusion policy generates complex grasping strategies by producing continuous action sequences, suitable for unstructured environments. Grasp pose detection rapidly generates and evaluates grasp candidate poses based on point cloud geometry, with high computational efficiency suitable for embedded or real-time systems.
Human pose perception estimates the spatial distribution of human joints and is an important foundation for behavior understanding and collaborative decision-making. OpenPose handles real-time multi-person keypoint detection, suitable for crowd scenarios. HRNet achieves high localization accuracy in single-person pose estimation, suitable for fine motion analysis. mm-Pose performs three-dimensional pose perception based on millimeter-wave radar and maintains stable output under limited illumination or visual occlusion.
Human-robot task allocation addresses multi-agent collaboration scenarios, allocating tasks according to task requirements and agent capabilities. Market-based multirobot coordination completes dynamic task allocation through bidding mechanisms, suitable for heterogeneous agents and online task insertion scenarios. The Hungarian algorithm solves one-to-one optimal assignment problems, suitable for centralized task allocation with complete information and controllable scale.
11. Inside the Cerebellum System
The cerebellum system converts task decisions and motion intentions output by the brain system into physical motion that can be executed stably, directly addressing dynamic constraints, real-time feedback and safe execution requirements. The system focuses on motion generation, control and constraints, and does not undertake task semantic understanding or reasoning.
Manipulation control serves as the direct interface between the cerebellum system and execution hardware, converting task-level motion intentions into stably executable control commands. Impedance control achieves compliant interaction by adjusting equivalent stiffness and damping at the end effector. Admittance control maps external forces into motion corrections, suitable for human-guided manipulation. Hybrid position-force control constrains pose and contact force separately within the task space, commonly used in assembly and contact operations. PID, as a basic closed-loop control method, remains widely adopted in industrial execution layers requiring high reliability.
Gait coordination generates and coordinates motion patterns with temporal consistency to ensure stability of continuous walking or periodic motion. DeepMimic generates highly dynamic motion patterns by imitating reference actions, suitable for simulation or skill generation. ZMP, based on the zero moment point stability criterion, is commonly used for robust bipedal gaits in structured scenarios. RMA improves walking robustness of legged robots in unknown terrain through online adaptation to environmental change.
Balance control maintains overall posture and system stability during motion. LQR obtains efficient feedback control laws based on linearized models. MPC explicitly handles multiple constraints through rolling optimization, suitable for balance and motion control under complex operating conditions. HZD guarantees theoretical stability of periodic gaits by constructing virtual constraints, commonly used in the analysis of bipedal and quadrupedal systems.
Path planning searches for feasible traversable paths given an environment model and start and end states. A-star suits efficient shortest path search in discrete environments. RRT*-Connect rapidly explores high-dimensional configuration spaces and progressively optimizes path quality. BFMT and BIT* improve search efficiency and solution quality in complex environments through asymptotically optimal sampling strategies.
Trajectory planning introduces the time dimension on the basis of path results, generating executable trajectories that satisfy kinematic and dynamic constraints. Polynomial parameterization generates smooth trajectories by satisfying boundary conditions. B-splines offer good local adjustability, suitable for online adjustment. TOPP obtains time-optimal execution plans satisfying constraints through time parameterization.
Collision detection identifies potential collision risks during planning and execution. SAP suits rapid broad-phase screening. BVH reduces fine detection overhead through hierarchical bounding structures. SAT performs exact collision determination between convex bodies. RobotSDF supports real-time obstacle avoidance and contact modeling under complex morphologies through continuous distance field representation.
Trajectory tracking ensures that actual robot motion stably follows the desired trajectory. ARC combines adaptive and robust feedback to address model uncertainty. MPC achieves high-precision, constraint-aware tracking through online optimization. GVF achieves global convergence through vector field guidance. DWA suits real-time following and obstacle avoidance for mobile robots in dynamic environments.
Teleoperation enhancement addresses human-robot collaborative control scenarios, constraining and correcting manual input to improve safety and efficiency. TRAC-IK maps end effector manipulation intentions into joint space. PDNN improves manipulation adaptability through a data-driven approach. Virtual fixtures guide manipulation through software constraints, commonly used in high-precision teleoperation.
Pose mapping establishes consistent motion representation between different morphologies or between humans and robots. DMP parameterizes demonstrated trajectories through an attractor system. AMP learns natural motion priors to improve motion expressiveness. Motion retargeting achieves high-fidelity motion mapping across morphologies through constrained optimization.
12. A Unified Data Interface Dictionary
In the proposed architecture, functional modules exist as basic organizational units of system capability, while algorithms are introduced only as concrete implementations of functions. System design does not assume the existence of a specific algorithm, and interaction between modules does not depend on internal intermediate states or private data representations within algorithms. To guarantee this, the functional layer imposes unified constraints on module inputs and outputs and introduces an explicit data interface specification.
Under this framework, any algorithm satisfying the data interface requirements of the corresponding functional module can be connected to the system as an implementation of that function. Algorithm replacement occurs only within a module and does not require synchronous modification of task flows or upstream and downstream modules. By confining algorithm differences within interface boundaries, dependency relationships between functional modules are fixed at a stable data interaction layer, making algorithm replacement and evolution a structural operation rather than an additional engineering transformation process.
| Interface type | Name | Abbreviation | Description |
|---|---|---|---|
| Sensor data | SensorData | SD | Raw multimodal sensor data including vision, depth, IMU, tactile and lidar |
| Robot kinematic state | RobotKinematicState | RKS | Joint angles, overall motion posture, velocity and related kinematic information |
| Scene representation | SceneRepresentation | SR | Unified description of environmental geometric structure and topology, including colored point clouds, voxel grids, 3D Gaussian splatting and occupancy maps |
| Semantic representation | SemanticEntitySet | SE | Structured set composed of object semantic categories and related attributes |
| Target representation | TargetDescriptor | TD | Abstract feature set of one or more targets, including position, kinematic parameters and category |
| Scene graph | SceneGraph | SG | Graph-structured environment representation including spatial relations, semantic relations and entity connections |
| Human or body state | BodyState | BS | Human skeleton, pose and related information |
| Task intent | TaskIntent | TI | Task intent description for the overall task and its constituent objects, including task objectives and human intention |
| Task representation | TaskGraph | TG | Graph-structured representation composed of task elements and their sequential connections |
| Safety constraints | ConstraintSet | CS | Collision constraints, posture constraints, stability and boundary conditions |
| Behavior representation | ActionPrimitive | AP | Atomic action or skill unit of a robot or human |
| Trajectory representation | Trajectory | Tra | Sequence of robot motion paths in the spatiotemporal domain |
| Control command | ControlCommand | CC | Low-level control quantities driving robot motion, such as velocity and torque |
13. How Software Developers Assemble Functions and Algorithms
On the basis of the instantiated functional and algorithm layer design, software developers do not participate in underlying architecture design. Instead, they treat the already stabilized functional abstractions and algorithm interfaces as given and transform system construction into function selection and algorithm configuration. In concrete use, the framework involves two core steps.
First, users design the overall execution flow of a task according to the specific application scenario and task objective, and decompose it into a number of core functional modules already defined in the functional layer, clarifying the invocation relationships and sequence of each function during task execution, thereby forming a functional composition structure corresponding to the task.
Second, once the functional composition is determined, users select appropriate implementation schemes for each functional module in the algorithm layer according to operating environment characteristics, real-time requirements and computing resource constraints, completing the instantiation mapping from functions to algorithms. To reduce engineering burden during composition, the function-to-algorithm correspondence table and the functional support relationship diagram provide explicit support for this process.
14. Experimental Validation Across Two Heterogeneous Platforms
To verify the feasibility and engineering value of the proposed architecture in practical application, the authors selected two representative embodied intelligence tasks and conducted experimental validation on two heterogeneous robot platforms. The tasks were deliberately chosen to be more complex than demonstrative tasks involving a single capability or simplified flow.
The first task is autonomous grasping with a mobile manipulator, a short-flow task with strong coupling between mobility and manipulation. The system is required to simultaneously complete environment perception, autonomous navigation and grasping in an unstructured environment. The second task is cross-floor patrol by a humanoid robot, a long-flow composite task involving cross-space, multi-stage decision-making, placing higher demands on task organization capability and execution reliability.
These two tasks correspond to common but difficult application forms in embodied intelligence systems. Compared with desktop organization, single-arm grasping or simple navigation, they are closer to real application scenarios in terms of functional coverage, process complexity and system integration difficulty.
| Category | Comparison dimension | Mobile manipulator grasping task | Humanoid robot patrol task |
|---|---|---|---|
| Hardware platform | Manipulation platform | Single arm | Dual arm |
| Hardware platform | Degrees of freedom | 3 (chassis) + 6 (manipulator) + 1 (gripper) | 29 (whole body) + 6 × 2 (dexterous hands) |
| Hardware platform | Mobility mode | Tracked chassis | Bipedal walking |
| Hardware platform | Sensor configuration | Lidar, stereo camera, IMU, microphone | Lidar, stereo camera, IMU, microphone |
| Task complexity | Task flow | Short flow | Long flow |
| Task complexity | Decision mode | Multi-stage | Multi-stage |
| Task complexity | Manipulation mode | Simple | Complex |
| Scenario difference | Workspace | Indoor unstructured environment | Cross-floor structured space |
| Scenario difference | Interaction objects | Fixed | Complex and diverse |
| Scenario difference | Execution time | Relatively short | Long |
15. Task One: Autonomous Grasping With a Mobile Manipulator
For the mobile manipulator autonomous grasping task, the overall objective is for the robot to complete a human voice instruction within a single room, such as fetching a bottle of water. Analysis of the operation process involves ten capability modules: natural language understanding, task planning, SLAM, path planning, object localization, trajectory planning, trajectory tracking, collision detection, grasp strategy generation and manipulation control. It is a typical mobility-manipulation strongly coupled task.
In the constructed functional flow, the mobile manipulator first parses the target object as a water bottle through the natural language understanding module, and the task planning module decomposes the instruction into staged operations: determine the spatial position of the bottle, navigate to the grasping point, and execute the grasping task.
In the first stage, the SLAM module builds a three-dimensional model of the environment, the object localization module determines the position of the bottle in three-dimensional space, and the path planning module generates a collision-free safe path. In the second stage, the SLAM module provides precise self-localization information and a three-dimensional model of the surrounding environment during movement, the trajectory planning module performs local path obstacle avoidance planning, and the trajectory tracking module carries out path-following navigation. In the third stage, the object localization module localizes the target to be grasped in real time, the grasp strategy generation module provides the robot stopping and manipulation pose, the collision detection module provides collision detection information for strategy generation, the trajectory planning module generates the manipulator manipulation trajectory, and the manipulation control module completes the operation according to the trajectory. These functional modules are organized into a clear information flow and control flow according to task execution sequence, forming a complete embodied intelligence architecture.
In the algorithm configuration stage, several choices were made in view of incomplete sensing, dynamic occlusion and mobility-manipulation coupling in indoor unstructured environments. Natural language understanding used the AV2Vec method based on sequence modeling to support multimodal parsing of voice instructions. SLAM used the LIO-SAM framework with tightly coupled lidar and inertial sensing, adapting to indoor line and plane geometric features. Path planning used the A-star algorithm for global navigation path search, suitable for short-range indoor spaces. Object localization used Frustum-PointNet to achieve robust instance recognition of diverse objects. Trajectory planning used B-splines to generate smooth local obstacle avoidance optimization trajectories, and collision detection used RobotSDF for continuous collision detection to ensure manipulation safety in complex occluded spaces. Grasp strategy generation used the FIRM algorithm to form grasp pose generation with joint geometric and mechanical optimization, and manipulation control used a PID controller to achieve smooth tracking of the grasping trajectory.
The authors note that this algorithm configuration is only one way of combining algorithm modules through the functional-to-algorithm mapping of the architecture. With the task functions unchanged, different algorithms within the architecture can be replaced or upgraded to decouple algorithm research and development from software development.
Under this functional composition and algorithm configuration, tests were conducted on target objects distributed at five different positions in the environment. The system was able to complete the grasping task stably under the same functional flow for all of them. The FIRM algorithm was able to stably select the solution with the highest reachability and manipulation margin among multiple candidate poses, and the manipulability distribution at different spatial positions showed consistency. The collision detection module based on RobotSDF maintained good continuity and precision in distance field estimation; compared with traditional geometry-based detection methods, implicit morphology representation produced no obvious jumps during collision detection and achieved sub-millisecond collision detection efficiency, enabling real-time trajectory generation. The trajectory planning module generated smooth, collision-free grasping trajectories satisfying the precision requirements of grasping operations in multi-obstacle spaces.
Overall, the first task verified that in a complex embodied task requiring the coordination of multiple capabilities, the proposed functional abstraction architecture can support the construction of a complete task flow. Through reasonable combination of multiple functional module categories, the composite robot achieved autonomous mobility and manipulation in a complex spatial environment while maintaining good execution stability.
16. Task Two: Cross-Floor Patrol by a Humanoid Robot
The humanoid robot cross-floor patrol task is a composite task spanning time sequences, involving serialized stages and conditional branches. It covers multiple cyclic stages and requires the robot to continuously invoke perception, planning and execution capabilities across different spatial regions and environmental states.
Based on the function-module-centered architecture, the functional organization structure comprises ten capability modules: natural language understanding, task planning, SLAM, path planning, trajectory tracking, object detection, object localization, manipulation control, environment reconstruction and trajectory planning. Although the task content and flow are entirely different from the first task, only one functional module changed.
In the constructed functional flow, the humanoid robot plans the task into five distinct stages according to the received language instruction. The first stage is autonomous navigation on the first floor, moving autonomously from the current position to the elevator entrance. Here the SLAM module completes self-localization, the environment reconstruction module completes precise map reconstruction, the path planning module generates a path to the target based on the relationship between the pre-stored map and the current map, and trajectory tracking completes path-following navigation control.
The second stage is entering the elevator. Object detection detects the QR code to complete elevator identification recognition and recognizes button and elevator door states, object localization completes precise localization of the elevator button, SLAM provides real-time calibrated self-localization information, path planning provides path planning for the humanoid robot entering the elevator and generates a manipulator path for pressing the button, trajectory tracking completes path-following motion control, trajectory planning smooths the manipulator operation trajectory, and manipulation control drives the manipulator according to the path to complete the elevator button operation.
The third stage is exiting the elevator. SLAM provides self-localization information, object detection provides elevator door state information, path planning provides the elevator exit path, and trajectory tracking completes path-following motion control.
The fourth stage is autonomous navigation on the second floor, functionally identical to first-floor navigation, with the navigation target being the room door specified by the instruction.
The fifth stage is opening the door. Object detection detects the door handle, object localization provides the precise three-dimensional spatial position of the door handle, and path planning, trajectory planning and manipulation control operate similarly to the second stage but with the door handle as the manipulation object, completing the rotation of the handle and the pushing of the door. Although the content of each task stage differs, the functional modules of the humanoid robot show strong reusability across stages.
In algorithm selection, given the long execution time and varied scenarios of the patrol task, the emphasis was on maturity and long-term operational reliability. Although the functional modules are highly similar to those of the first task, the algorithm configuration can differ considerably. Natural language understanding used the Llama3-8B model, balancing semantic understanding capability with deployability on edge computing, able to parse patrol instructions while assisting in generating a preliminary task structure. Object detection and object localization both used YOLO to support recognition of key targets such as buttons and doors with high real-time performance. Since the manipulation environment contains few obstacles, manipulator path planning used RRT*-Connect, achieving efficient feasible path search in the high-dimensional configuration space of the manipulator. Environment reconstruction, as a newly added functional module, used the ICP algorithm to complete precise global relocalization relative to the existing map. Algorithm configurations for other functional modules remained unchanged relative to the first task.
In total, although only one functional module changed, four algorithm modules changed between the second task and the first task.
The cross-floor patrol test evaluated key functional modules quantitatively, focusing on trajectory tracking accuracy, environment reconstruction consistency, and target recognition and localization accuracy.
Results showed that under continuous patrol and cross-floor movement conditions, the robot was able to stably achieve centimeter-level trajectory tracking accuracy and complete reliable stops at designated positions, without obvious cumulative deviation or instability. In environment reconstruction, the real-time three-dimensional map maintained good spatial consistency with the real environment, with the vast majority of point cloud registration errors below 0.04 m. At the same time, the system showed no map degradation or structural drift during long-term operation, and environment reconstruction and localization during movement demonstrated good stability. In target perception, the real-time target recognition and localization module was able to continuously and accurately recognize multiple types of task targets, such as elevator buttons, QR codes, door handles and elevator door states, with key target three-dimensional localization errors below 1 cm. Experimental statistics showed that object detection and localization accuracy remained above 98 percent, providing reliable input for environmental interaction and reliable task execution.
These results indicate that the embodied intelligence system built under the proposed framework can maintain stable operation in long-duration, multi-stage complex tasks, with functional modules effectively coordinated under a unified architecture. The system not only completes the complex patrol flow but also reaches engineering-usable levels in key perception and execution metrics.
17. Quantifying Migration Cost With the Task Transfer Index
To evaluate the improvement in task transferability achieved by the proposed architecture in embodied intelligence software construction, the authors introduce the Task Transfer Index (TTI) as a unified evaluation metric. TTI is defined as the additional construction workload required to migrate the system to the second task, given that the software system for the first task has already been built and is operational and verifiable. Unlike evaluation approaches focused on algorithm performance or single-development cost, TTI measures incremental workload directly, reflecting the engineering cost of reuse and migration.
Under a unified evaluation framework, TTI does not presuppose a system construction paradigm: it does not distinguish whether a system is organized directly from algorithm modules or from functional abstraction. For quantitative analysis, work content that must be newly added or significantly adjusted to meet the requirements of the second task during migration is divided into task-level and algorithm-level parts.
Task-level additional workload mainly originates from changes in task objectives, flows and functional requirements, and includes three types of work increment: task semantic analysis, definition of interface information flow between nodes, and exception handling logic.
Algorithm-level additional workload covers all direct modifications and coupling propagation work occurring during task migration due to algorithm adaptation. It includes four categories: newly added or replaced algorithm modules required by changes in algorithm capability or modeling assumptions; retraining of existing models; reconfiguration achievable only through parameter adjustment, threshold setting, cost weight or control gain tuning; and internal interface adaptation within algorithm programs.
To ensure the reproducibility and consistent comparability of TTI across different tasks and different methodological paradigms, the authors propose a systematic approach combining hierarchical modeling with discrete event counting to define the additional construction workload generated during task migration. TTI provides a basis for quantifying the reconstruction cost of embodied intelligence software systems. Using function point analysis and similar functional size measurement methods widely modeled in the software size measurement field as a benchmark, the paper defines the hierarchical mathematical expression of TTI and the symbolic meaning and standard of each counting item.
The additional construction workload migrating from the first task to the second is decomposed into task-level and algorithm-level parts, denoted as the task-level increment and the algorithm-level increment respectively. The task-level increment consists of three categories of newly added or significantly modified events at the functional level: the number of functional nodes corresponding to task flow change, the number of interface and invocation logic adjustments, and the number of newly added or modified functional constraints and exception handling logic reflecting adjustments to preconditions, exception branches or recovery strategies affecting the task execution path. The algorithm-level increment consists of four categories of algorithm adaptation events: the number of newly added or replaced algorithm modules, that is, introducing a completely new algorithm implementation with independent inputs and outputs; the number of model retraining events, describing retraining of existing models due to changes in environment, sensor configuration or data distribution; the number of parameter reconfiguration events, referring to adjustment of parameters, thresholds or cost weights to satisfy new task constraints without changing algorithm structure or model weights; and the number of interface adaptation items, characterizing data format adaptation work performed to enable algorithms to operate in coordination with the rest of the system.
| Layer | Sub-item | Traditional algorithm module composition | Function abstraction architecture (proposed) |
|---|---|---|---|
| Task level | Task flow change | 6 | 6 |
| Task level | Interface information flow definition | 10 | 0 |
| Task level | Exception handling logic | 1 | 1 |
| Task level | Subtotal | 17 | 7 |
| Algorithm level | New or replaced algorithm modules | 5 | 0 |
| Algorithm level | Model retraining | 1 | 1 |
| Algorithm level | Parameter reconfiguration without retraining | 5 | 5 |
| Algorithm level | Interface adaptation | 10 | 0 |
| Algorithm level | Subtotal | 21 | 6 |
At the task level, task flow change depends only on task semantics. Because the second task flow analysis involves one overall control flow module and five sub-task flow control modules, the construction of these six modules is additional work required under both design paradigms. Regarding interface information flow definition, the traditional algorithm module composition method requires redefinition of interface information flow and flow structure during task migration because the task flow depends directly on concrete algorithms. Once an algorithm module changes, the upstream and downstream task connections related to it must have their information flow relationships redefined. This part is not required under the functional abstraction architecture, because the information flow relationships of functional modules are already standardized and stabilized within the architecture. Exception handling logic is also strongly task-related, so it must be reconstructed under both paradigms.
At the algorithm level, the traditional method involves five new algorithm module workloads, including adding object detection and environment reconstruction modules and adjusting natural language processing, object localization and trajectory planning algorithms. In the proposed method, this part is completed by framework developers. Similarly, the interface adaptation work caused by algorithm replacement is also completed by framework developers. However, model retraining and parameter reconfiguration required by algorithm modules are directly related to task content, so the workload remains the same under both paradigms.
Overall, after introducing the proposed functional abstraction architecture, when migrating from the first task to the second, task-level additional workload decreased from 17 items to 7, and algorithm-level additional workload decreased from 21 items to 6. Under this architecture, new work mainly comes from unavoidable adaptation requirements brought by the task itself, such as exception handling and parameter adjustment. Migration cost is stably constrained within the scope of task semantic change and algorithm task-adaptation modifications, effectively avoiding the additional engineering burden caused by algorithm coupling in traditional methods.
18. Conclusions and Outlook
The study addresses the problems of insufficient software reusability and high task migration cost that are widespread in embodied intelligence systems across multi-task applications. It proposes a task-transferable system architecture based on functional abstraction. The architecture introduces a stable functional layer between task requirements and concrete algorithm implementations, uniformly converging task change into functional composition and parameter configuration problems, thereby preventing task adjustment from directly triggering algorithm interface reconstruction or system structure change.
On this basis, the authors constructed a complete functional system instance covering perception, cognition, planning and execution, and realized one-to-many mapping between functions and algorithms through unified data interfaces, providing an engineering instantiation of the proposed architecture paradigm.
In experimental validation, system instantiation and experimental testing were completed on two complex tasks: mobile manipulator grasping and humanoid robot cross-floor patrol. Results show that without introducing new functional definitions or modifying the overall architecture, adaptation to different tasks and platforms can be achieved solely through functional composition adjustment and algorithm configuration, with the system reaching engineering-usable levels in key metrics such as localization, trajectory tracking and target recognition. Further quantitative analysis of additional workload during system migration using the Task Transfer Index showed that, compared with the traditional approach of directly composing algorithm modules, the proposed architecture significantly reduces additional development cost, with new work concentrated in unavoidable task-strongly-related links such as task semantic analysis, exception handling logic and parameter adjustment.
The authors acknowledge that the method has limitations. The work does not improve the internal mechanisms of specific perception, planning and control algorithms but instead organizes and reuses existing algorithms as functional implementation units, so perception accuracy, planning efficiency, control stability and environmental adaptability in actual tasks remain constrained by the performance boundaries of underlying algorithms. In addition, the current function-to-algorithm mapping relationship still relies mainly on manual configuration, and an automated algorithm selection and parameter adaptation mechanism oriented to task constraints, platform characteristics, sensor configuration, computing resources and real-time requirements has not yet been formed. Under more complex tasks and more heterogeneous platform conditions, system construction still requires developers to exercise a degree of experiential judgment and engineering debugging in accordance with task characteristics.
Future work will further address multi-task, multi-platform and complex real-world scenarios and conduct research on automated function-algorithm configuration, with the aim of further improving the generality, robustness and engineering applicability of the proposed architecture. The work was supported by the National Natural Science Foundation of China under grants 62173220 and 62303294.