Researchers at Shanghai University have proposed a system architecture for embodied intelligence that inserts a stable functional layer between task requirements and algorithm implementations, converting task variation into a problem of functional composition and configuration. The framework was instantiated on two very different robotic platforms — a mobile manipulator performing autonomous grasping and a humanoid robot conducting cross-floor patrols — and was accompanied by a new quantitative metric, the Task Transfer Index, intended to measure how much new engineering work a task migration actually costs. The reported results indicate that the function-abstraction approach substantially reduces the incremental workload of moving an embodied intelligence system from one task and platform combination to another, compared with conventional practice in which application software is assembled directly from algorithm modules.
The work addresses a persistent engineering bottleneck in embodied intelligence: while learning-based methods continue to improve at individual skills, the software that surrounds those skills remains largely custom-built for a specific task, a specific robot and a specific application scenario. Teams report that reuse across tasks is limited, switching costs are high, system extension is difficult and maintenance complexity grows quickly. The proposed architecture is positioned not as a new perception, planning or control algorithm, but as a structural answer to how existing algorithms should be organized so that task change does not propagate down into the algorithm layer of an embodied intelligence system.
- The transfer problem that keeps embodied intelligence projects expensive
In real deployments, a concrete application of embodied intelligence is jointly determined by three elements: the application scenario, the task flow and the robot platform. Across different applications these three elements commonly differ substantially. Conventional system construction binds task logic, algorithm implementation and software architecture tightly together, so that a change in any one of them typically triggers large-scale modification, making continuous iteration and cross-task reuse difficult.
The authors frame the issue in terms of structural growth rather than algorithmic capability. When developers build a task-level information flow directly from algorithm modules, the algorithm pool keeps expanding as task types accumulate. Designing, debugging and maintaining the resulting information flows gradually becomes a high-complexity manual process, because developers must repeatedly handle interface adaptation both between algorithms and between algorithms and tasks. Development cost then grows approximately linearly with the number of tasks and exhibits combinatorial complexity with respect to the size of the algorithm pool, so the system becomes hard to evolve and reuse over the long term. In this reading, the missing ingredient in embodied intelligence is not another model but a mechanism that systematically confines task variation to composition and configuration at the architectural level, isolating task requirements, platform differences and algorithm implementations from one another.
The paper also draws a careful distinction about what transferability means here. It is not the ability of a single algorithm or model to generalize its execution across many scenes and tasks. Instead it refers to the systemic capacity of the software architecture of an embodied intelligence system to support rapid construction, functional configuration, code reuse and continuous evolution when faced with new combinations of task requirements, robot platforms and application scenarios. That distinction matters because it shifts the evaluation criterion away from benchmark success rates and toward engineering effort.
- Two dominant paradigms and where each one loses transferability
Current architectures for embodied intelligence broadly fall into two families. The first is the end-to-end architecture built around holistic policy learning, in which perception, decision-making and manipulation are integrated into a single learning framework so that the system generates low-level control actions directly from raw multimodal sensor input. This paradigm has been reinforced by large language models and vision-language models, particularly for high-level semantic understanding, task description and instruction generation.
Representative systems include OpenVLA, which builds a unified vision-language-action model framework enabling a robot to output joint-level motion commands directly from language and image inputs through a single policy; PaLM-E, which tokenizes visual information and robot state into a large language model to generate executable action strategies within one model; and RT-2, which transfers large-scale web knowledge into concrete robotic manipulation tasks such as object placement and tabletop organization. Related efforts attempt to strengthen hierarchical expression inside end-to-end frameworks: RT-H introduces language-based hierarchical action representations on top of RT-2 to compensate for the shortcomings of purely end-to-end policies in complex task planning and execution, while ChatGPT for Robotics uses code as an intermediary between semantic instructions and low-level control to improve interpretability and controllability. Even with these intermediate representations, training and inference still rely on a unified model performing the whole mapping from perception to action, so such approaches remain extensions of the end-to-end paradigm.
The paper acknowledges that end-to-end learning has made progress in controlled settings and in certain policy-learning tasks, and that large-model-enabled decision mechanisms have raised success rates for individual tasks. It argues, however, that such methods cannot be directly extended into a general implementation path for embodied intelligence. From an architectural standpoint, end-to-end approaches implicitly couple task structure, environmental assumptions, robot morphology and control interfaces into a single policy representation, so generalization tends to be limited to scenarios where task distributions and platform conditions are relatively homogeneous. That coupling is described as the reason cross-task and cross-platform reuse is difficult, with transfer depending on parameter fine-tuning or retraining rather than system-level capability reorganization. Theoretically, long-horizon physical interaction safety and continual adaptation to new tasks are not fully accounted for, and explicit robot and environment parameter constraints are hard to embed. In engineering practice, robustness and reliability reportedly degrade markedly under real conditions such as illumination change and dynamic environments, while high data acquisition costs for physical interaction, heavy compute and real-time demands, and difficulty in explaining and safety-verifying system behavior remain bottlenecks for deploying embodied intelligence at scale.
The second family is the hierarchical, functionally modular architecture, which has deeper roots in robotics and is often treated as the practical route to complex robot behavior. Since the classical three-layer perception-planning-execution architecture, behavior-level control, and the separation of task planning from motion planning, modular and hierarchical design has been widely used to build robots for complex scenarios. Explicit decoupling of perception, cognition, decision-making and execution is generally regarded as a precondition for maintainability, interpretability and debuggability. In embodied intelligence research this idea has evolved into layered system designs, such as frameworks that model multimodal perception, compliant manipulation and process-level decision-making separately for complex manufacturing, or safety-oriented layered frameworks that systematically define and safeguard perception reliability, decision safety and execution controllability. Explicit hierarchy provides the structural basis for safety verification, fault isolation and system auditing that real deployments require.
With the rise of foundation models, hierarchical architectures have also been proposed as a realistic integration path. Large language models or vision-language models can serve as high-level cognition and reasoning modules for task understanding, symbolic reasoning or high-level planning, while low-level perception and control stay in dedicated modules that satisfy real-time, stability and safety requirements. This “model as module” arrangement relieves some of the compute, data and reliability pressure of end-to-end designs. Yet existing modular systems still depend heavily on manual modeling of task flows and functional boundaries at design time, and are usually constrained by the hardware interfaces and middleware of a particular platform, so reuse across tasks, platforms and scenarios still carries high engineering cost. The open question, the authors argue, is how to preserve clarity and controllability while introducing a more general representational mechanism and automated configuration capability.
- A three-layer answer: task, function and algorithm
The proposed framework starts from task requirements and maps them level by level down to functional structure and technical implementation. Rather than organizing task requirements as communication between concrete algorithm nodes, as is common in robot operating system practice, it builds an explicit three-level model consisting of a task layer, a function layer and an algorithm layer, and introduces function abstraction as the middle structural layer. The authors note that while a robot operating system provides general communication and runtime support for building embodied intelligence systems, it cannot by itself decompose task requirements or guarantee that different algorithms are interchangeable at the level of functional semantics. In contrast, the proposed design shifts system construction from the combination of concrete algorithm modules to the combination and configuration of functional capabilities.
Under this arrangement, a change in task affects only functional composition and configuration and no longer triggers reconstruction of the underlying algorithm architecture, giving embodied intelligence a unified platform for rapid construction, flexible migration and continuous extension across tasks, platforms and application scenarios. The authors state explicitly that the framework does not aim to improve the internal mechanisms or performance of specific perception, planning or manipulation algorithms, and does not introduce new learning models; existing algorithms and models are treated as reusable functional components, and the focus is on how they are organized at system level under task-driven conditions.
From a conceptual standpoint the contrast with conventional practice is described as follows. Traditional development requires engineers to build task-level information flows directly on top of algorithm modules; as task types and algorithm modules multiply, the algorithm pool expands and the design, debugging and maintenance of those flows becomes a manual process of growing complexity. The proposed alternative treats function as the relatively stable abstract unit in an embodied intelligence system and places the function layer between the task layer and the algorithm layer, so that uncertainty on the task side is isolated from continuous evolution on the algorithm side. Application developers concentrate on the organizational relationship between task requirements and functional capabilities, while framework developers maintain the mapping from functions to algorithms and the supporting tooling beneath it.
During system construction and evolution, developers configure the information flow of the embodied agent through functional nodes, and the correspondence between the function layer and the algorithm layer remains transparent and stable to them. When task requirements change, the system adapts by adjusting the combination of functional nodes without modifying concrete algorithm implementations; when algorithm capability is updated or extended, new algorithms are connected through existing functional interfaces without architecture-level adjustment for a particular task. Task variation is thus confined to functional composition and algorithm variation to functional implementation, providing a stable basis for reuse and extension under multi-task, multi-platform conditions.
- Inside the functional layer: three subsystems and twenty-five core capabilities
The function layer sits at the core of the architecture and aims at a stable abstraction of the basic capabilities an embodied intelligence system needs during task execution. It is concerned with what types of capability the system can deliver rather than how those capabilities are implemented. It is organized around the typical information flow and capability loop of an embodied agent into three subsystems. The edge perception system acquires local environment and body state in real time with an emphasis on high frequency and low latency. The brain system handles scene understanding, task decision-making and plan generation with an emphasis on semantic abstraction and task reasoning. The cerebellum system handles motion execution, stability control and human-robot interaction with an emphasis on continuous control and stability assurance.
Depending on the task, these subsystems are concretized into specific capabilities such as environment perception and modeling, localization and navigation, target interaction, motion planning and behavior execution. Each function corresponds to a capability unit that a task can invoke directly. Functional definitions emphasize four properties: task-oriented semantics, so that descriptions serve task goals and execution logic without introducing algorithm or implementation detail; composability, so that different functional modules can be combined flexibly into diverse task flows; cross-platform stability, so that definitions remain consistent across robot morphologies, sensor configurations and application scenarios; and interface clarity, so that each function provides its service through a well-defined input-output relationship and acts as a standard unit of invocation inside the system.
Based on the core inputs and outputs generated during task execution, and following the principles of input-output closure, replaceability and verifiability, the authors abstract twenty-five core functional modules covering perception, cognition, planning and execution. Input-output closure means that module inputs and outputs use standardized interfaces; replaceability means that different algorithms implementing the same function can be substituted seamlessly; verifiability means that module input data can be simulated and output results can be measured.
| Subsystem | Module count | Abstracted functional modules |
|---|---|---|
| Edge perception | 10 | Simultaneous localization and mapping; scene recognition; environment reconstruction; object detection; object localization; object tracking; terrain classification; semantic map construction; tactile recognition; hand pose recognition |
| Brain | 6 | Behavior recognition and intention inference; natural language understanding; task planning; grasp strategy generation; human pose perception; human-robot task allocation |
| Cerebellum | 9 | Manipulation control; gait coordination; balance control; path planning; trajectory planning; collision detection; trajectory tracking; teleoperation enhancement; pose mapping |
The edge perception system is the functional layer that interacts most directly with sensor data, covering simultaneous localization and mapping, scene recognition, environment reconstruction, object detection, object localization, object tracking, terrain classification, semantic map construction, tactile recognition and hand pose recognition. These correspond to the basic perception needs of an embodied agent while moving, manipulating and interacting with people, and their outputs can be called by upper layers in a unified spatial coordinate frame or semantic annotation form.
Within that group, simultaneous localization and mapping provides the unified spatial reference frame and is the most foundational capability for localization and consistency maintenance, with mature implementation paths that differ in sensor configuration, motion state and real-time demand. Scene recognition performs place re-identification and historical environment matching to support global consistency and drift correction, typically relying on global descriptors and fast retrieval. Environment reconstruction recovers three-dimensional scene structure from multi-view or multimodal observations to provide a geometric basis for target perception and spatial understanding. Object detection, localization and tracking serve concrete interaction targets, outputting category information, spatial position and motion state as preconditions for grasping, obstacle avoidance and collaboration. Terrain classification evaluates traversability for locomotion planning and risk avoidance. Semantic map construction associates semantic information with geometric structure to form an environment representation usable directly by planning and interaction modules, while remaining at the perception level and not performing task reasoning. Tactile recognition is the only module in this subsystem that depends directly on contact information, supplementing vision and geometry in near-field manipulation and fine interaction. Hand pose recognition estimates human hand keypoints and posture for teleoperation, teaching and interactive control.
The brain system converts environment, target and state information from the edge perception system into structured task decisions and behavior strategies, focusing on task understanding, behavior organization and collaborative decision-making rather than low-level motion control. Its six modules cover the main cognitive needs of autonomous operation and human-robot collaboration: behavior recognition and intention inference and human pose perception interpret human behavior and operational intent; natural language understanding parses high-level semantic input from people; task planning organizes task goals into a subtask structure with temporal relations; grasp strategy generation produces a grasp plan at the task level; and human-robot task allocation coordinates the division of work between robots and people in multi-agent scenarios. Outputs from this subsystem are primarily at task and strategy level and do not involve continuous control quantities.
The cerebellum system converts task decisions and motion intentions from the brain into stably executable physical motion, facing dynamic constraints, real-time feedback and safe execution directly. Its nine modules cover the main closed-loop capabilities from motion scheme generation to online execution. Path planning and trajectory planning generate feasible motion schemes under task goals and environmental constraints; trajectory tracking and manipulation control land planning results stably on the execution layer; gait coordination and balance control maintain stable walking and posture for legged and humanoid platforms; collision detection acts as a safety constraint module to avoid dangerous contact and failed motion; teleoperation enhancement improves stability and fault tolerance for human-robot collaboration and remote operation; and pose mapping establishes consistent motion expression across different body structures or between a human and a robot, supporting teaching and cross-platform reuse. Outputs from this subsystem are mainly continuous control commands and execution-level constraints.
- The algorithm layer as a substitutable implementation space
The algorithm layer implements the system capabilities defined in the function layer, and its focus is the selection, combination and continuous optimization of specific technical approaches. Unlike the function layer, which carries stable capability abstraction, the algorithm layer is explicitly designed as a layer that can evolve and be replaced. Algorithms are treated as technical means for realizing functions rather than as the organizing core of the system, so that system structure does not need frequent adjustment when algorithms are updated or technical routes change.
Four design principles govern this layer. The first is a one-to-many mapping relationship: functions and algorithms do not correspond one to one, and the same function can be implemented by multiple algorithms. The second is interface consistency: algorithm modules implementing the same function keep the same input and output data forms, so algorithms can be substituted without additional adaptation cost. The third is replaceability and extensibility: algorithm implementations can be dynamically replaced or extended according to task requirements and system resources without affecting functional definitions. The fourth is evolution friendliness: the introduction of a new algorithm serves only as an additional implementation option for an existing function and does not trigger changes to functional structure or task organization.
Because of this explicit decoupling, interchangeable algorithm modules are guaranteed in advance by the architecture with respect to program interfaces, data semantics and invocation patterns. In deployment, the overall capability of an existing task software system can therefore be upgraded rapidly by directly replacing or upgrading algorithm modules, without modifying the original task flow, functional organization or system integration structure. The algorithm layer thus becomes a relatively independent and continuously evolving technical space that can absorb algorithmic innovation while functional abstraction and task logic remain stable.
| Functional module group | Representative algorithm implementations available in the mapping |
|---|---|
| Localization and mapping | LIO-SAM, VINS, ORB-SLAM, FAST-LIO2 |
| Scene recognition | Scan Context, DBoW2, iBoW-LCD |
| Environment reconstruction | Structure from motion, 3D Gaussian splatting, multi-view stereo, iterative closest point |
| Object detection | YOLO, Mask-RCNN, DETR |
| Object localization | Frustum PointNet, CenterNet3D, SMM-POD, YOLO-based pipelines |
| Object tracking | JRMOT, GlobalTrack, ODTrack |
| Terrain classification | Random forest, DeepTerramechanics, lightweight boundary-assisted network |
| Semantic mapping | Panoptic-Mapper, SA-LOAM |
| Tactile recognition | Transferable tactile transformers, deep tactile CNN, GelSight-based CNN |
| Hand pose recognition | MediaPipe, DeepPose |
| Behavior and intention inference | Social LSTM, spatial temporal graph convolutional networks |
| Natural language understanding | TinyBERT, Llama3-8B, Qwen3, AV2vec |
| Task planning | Behavior trees, PDDL, OpenVLA, RT-H |
| Grasp strategy generation | Dex-Net, diffusion policy, grasp pose detection |
| Human pose perception | OpenPose, HRNet, mm-Pose |
| Human-robot task allocation | Market-based multi-robot coordination, Hungarian algorithm |
| Manipulation control | Impedance control, admittance control, hybrid position/force control, PID |
| Gait coordination | DeepMimic, zero moment point, rapid motor adaptation |
| Balance control | Linear quadratic regulator, model predictive control, hybrid zero dynamics |
| Path planning | A*, RRT*-Connect, bi-directional fast marching trees, batch informed trees |
| Trajectory planning | Polynomial parameterization, B-spline, time-optimal path parameterization |
| Collision detection | Sweep and prune, bounding volume hierarchies, separating axis theorem, RobotSDF |
| Trajectory tracking | Adaptive robust control, model predictive control, guidance vector field, dynamic window approach |
| Teleoperation enhancement | TRAC-IK, data-driven teleoperation models, virtual fixtures |
| Pose mapping | Dynamic movement primitives, adversarial motion priors, motion retargeting |
To keep module interaction independent of any algorithm’s internal states or private data representations, the function layer constrains the input and output forms of modules through an explicit data interface specification. Any algorithm satisfying the data interface requirements of the corresponding functional module can be connected as an implementation of that function. Algorithm replacement then occurs only inside a module and requires no synchronized modification of the task flow or of upstream and downstream modules. By confining algorithmic differences within interface boundaries, dependencies between functional modules are fixed at a stable data interaction layer, making algorithm substitution and evolution a structural operation rather than an additional engineering retrofit.
| Interface type | Abbreviation | Description |
|---|---|---|
| Sensor data | SD | Raw multimodal sensor data such as vision, depth, inertial measurement, tactile and lidar streams |
| Robot kinematic state | RKS | Joint angles, overall motion posture, velocities and related kinematic information |
| Scene representation | SR | Unified description of environment geometry and topology, including colored point clouds, voxel grids, Gaussian representations or occupancy maps |
| Semantic entity set | SE | Structured collection of object semantic categories and associated attributes |
| Target descriptor | TD | Abstract feature set describing one or more targets, including position, kinematic parameters and category |
| Scene graph | SG | Graph-structured environment representation containing spatial relations, semantic relations and entity connections |
| Body state | BS | Human skeleton, body posture and related information |
| Task intent | TI | Description of overall task and element-level intent, including task goals and human intention |
| Task graph | TG | Graph representation composed of task elements and their ordering relations |
| Constraint set | CS | Collision constraints, posture constraints, stability and boundary conditions |
| Action primitive | AP | Atomic action or skill units of a robot or human |
| Trajectory | Tra | Robot motion path sequence in the spatiotemporal domain |
| Control command | CC | Low-level control quantities driving robot motion, such as velocity or torque |
For application developers, system construction is thereby reduced to two steps: designing the overall task execution flow and decomposing it into core functional modules already defined in the function layer, with their invocation relations and ordering; and then choosing an implementation for each module in the algorithm layer according to environment characteristics, real-time requirements and computing resources, using the function-algorithm correspondence and the functional support diagram as explicit aids. The authors also note that the instantiated design clarifies the organizational relationship between functional modules and algorithm implementations and does not replace a complete software runtime management mechanism; compute resource scheduling, execution priority of functional modules and anomaly monitoring during task execution still need to be designed by developers for the specific application and platform.
- Validation on two platforms built on the same architecture
To test the architecture’s applicability and engineering value, the authors selected two representative tasks on two heterogeneous robot platforms. Rather than choosing demonstrative tasks limited to a single capability or a simplified flow, they deliberately picked tasks with high complexity in both research and engineering practice, so that functional coverage, flow complexity and integration difficulty are closer to real applications.
| Comparison dimension | Mobile manipulator grasping task | Humanoid cross-floor patrol task |
|---|---|---|
| Manipulation platform | Single arm | Dual arm |
| Degrees of freedom | 3 chassis, 6 arm, 1 gripper | 29 whole body, 6 by 2 dexterous hands |
| Locomotion | Tracked chassis | Bipedal walking |
| Sensing configuration | Lidar, stereo camera, inertial measurement unit, microphone | Lidar, stereo camera, inertial measurement unit, microphone |
| Task flow | Short, multi-stage, tightly coupled | Long, multi-stage decision-making |
| Workspace | Indoor unstructured environment | Cross-floor structured building space |
| Interaction objects | Fixed set of targets | Diverse, including human facilities |
The two tasks differ across hardware platform, task complexity and application scenario. In hardware terms, the composite mobile manipulation platform has a relatively clear structure and fewer motion constraints and is suitable for verifying how functional modules combine and how efficiently a system can be built when movement and manipulation are strongly coupled. The humanoid platform has highly redundant whole-body degrees of freedom and strict dynamic balance constraints and represents one of the more complex morphologies in current embodied intelligence engineering. Building system instances on both platforms tests whether the architecture adapts to and can be reused across differences in robot morphology.
In terms of complexity, the grasping task is a short-flow, highly coupled task whose core challenge is tight coordination of navigation, perception and grasping in an unstructured environment, requiring parallel invocation and rapid switching among multiple functions within limited time. The patrol task is a long-flow, multi-stage decision task that includes cross-space movement, state switching and repeated decision cycles, placing higher demands on task organization, long-term stability and exception handling. The two also correspond to different deployment conditions: the grasping task occurs in an indoor unstructured environment with a limited spatial range but complex object distribution and occlusion, requiring high local perception accuracy and immediate response, whereas the patrol task faces a building space spanning floors with a larger environmental scale, a longer task cycle and interaction with typical human facilities such as elevators and doors.
- Case one: autonomous grasping with a mobile manipulator
For the grasping task, the overall goal is for the robot to execute a spoken instruction within a single room, such as fetching a bottle of water. Analysis of the work process yields ten capability modules: natural language understanding, task planning, simultaneous localization and mapping, path planning, object localization, trajectory planning, trajectory tracking, collision detection, grasp strategy generation and manipulation control. The authors classify this as a typical mobile-manipulation strongly coupled task, and note that the flow control module is not one of the architecture’s functional modules and must be built separately for the task.
In the constructed functional flow, the mobile manipulator first parses the target object as a water bottle through natural language understanding and decomposes the instruction through task planning into three stages: determining the spatial position of the bottle, navigating to the grasping point, and executing the grasping task. In the first stage, simultaneous localization and mapping builds a three-dimensional model of the environment, object localization determines the position of the bottle in three-dimensional space, and path planning generates a collision-free safe path. In the second stage, the localization and mapping module provides precise self-localization and a three-dimensional environment model during movement, while trajectory planning performs local obstacle-avoidance planning and trajectory tracking performs path-following navigation. In the third stage, object localization re-localizes the target in real time, grasp strategy generation provides the stopping and manipulation pose, collision detection supplies collision information for strategy generation, trajectory planning generates the arm motion trajectory, and manipulation control completes the operation according to the trajectory. These modules are organized into a clear information flow and control flow in task execution order, forming a complete embodied intelligence architecture.
At the algorithm configuration stage, the system was set up for the sensing incompleteness, dynamic occlusion and mobile-manipulation coupling characteristic of indoor unstructured environments. Natural language understanding used an AV2vec-based method supporting multimodal parsing of spoken instructions. Localization and mapping used the tightly coupled lidar-inertial LIO-SAM framework to suit indoor line and plane geometry. Path planning used A* for global navigation path search, appropriate for short-range indoor spaces. Object localization used Frustum-PointNet for robust instance identification of diverse objects. Trajectory planning used B-splines to generate smooth locally optimized obstacle-avoidance trajectories, and collision detection used RobotSDF for continuous collision checking to safeguard manipulation in cluttered space. Grasp strategy generation used the FIRM algorithm to produce grasp poses jointly optimized over geometry and mechanics, and manipulation control used a PID controller to track the grasping trajectory smoothly.
The authors emphasize that this configuration is only one way of combining algorithm modules through the function-to-algorithm mapping; with the task functions unchanged, algorithms in the architecture can be replaced or upgraded to decouple algorithm research and development from software development.
In execution, the system was tested on five target objects distributed at different positions in the environment, and it completed the grasping task stably under the same functional flow in all cases. At the module level, FIRM selected stably, among multiple candidate poses, the solution with the highest reachability and manipulation margin, with consistent manipulability distribution across different spatial positions. The RobotSDF-based collision detection module maintained continuity and accuracy in distance-field estimation; compared with conventional geometry-based detection, the implicit morphology representation showed no obvious jumps during collision checking and achieved sub-millisecond collision detection efficiency, enabling real-time trajectory generation. Trajectory planning produced smooth, collision-free grasping trajectories meeting the accuracy requirements of manipulation in multi-obstacle space. The authors conclude that the first case demonstrates that the function-abstraction architecture can support the construction of a complete task flow in a complex embodied intelligence task where multiple capabilities must work together, achieving autonomous movement and manipulation in a complex spatial environment with stable execution.
- Case two: cross-floor patrol with a humanoid robot
The patrol task is a cross-temporal, serialized composite task with conditional branches, involving multiple cyclic phases and requiring the robot to call perception, planning and execution capabilities continuously across different spatial regions and environmental states. The functional organization built for it comprises ten core modules: natural language understanding, task planning, simultaneous localization and mapping, path planning, trajectory tracking, object detection, object localization, manipulation control, environment reconstruction and trajectory planning. Although the task content and flow differ markedly from the first case, only one functional module changed between the two task builds.
In the constructed flow, the humanoid robot plans the received spoken instruction into five stages. The first stage is autonomous navigation on the ground floor from the current position to the elevator entrance, where localization and mapping provides self-localization, environment reconstruction builds a precise map, path planning generates a route to the target based on the relation between the stored map and the current map, and trajectory tracking performs path-following control. The second stage is entering the elevator, where object detection recognizes the elevator through a two-dimensional code and identifies buttons and door state, object localization precisely locates the elevator button, localization and mapping provides real-time calibrated self-localization, path planning produces both the robot’s route into the elevator and the arm’s path to the button, trajectory tracking performs path following, trajectory planning smooths the arm trajectory and manipulation control drives the arm to operate the button. The third stage is exiting the elevator, where localization and mapping supplies self-localization, object detection reports elevator door state, path planning provides the exit route and trajectory tracking follows it. The fourth stage is autonomous navigation on the second floor, functionally identical to ground-floor navigation but targeting a room door specified by the instruction. The fifth stage is opening the door, where object detection finds the door handle, object localization gives its precise three-dimensional position, and path planning, trajectory planning and manipulation control operate as in the second stage but act on the handle to rotate it and push the door open. Across these stages, the functional modules of the humanoid robot are strongly reusable even though the content of each stage differs.
Because the patrol task runs for a long time across many scene changes, algorithm selection emphasized maturity and long-term reliability of operation. Natural language understanding used the Llama3-8B model, balancing semantic understanding capability with deployability on edge computing, and can assist in generating a preliminary task structure while parsing patrol instructions. Object detection and object localization both used YOLO to support real-time recognition of key targets such as buttons and doors. Since the operating environment contains few obstacles, the arm path planning used RRT*-Connect to achieve efficient feasible path search in the high-dimensional configuration space of the manipulator. Environment reconstruction, the newly added functional module, used the iterative closest point algorithm to achieve precise global relocalization relative to the existing map. The remaining modules kept the same algorithm configuration as in the first task. In summary, although only one functional module changed, the number of algorithm modules that differed between the second task and the first was four.
In testing, the system maintained centimeter-level trajectory tracking accuracy during continuous patrol and cross-floor movement, completing reliable stops at designated positions without obvious accumulated deviation or loss of stability. In environment reconstruction, the real-time three-dimensional map retained good spatial consistency with the real environment, with the vast majority of point cloud registration errors below 0.04 meters. During long-running operation the system showed no map degradation or structural drift, indicating stable environment reconstruction and localization during movement. In target perception, the real-time recognition and localization module continuously and accurately identified multiple task-relevant object types, including elevator buttons, two-dimensional codes, door handles and elevator door states, with key target three-dimensional localization error below one centimeter. The reported statistics show detection and localization accuracy consistently above 98 percent, providing reliable input for environmental interaction and dependable task execution. The authors conclude that the system built under the framework operates stably in a long-sequence, multi-stage complex task, with functional modules effectively coordinated under a unified architecture and key perception and execution indicators reaching an engineering-usable level.
- Quantifying migration cost with the Task Transfer Index
To assess how much the architecture improves task transferability in embodied intelligence software construction, the authors introduce the Task Transfer Index as a unified evaluation metric. It is defined as the amount of additional construction work required to migrate the system to the second task, given that the software system for the first task has already been built, is runnable and has been verified. Unlike evaluations focused on algorithm performance or single-development cost, the index measures incremental workload directly, reflecting the engineering cost of reuse and migration. It does not presuppose a construction paradigm, meaning it does not distinguish between systems assembled directly from algorithm modules and systems organized around functional abstraction. For quantification, the work that must be added or significantly adjusted to meet the second task’s requirements is divided into a task-level part and an algorithm-level part.
Task-level added workload arises mainly from changes in task goals, flows and functional requirements, and includes three types of work increment: task semantic analysis, definition of interface information flows between nodes, and exception handling logic. Algorithm-level added workload covers all direct modifications and coupling propagation caused by algorithm adaptation during migration, including replacement or addition of algorithm modules where algorithm capability or modeling assumptions change; retraining of existing models; reconfiguration through parameters, thresholds, cost weights or control gains; and adaptation of interfaces inside algorithm programs. The metric is defined through layered modeling combined with discrete event counting to support reproducibility and consistent comparability across tasks and paradigms, with the authors noting functional size measurement methods such as function point analysis as the reference basis for software scale measurement.
| Layer | Sub-item | Conventional algorithm-module composition | Function-abstraction architecture |
|---|---|---|---|
| Task level | Task flow change | 6 | 6 |
| Task level | Interface information flow definition | 10 | 0 |
| Task level | Exception handling logic | 1 | 1 |
| Task level | Subtotal | 17 | 7 |
| Algorithm level | New or replaced algorithm modules | 5 | 0 |
| Algorithm level | Model retraining | 1 | 1 |
| Algorithm level | Parameter reconfiguration without retraining | 5 | 5 |
| Algorithm level | Interface adaptation | 10 | 0 |
| Algorithm level | Subtotal | 21 | 6 |
At the task level, task flow change depends only on task semantics. Because the analysis of the second task involves one supervisory flow module and five subtask flow control modules, the construction of these six modules is newly required work under both design paradigms. On interface information flow definition, the conventional approach depends directly on concrete algorithms, so task migration requires redefining interface information flows and flow structure; once an algorithm module changes, the upstream and downstream task connections must have their information flow relations redefined. Under the function-abstraction architecture, that work is unnecessary because the information flow relations between functional modules are already specified and fixed. Exception handling logic is strongly task-related, so it must be rebuilt under both paradigms.
At the algorithm level, the conventional method involves five new algorithm module workloads: adding object detection and environment reconstruction modules and adjusting the algorithms for natural language processing, object localization and trajectory planning. In the function-abstraction architecture, this work is carried out by framework developers rather than by the application developer performing the migration. Similarly, the interface adaptation work caused by algorithm replacement is handled by framework developers. The retraining and parameter reconfiguration demanded by the algorithm modules are directly related to task content and therefore remain the same under both paradigms.
Overall, after introducing the function-abstraction architecture, task-level added workload for migration between the two cases falls from 17 items to 7, and algorithm-level added workload falls from 21 items to 6. New work under the architecture comes mainly from exception handling and parameter adjustment unavoidably induced by the task itself, so migration cost is stably confined to task semantic change and task-adaptive algorithm modification, avoiding the extra engineering burden caused by algorithm coupling in conventional practice.
- What the architecture does and does not claim
The central claim is that function abstraction improves the transferability and engineering reusability of embodied intelligence systems at the architectural level. The task layer expresses what must be achieved, the function layer expresses what capability is needed, and the algorithm layer expresses how that capability is realized, with an explicit mapping and unified data interfaces connecting them. Because task changes are absorbed by functional composition and parameter configuration while algorithm changes are absorbed by the function-to-algorithm mapping, the software structure of an embodied intelligence system does not have to be rebuilt each time the application changes.
The authors are equally explicit about the limits of the work. The study does not improve the internal mechanisms of specific perception, planning and control algorithms; existing algorithms are organized and reused as functional implementation units, so the perception accuracy, planning efficiency, control stability and environmental adaptability of the system in real tasks remain bounded by the performance limits of the underlying algorithms. In addition, the current function-to-algorithm mapping still relies mainly on manual configuration, and there is not yet an automated mechanism for algorithm selection and parameter adaptation driven by task constraints, platform characteristics, sensor configuration, computing resources and real-time requirements. For more complex tasks and more heterogeneous platforms, system construction therefore still requires a degree of expert judgment and engineering tuning by developers.
Future work is directed at automated functional and algorithmic configuration for multi-task, multi-platform and complex real-world scenarios, with the aim of further improving the generality, robustness and engineering applicability of the architecture. The reported experiments are presented as evidence that a stable functional layer can turn task variation into a configuration problem for embodied intelligence, and that the resulting reduction in migration workload can be measured rather than merely asserted.
The work was supported by the National Natural Science Foundation of China under grants 62173220 and 62303294. The authors are affiliated with the Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, the Bionic Intelligent Robot Dreamworks for Special-purpose at Shanghai University, the School of Mechatronic Engineering and Automation at Shanghai University, and the School of Future Technology at Shanghai University. The corresponding author is Yusheng Yang of Shanghai University.