The optimization of scheduling within flexible job shops represents a critical research frontier in digital manufacturing science. A persistent and formidable challenge is the inherent dynamism of the production floor, where unpredictable disturbances—such as sudden machine failures, rush order insertions, or material shortages—routinely disrupt carefully laid plans. These events cascade into operational inefficiencies, including suboptimal resource allocation, missed delivery deadlines, and escalating production costs. In recent years, distributed, multi-agent systems have emerged as a promising architectural paradigm to imbue manufacturing systems with the resilience and responsiveness needed to navigate this uncertainty. By decentralizing decision-making, these systems aim to localize the impact of disturbances and accelerate recovery. However, conventional virtual agents, often designed purely from an informational or functional perspective, possess a fundamental limitation: they are disembodied. They lack a direct, physical connection to the workshop environment, forcing them to rely on indirect data streams and frequent, costly communication to understand and react to the physical world. This paper argues that the next evolutionary step lies in embodied AI robot scheduling agents. By integrating the dynamic behavioral states of physical entities—the machines, autonomous vehicles, and manipulators themselves—into the agent’s cognition, we can create systems that perceive in real-time, act autonomously, and adapt intelligently based on physical feedback. This work proposes a metamodel-driven methodology for constructing such embodied AI robot agents within flexible job shops. Through instantiation of this unified model, we architect a distributed multi-agent scheduling system where each agent, as an embodied AI robot, can adjust its local strategy before engaging in system-wide coordination. Coupled with a novel Q-bargaining game negotiation mechanism, this approach significantly enhances schedule stability, reduces communication overhead, and improves global optimization performance when facing disruptions, paving the way for more robust and intelligent manufacturing operations.
The core distinction between an embodied AI robot and a traditional software agent in a scheduling context lies in their mode of interaction with the environment and their decision-making autonomy in the face of disturbances. An embodied AI robot is an intelligent system integrated with a physical carrier, enabling direct, real-time interaction with the physical workshop. It perceives its own behavioral state (e.g., spindle speed, AGV location, gripper status) and the immediate environment through onboard sensors. When a disturbance occurs, it can proactively make and execute decisions—like resequencing its local task queue or altering its path—based on this direct physical feedback. In contrast, a non-embodied, virtual agent exists solely in the information space. It learns of state changes through database updates or messages from other systems, cannot directly drive physical actions, and typically must trigger a re-negotiation protocol across the agent network to find a new global solution, a process that is often slower and more communication-intensive. This fundamental difference is summarized in the table below:
| Aspect | Embodied AI Robot Scheduling Agent | Non-embodied (Virtual) Agent |
|---|---|---|
| Definition & Structure | An intelligent agent integrated with a physical entity (machine, AGV, robot) capable of direct interaction with the physical world. | A software-based intelligent system with no physical embodiment or direct physical interaction capability. |
| State Acquisition | Direct, real-time perception via integrated sensors; feedback through physical action execution. | Indirect perception via information system interfaces (APIs, databases); passive reception of status data. |
| Scheduling Strategy Generation | Proactive, autonomous adjustment of local behavior (task sequence, path) upon sensing disturbances. Exhibits strong system stability. | Typically requires re-negotiation at a global or local level upon disturbance, based on updated state data. Lacks individual-level strategy adjustment capability. |
The manufacturing floor is a complex ecosystem of diverse resources—CNC machining centers, automated guided vehicles (AGVs), robotic arms, storage systems—each with vastly different physical capabilities and action spaces. Simply equipping each with intelligence risks creating a heterogeneous swarm of agents that cannot understand each other, hindering coordination. To ensure effective collaboration, a unified modeling language is essential. This is where the concept of a metamodel becomes critical. A metamodel is a model that defines the structure and semantics of other models. In our context, it provides a formal, standardized blueprint for constructing any embodied AI robot in the job shop, ensuring consistency in their structure, attributes, and relationships regardless of their specific function. Our proposed Flexible Job Shop Embodied Scheduling Agent (FJSESA) metamodel is built upon a layered abstraction framework (M0 to M3). The instance layer (M0) consists of real-world atomic entities like a specific AGV or milling machine. The model layer (M1) abstracts these into unified models for logistics, production, and storage-type atomic agents. The metamodel layer (M2) defines the fundamental building blocks and rules for creating these models, and the meta-metamodel layer (M3) provides the foundational language for defining metamodels themselves. The core innovation lies in the structure of the embodied AI robot agent itself, which extends traditional agent layers (Interaction, Decision, Adaptation) by incorporating a tightly coupled Physical Entity Layer and a Behavioral Space. This creates a “body” for the agent. The key modules are:
- Communication & Interaction Module: The agent’s information processor, handling communication with other agents and fusion of sensor data.
- Analysis & Decision Module: The cognitive center, performing reasoning, scheduling action planning, and generating control commands.
- Unit Behavior Module: The execution unit, responsible for carrying out specific actions (e.g., “execute rough milling program a1”, “move from coordinate X to Y”).
- Behavioral Layer: The integrator, managing the continuous “sense-plan-act” loop between the information and physical spaces.
- Physical Entity: The hardware body—sensors, actuators, mechanical parts—that perceives and acts upon the environment.
The FJSESA metamodel formally defines these concepts. Its core elements are: Physical Entity (PhyEnt), Communication & Interaction (ComInt), Analysis & Decision (AnaDec), and Unit Behavior/Service Capability (SerCap). Their relationships—such as Composition (a Physical Entity is composed of control and sensor modules), Aggregation (Unit Behavior aggregates production, logistics, and storage actions), and Dependency (Analysis depends on real-time state)—are rigorously specified. This formalization ensures that every instantiated embodied AI robot, whether a milling machine or an AGV, adheres to the same structural blueprint, enabling seamless semantic understanding and interaction. For example, an AGV agent’s Physical Entity includes its drive unit and location sensors; its Unit Behavior includes actions like “navigate from point A to B”; and its Analysis & Decision module can generate commands based on real-time traffic perception.
Instantiating the FJSESA metamodel yields a population of structurally consistent atomic embodied AI robot agents. To manage complexity, we organize them into a two-tier architecture based on the Service Unit concept. Atomic Unit Agents (e.g., a single CNC machine, a single AGV) reside at the lower tier, directly controlling their physical hardware. Groups of co-located or functionally similar atomic agents are logically clustered under a managing Service Unit Virtual Agent, deployed on an edge industrial computer. At the highest level, a Workshop Management Agent coordinates between Service Units. This creates a hybrid distributed system: atomic embodied AI robot agents can make fast, local decisions based on physical feedback, while virtual management agents handle higher-level task allocation and coordination. Crucially, each agent type has access to different information and thus employs a distinct set of distributed scheduling strategies, as shown in the following table:
| Agent Type | Information Access | Strategy Name | Description |
|---|---|---|---|
| Workshop Agent | Global order information (due dates, priorities) | ATP | Assign the job that arrived earliest. |
| HUP | Assign the job with the highest urgency priority. | ||
| RPP | Assign the job with the most remaining operations. | ||
| EDP | Assign the job with the earliest delivery date. | ||
| Service Unit Agent | Machine load and status within its unit | STP | Select the machine with the shortest processing time for the task. |
| LRP | Select the machine with the lowest current load rate. | ||
| SQP | Select the machine with the fewest queued tasks. | ||
| Atomic Unit Agent (Embodied AI Robot) | Its own task queue and real-time execution state | TPX | Swap two non-adjacent operations in its local queue. |
| PFI | Pick and insert an operation elsewhere in its queue. | ||
| NBE | Swap two adjacent operations in its queue. | ||
| HSE | Swap two halves of its queue. | ||
| SQI | Reverse the order of a subsequence of operations. |
The ability of an atomic embodied AI robot to adjust its local strategy (e.g., using NBE or SQI) before participating in system coordination is a key advantage. To orchestrate these agents effectively, we developed a Q-Bargaining Game negotiation mechanism. This hybrid approach combines the adaptive learning of Q-learning with the strategic interaction of multi-player bargaining games. Each embodied AI robot agent (the “bargainer”) uses a Q-learning algorithm to explore its local strategy space (from the table above) and learns the expected utility (e.g., inverse of its completion time) for different actions in different states. The exploration-exploitation balance is managed by a dynamically decaying $\epsilon$-greedy policy:
$$\epsilon_t = \frac{1}{1 + e^{10 \times (0.6 – \frac{t}{t_{\text{max}}})}}$$
where $t_{\text{max}}$ is related to the problem scale. The agent then forms a set of potential strategies with their computed payoffs. In the bargaining phase, these “bargainer” agents submit their candidate strategy sets to a “counter-bargainer” agent (e.g., a Service Unit manager). The counter-bargainer evaluates the combined impact of these strategies on the global objective (e.g., total makespan) and selects the combination that maximizes system-wide utility. This creates a cooperative-competitive dynamic where individual embodied AI robot agents seek good local outcomes, but the system converges towards a globally efficient schedule.
The theoretical benefit in communication efficiency is significant. In a traditional multi-agent system, every local action choice (e.g., a machine picking its next job) typically requires a communication event to update the global state. For a job with $j$ operations, where each operation can be performed on $m$ machines with $b$ possible behavioral actions per machine, the communication complexity grows rapidly. The total number of communications $T_{\text{traditional}}$ can be approximated by the sum of choices at each step:
$$T_{\text{traditional}} \approx \sum_{k=1}^{j} (b \cdot m – k)$$
In our embodied AI robot system, each agent first develops a full local strategy set *internally* via its own perception and decision modules. It then communicates this entire set once for the bargaining process. The communication complexity $T_{\text{embodied}}$ is thus largely independent of the number of internal action evaluations and is significantly lower:
$$T_{\text{embodied}} \approx \sum_{k=1}^{j} (m – k)$$
This reduction in message-passing is a direct result of endowing agents with embodied, self-contained decision-making capability.
We validated our metamodel and scheduling system using two real-world case studies from small-scale structural component manufacturing workshops. Case 1 featured an integrated system with an automated storage/retrieval system (AS/RS), laser marker, CNC machining centers, AGVs, and collaborative robots. Case 2 comprised a conveyor-linked system with a gantry robot, engraving machine, laser marker, and inspection station. The key processing time data for jobs in each case is summarized below:
| Job | Op. | MU1 | MU2 | MU3 | MU4 | MU5 | MU6 |
|---|---|---|---|---|---|---|---|
| J1 | O11 | 3 | 3 | 5 | – | – | – |
| O12 | – | – | – | 8 | 3 | 9 | |
| J2 | O21 | 5 | 7 | 3 | – | – | – |
| O22 | – | – | – | 2 | 6 | 7 |
Instantiation of the FJSESA metamodel for resources like AGVs and CNC machines ensured a uniform structure across all agents, providing the foundation for effective collaboration. We compared our Embodied Service Unit Multi-Agent System (ESU-MAS) against a state-of-the-art non-embodied Service Unit MAS (SU-MAS) under three dynamic scenarios: 1) New job arrival, 2) Machine failure at t=5 min for 3 min duration, and 3) Scalability test with increased resources and orders. Performance was measured by schedule stability $P(e)$ (relative deviation in total completion time post-disturbance), average inter-agent communication count $T_{ave}$, and average negotiation response time $L_{ave}$.
The results were compelling. Under the new job arrival disturbance, ESU-MAS improved schedule stability by an average of 42.75%, reduced communication traffic by 58.33%, and decreased response time by 32.27% compared to SU-MAS. When handling a machine failure, the improvements were even more pronounced: stability improved by 42.88%, communications dropped by 62.5%, and response time was reduced by 33.28%. The scalability tests confirmed that the communication advantage of the embodied AI robot approach grows as the system scales, making it significantly more efficient and robust for larger workshops.
| Metric | Case 1 (SU-MAS) | Case 1 (ESU-MAS) | Case 2 (SU-MAS) | Case 2 (ESU-MAS) |
|---|---|---|---|---|
| Stability $P(e)$ (lower is better) | 0.486 | 0.235 | 0.525 | 0.347 |
| Avg. Comm. $T_{ave}$ | 16.5 | 5.5 | 6.0 | 3.0 |
| Avg. Resp. Time $L_{ave}$ (ms) | 184.17 | 104.17 | 95.26 | 65.63 |
Furthermore, we evaluated the global optimization capability of our Q-Bargaining Game mechanism within the ESU-MAS framework against other multi-agent negotiation protocols (e.g., Artificial Immune, Evolutionary Game) on established benchmark problems (Brandimarte’s MK01-10 and Kacem’s KM01-05). Our method consistently achieved makespans equal to or very close to the known best solutions, demonstrating superior optimization performance. The average relative error (RE) of our method was 21.9%, outperforming the next best method by 22.6% and others by over 56%. This shows that the strategy diversity enabled by embodied AI robot agents, combined with intelligent coordination, effectively avoids local optima and finds high-quality schedules.
In conclusion, this research presents a comprehensive methodology for deploying embodied AI robot agents in flexible manufacturing shops. The FJSESA metamodel provides the essential blueprint for building structurally consistent, physically grounded intelligent agents. The subsequent multi-agent system architecture and the Q-Bargaining Game negotiation mechanism leverage this embodiment to create a scheduling solution that is not only more robust and responsive to disturbances but also more computationally efficient and globally effective. By shifting from a paradigm of centralized or loosely-coupled virtual coordination to one of distributed, physically-aware intelligence, we unlock a new level of resilience and performance for dynamic manufacturing environments. Future work will focus on integrating more advanced sensory feedback (e.g., vision for quality checks) and deep reinforcement learning models into the embodied AI robot‘s decision module, further closing the loop between the physical and digital worlds for truly cognitive manufacturing systems.