With the rapid evolution of the automotive industry towards greater intelligence and connectivity, the intelligent cockpit has emerged as the central platform for enhancing the driving and riding experience. By integrating advanced visual displays, speech recognition, haptic feedback, and even olfactory systems, these cockpits create a novel human-machine interaction (HMI) environment based on multimodal fusion, aiming to deliver more convenient, comfortable, and personalized services.
However, this technological convergence, while creating value, introduces unprecedented complexity and potential risks. Firstly, on the visual front, infrared (IR) fill lights used for biometric recognition and large-area, high-brightness displays can produce glare or invisible light radiation, causing visual distraction or fatigue for the driver; prolonged exposure, particularly to specific wavelengths, may pose risks to ocular health. Secondly, in the tactile domain, complex touch operations, though enriching interaction, may excessively occupy the driver’s manual resources, increasing the probability of operational distraction and impacting driving safety. Finally, regarding the olfactory sense, volatile organic compounds (VOCs) emitted by cabin fragrance systems or new interior materials (e.g., certain leathers, plastics), if not properly controlled in concentration and composition, can cause discomfort or even health concerns for occupants. Traditional testing methodologies struggle to comprehensively and quantitatively assess these novel interaction risks, creating an urgent need to construct a testing platform capable of simulating human perception and behavior with high fidelity.
To address these challenges, our research aims to develop a dedicated bionic robot system for intelligent cockpit interaction safety testing. The core innovation of this system lies in integrating self-developed high-performance optoelectronic sensors and flexible pressure sensors onto a bionic robot platform, simulating human visual and tactile perceptual capabilities. By replicating the interaction behaviors of real users while synchronously collecting multi-physics field data, this bionic robot system provides a novel testing methodology that is high-precision, quantifiable, and repeatable for evaluating cockpit interaction safety. This offers robust data support for assessing and mitigating potential human-factor risks, holding significant importance for promoting the safety design and standardization of HMI in intelligent vehicles.
The realization of high-fidelity, biomimetic testing for intelligent cockpit HMI scenarios hinges on the development of a bionic robot that closely mimics a human being in both structure and material properties. This bionic robot must not only possess the geometric features of the human form but, more critically, approximate human dynamic characteristics such as force transmission and sensory feedback to ensure the validity and reliability of test data.
1. Design and Selection of the Biomimetic Material System
The biomimetic design of the bionic robot originates from a materials science deconstruction of human biological structures, selecting corresponding engineering materials based on the functional requirements of each body part.
The robot’s head and neck, serving as the base for supporting critical sensors (e.g., cameras, microphones), require high stiffness and strength for stability. We selected Polyether Ether Ketone (PEEK) as the primary material. PEEK is a specialty engineering plastic whose elastic modulus (approximately 15 GPa) and Poisson’s ratio (0.3) are close to those of human bone. It also offers excellent fatigue resistance, creep resistance, and lightweight properties, perfectly simulating the supportive and protective functions of the skeletal system.
The robot’s motion and force execution require simulation of human softness and elasticity. Thermoplastic Polyurethane (TPU) was chosen to construct the robot’s “muscle” tissue. TPU possesses superior abrasion resistance, high elasticity, and adjustable tear strength. Configured with an elastic modulus of approximately 1 MPa and a Poisson’s ratio of 0.5, it can simulate the mechanical behavior of real muscles undergoing large deformation, absorbing shock, and transmitting force, which is key to achieving biomimetic motion.
The robot’s outer skin needs a soft touch, appropriate coefficient of friction, and adaptability for sensor embedding. Silicone rubber was selected as the ideal skin material due to its non-toxicity, excellent biocompatibility, high chemical stability, and tactile feel very similar to human skin. For the tendons connecting “bone” and “muscle,” a specially formulated elastic material was used, with an elastic modulus of about 100 MPa and a Poisson’s ratio of 0.5. This material can transmit tensile forces while providing necessary extensibility and resilience, ensuring smooth and natural movement.
| Robot Component | Biomimetic Role | Selected Material | Key Mechanical Properties (Approx.) | Simulated Human Tissue |
|---|---|---|---|---|
| Head, Neck Frame | Structural Support & Sensor Mount | PEEK | E ≈ 15 GPa, ν ≈ 0.3 | Cortical Bone |
| Arms, Torso Core | Force Execution & Motion | TPU | E ≈ 1 MPa, ν ≈ 0.5 | Muscle Tissue |
| Outer Shell / Skin | Tactile Interface & Sensor Substrate | Silicone Rubber | Soft, High Friction, Biocompatible | Epidermis/Dermis |
| Connective Links | Force Transmission | Special Elastic Polymer | E ≈ 100 MPa, ν ≈ 0.5 | Tendons/Ligaments |
2. Biomimetic Integration of Multi-Modal Sensors
The perceptual system acts as the bionic robot‘s “nerve endings,” forming the basis for its interaction with the intelligent cockpit and data collection.
A self-developed, high-performance photodetector is embedded within the orbital cavity of the robot’s head to simulate the human eye. This sensor is sensitive not only to visible light but also specifically tuned to detect infrared (IR) fill lights (common at 850 nm and 940 nm wavelengths) used in cockpits for functions like face recognition and driver monitoring. Its high responsivity and low-noise characteristics enable precise quantification of visual perception impacts such as screen flicker and strong light glare, thereby allowing for the assessment of visual interaction safety. The performance can be characterized by its responsivity ($$R_\lambda$$) and specific detectivity ($$D^*$$):
$$R_\lambda = \frac{I_{ph}}{P_\lambda}$$
where $$I_{ph}$$ is the photocurrent and $$P_\lambda$$ is the incident optical power at wavelength $$\lambda$$. The specific detectivity is given by:
$$D^* = \frac{R_\lambda \sqrt{A \Delta f}}{I_n}$$
where $$A$$ is the detector area, $$\Delta f$$ is the bandwidth, and $$I_n$$ is the noise current.
Flexible pressure sensors are embedded in key touch-point areas of the robot, such as the fingertips and palms. These sensors are based on nano-composite materials, offering good linearity and mechanical durability. They precisely measure the magnitude, distribution, and duration of pressure generated during interaction with touchscreens and physical buttons, quantifying the required press force, sliding friction, etc. This data is used to assess the convenience and potential fatigue of touch interactions, as well as the safety risks arising from the occupation of manual resources. The sensor’s sensitivity ($$S$$) is a key parameter:
$$S = \frac{\Delta (\Delta I / I_0)}{\Delta P}$$
where $$\Delta I / I_0$$ represents the relative change in current and $$\Delta P$$ is the change in applied pressure.
Through the deep integration of this biomimetic material system and multi-modal sensors, we have successfully constructed a testing platform that highly simulates human biomechanical properties and perceptual capabilities. This bionic robot can perform interaction tasks—such as operating a touchscreen, observing screen information, and perceiving ambient light—in a manner closely resembling a human, while synchronously recording detailed physical parameters. It provides an unprecedented technical means for the objective and quantitative assessment of interaction safety in intelligent cockpits.
3. Development and Characterization of Core Sensors
3.1 Optoelectronic Sensor Development
For the detection of IR and blue light hazards, we developed photodetectors based on heterojunction composites of one-dimensional (1D) metal oxide semiconductor (MOS) nanowire arrays and two-dimensional (2D) graphene oxide (GO) films. Utilizing the advantages of GO—which exhibits p-type semiconductor properties with tunable energy levels due to introduced oxygen functional groups—we formed a heterojunction structure with the 1D MOS nanowires. This material composite enables energy band matching and performance complementarity, effectively reducing dark current and noise while significantly enhancing the hole gain of the device based on n-type oxide semiconductors. This leads to a comprehensive improvement in the detection capabilities of the photodetector. The MOS nanowire arrays were synthesized via a hydrothermal method. The optoelectronic devices were then fabricated using semiconductor processing techniques. The typical spectral responsivity ($$R_\lambda$$) curve of the fabricated device is shown conceptually below, demonstrating its suitability for simulating the human eye in the bionic robot.
| Parameter | Value at 450 nm (Blue) | Value at 850 nm (IR) | Value at 940 nm (IR) | Units |
|---|---|---|---|---|
| Peak Responsivity ($$R_\lambda$$) | 0.35 | 0.28 | 0.22 | A/W |
| Specific Detectivity ($$D^*$$) | 2.1 × 1011 | 1.8 × 1011 | 1.5 × 1011 | Jones |
| Response Time (Rise, 10%-90%) | < 150 | ms | ||
| Dark Current ($$I_d$$) | < 1 | nA | ||
3.2 Pressure Sensor Development
We fabricated piezoresistive sensing films by compositing elastic matrices like Polydimethylsiloxane (PDMS) or TPU with micro-nano conductive materials such as Multi-Walled Carbon Nanotubes (MWCNTs). A three-dimensional conductive network was constructed within the elastic matrix using solution blending and ultrasonic dispersion methods. A multi-layer encapsulation structure was employed: an inner PI rigid frame protects the sensor and circuitry, a surrounding PDMS elastic layer buffers strain, and an outermost layer of medical-grade adhesive enhances conformability and biocompatibility. The sensor’s response to a 300 kPa pressure application and release cycle showed a response time and recovery time of approximately 547 ms each. The IV characteristics under varying pressures from 25 kPa to 400 kPa demonstrated excellent linearity, confirming its suitability for integration into the bionic robot‘s hands and fingers.
| Parameter | Value | Units / Conditions |
|---|---|---|
| Sensitivity ($$S$$) in Low Pressure Regime (<50 kPa) | 0.85 | kPa-1 |
| Linear Operating Range | 0 – 400 | kPa |
| Response Time (to 90% signal) | 547 | ms |
| Recovery Time (to 90% baseline) | 547 | ms |
| Hysteresis Error | < 5 | % FSO |
| Cycle Durability (Tested) | > 10,000 | Cycles @ 200 kPa |
4. Quantified Testing Data from Bionic Robot Evaluations
The integrated bionic robot system was deployed in a representative intelligent cockpit setup to perform standardized safety tests. The following data tables summarize key findings across visual, tactile, and environmental domains.
4.1 Visual Interaction Safety Assessment
The bionic robot was positioned in the driver’s seat, with its “eyes” aligned towards the central infotainment display. Tests measured hazards from active IR sources and display emissions.
| IR Source Type / Wavelength | Cockpit Function | Measured Irradiance at Cornea | Applicable Safety Limit (ICNIRP/ACGIH 8-hr) | Margin to Limit |
|---|---|---|---|---|
| Near-IR (850 nm) | Driver Monitoring Camera | 12.5 W/m² | 100 W/m² | 87.5 W/m² |
| IR Array (940 nm) | Face Recognition / Fatigue Detection | 8.7 W/m² | 100 W/m² | 91.3 W/m² |
While measured values are within safe limits, the bionic robot successfully quantified the exposure, providing a baseline for assessing potential additive effects from multiple IR sources or longer exposure durations.
| Display Mode / Content | Measured $$L_B$$ | Notes |
|---|---|---|
| Maximum Brightness, White Screen | 2.3 W·m⁻²·sr⁻¹ | Baseline maximum hazard |
| Maximum Brightness, Navigation Map | 1.9 W·m⁻²·sr⁻¹ | Typical application scene |
| Maximum Brightness, Dark Mode UI | 1.3 W·m⁻²·sr⁻¹ | Risk reduction via software |
| 50% Brightness, White Screen | 1.6 W·m⁻²·sr⁻¹ | Impact of brightness control |
The blue light weighted radiance $$L_B$$ is calculated according to the spectral weighting function $$B(\lambda)$$:
$$L_B = \frac{\int L_e(\lambda) \cdot B(\lambda) d\lambda}{\int B(\lambda) d\lambda}$$
where $$L_e(\lambda)$$ is the spectral radiance of the display. The data indicates that while instantaneous values are below typical safety thresholds (e.g., ~10 W·m⁻²·sr⁻¹ for brief viewing), the bionic robot provides crucial quantitative evidence that screen brightness and content are primary drivers of blue light hazard potential, supporting the design of protective features.
4.2 Tactile Interaction Safety Assessment
The bionic robot‘s finger, equipped with the flexible pressure sensor, performed repeated touch interactions on the cockpit’s central touchscreen. Metrics were collected to assess operational efficiency and distraction potential.
| Interaction Task (Menu Path) | Avg. Activation Force (N) | Avg. Sliding Distance (mm) | Task Completion Time (s) | Success Rate (%) | Distraction Risk Level* |
|---|---|---|---|---|---|
| Turn ON Air Conditioning (1-level) | 1.7 ± 0.2 | 15.2 ± 3.1 | 2.1 ± 0.3 | 100 | Low |
| Set Destination in Nav (2-level) | 1.8 ± 0.3 | 28.4 ± 5.6 | 3.5 ± 0.5 | 98 | Moderate |
| Adjust Advanced Audio Settings (3-level) | 2.0 ± 0.4 | 43.5 ± 8.2 | 5.8 ± 0.9 | 85 | High |
* Risk Level based on Task Completion Time: Low (<3s), Moderate (3-5s), High (>5s). The data from the bionic robot clearly quantifies the increase in required force, interaction distance, and—most critically—task completion time with deeper menu hierarchies. Tasks exceeding ~4-5 seconds are considered high-distraction-risk, providing objective, data-driven criteria for HMI design evaluation that are difficult to obtain consistently with human testers.
4.3 Extended Environmental Sensing Capability
To demonstrate the system’s extensibility, the bionic robot platform was equipped with a prototype environmental sensor package (VOC/particulate matter sensor) located in the chest cavity, simulating human inhalation zone monitoring.
| Condition / Time Elapsed | TVOC Concentration (ppb) | PM2.5 Concentration (μg/m³) | Notes |
|---|---|---|---|
| Initial Cabin (Cold Start) | 1250 | 12 | Elevated VOCs from materials |
| After 10 min (AC on, Recirculation) | 850 | 8 | Gradual reduction |
| After 20 min (AC on, Fresh Air) | 320 | 5 | Significant improvement with fresh air intake |
| Activation of Cabin “Air Purge” Function | < 100 | < 2 | Demonstrates system effectiveness |
5. Conclusion and Future Perspectives
This research successfully developed and demonstrated an integrated bionic robot system for the comprehensive safety evaluation of intelligent cockpits. The core achievement is the creation of a testing platform where biomimetic materials and custom-developed, high-precision sensors are seamlessly integrated into a bionic robot that replicates human interaction kinematics and perception. The system provides an objective, quantifiable, and repeatable testing paradigm that addresses the limitations of traditional methods in assessing novel risks from multimodal HMI.
The quantified test data obtained—from IR/blue light exposure levels to tactile interaction force/time profiles and environmental air quality—provides actionable insights for OEMs and suppliers. It enables the identification of specific design flaws (e.g., overly deep menu structures causing high distraction risk) and the validation of mitigation strategies (e.g., efficacy of dark modes or air purification systems). The formulas and performance metrics established for the sensors (e.g., $$R_\lambda$$, $$D^*$$, $$S$$) form a standardized basis for evaluation.
Future work will focus on enhancing the bionic robot‘s capabilities. This includes integrating more sophisticated actuators for dynamic postural adjustment during driving scenarios, adding thermal and humidity sensors for comprehensive comfort assessment, and developing advanced algorithms to synthesize the multi-sensor data into holistic safety scores. Furthermore, establishing standardized testing protocols using this bionic robot platform will be crucial for driving industry-wide improvements in intelligent cockpit safety, ensuring that technological innovation aligns with fundamental human factors and well-being principles.