Deep-learning-enabled breathable thermogalvanic hydrogel array for self-powered mental monitoring

Abstract

Facial expression is vital for assessing psychological health, especially in adolescents. Current facial expression recognition systems face challenges such as poor breathability, limited electromechanical performance, and insufficient environmental stability. Here, an air-permeable self-powered hydrogel array designed for continuous mental monitoring based on facial expression recognition is proposed. The remarkable breathability of the patch composed of thermogalvanic hydrogel and gelatin films is achieved by the through-holes structure design. Additionally, the hydrogel leverages a double network structure and phytic acid with rich hydrogen ions, achieving a trade-off between mechanical (1.07 MPa) and electrical (34.4 mS cm⁻¹) properties. Due to the multiple hydrogen bonds among phytic acid, glycerol, and water, the hydrogel maintains over 80% of its original electrical performance after 10 days, demonstrating its excellent environmental stability. Integrated with a deep learning algorithm, the hydrogel array can recognize six facial expressions with a high accuracy of 100% and provide long-term monitoring of mental states based on the positive-to-negative emotion ratio. This work paves the way for a new generation of wearable mental monitoring platform for healthcare and human–machine interfaces.

---

1. Introduction

Facial expression serves as a universal medium through which humans communicate emotions, attitudes, and psychological states, reflecting their internal mental and emotional well-being.^1–3 This form of non-verbal communication is especially crucial in understanding psychological health in adolescents, a group that is particularly vulnerable to emotional and mental challenges.⁴ The early detection of poor mental conditions, such as anxiety, depression, and stress disorders, during adolescence is critical for effective intervention and prevention.⁵ Current facial expression systems primarily rely on two types of technologies: image-based methods and physiological signal detection.^6–8 Image-based methods leverage computer vision and machine learning algorithms to analyze facial expressions captured by cameras.⁹ However, these methods face significant limitations, including vulnerability to lighting conditions, occlusions, and privacy concerns.¹⁰ On the other hand, physiological signal detection, such as facial electromyography (EMG) or electroencephalography (EEG), offers more reliable insights by correlating electrical signals with emotional states.^11–13 Nevertheless, these techniques often depend on bulky, rigid, and adhesive gel-based electrodes, which restrict their wearability and comfort during prolonged use.¹⁴ Moreover, the gold standard for psychological assessment, including diagnostic interviews and questionnaires, is time-consuming, subjective, and lacks real-time applicability.^15,16

Wearables have emerged as a promising solution to overcome the limitations of traditional systems by offering portability and real-time monitoring capabilities.¹⁷ Flexible, skin-integrated sensors, in particular, have shown the potential to revolutionize facial expression recognition.^18–20 For example, Lee et al. presented a self-powered, stretchable, and transparent personalized skin-integrated facial interface that overcomes the problems of bulkiness and batteries.²¹ Gao et al. proposed a dual-mode capacitive-electromyographic sensor for accurate facial expression recognition, combining capacitive pressure sensing and dry electrodes for improved performance and biocompatibility.²² Despite these advancements, current wearables still face challenges, such as poor breathability, limited mechanical and electrical performance, and insufficient environmental stability.^23,24 Most importantly, existing sensors often compromise long-term wearability, causing discomfort or irritation during extended use.^25,26

In response to the above problems, a breathable thermogalvanic hydrogel array with high environmental stability is proposed for continuous mental monitoring based on facial expression recognition. The patch leverages thermogalvanic hydrogel to acquire high-quality current signals with gelatin films as the adhesive substrates and encapsulation layers. The through-holes structure design endows the patch with superior air-permeability, enabling long-term skin attachment without any skin inflammation. The trade-off of the mechanical (1.07 MPa of stress) and electrical (34.4 mS cm⁻¹ of conductivity) performance is achieved by introducing the double network and phytic acid. Benefiting from the multiple hydrogen bonds between phytic acid, glycerol, and water, the thermopower and current can be maintained at 80% of their original values after 10 days. Moreover, the hydrogel exhibits remarkable recyclability and degradability owing to the reversible and dynamic noncovalent interactions. Combining with deep learning, the hydrogel array can realize the recognition of six different facial expressions with a high accuracy of 100%. Furthermore, the long-term mental monitoring is demonstrated by evaluating the ratio of positive and negative emotions (i.e., positive ratio). This study offers insights to develop a new generation of mental monitoring platform based on facial expression recognition for medical healthcare and human–machine interfaces.

2. Experimental section

2.1 Materials

Polyvinyl alcohol (molecular weight, 120 000 to 140 000), κ-carrageenan (MW = 788.7, ≥ 99%), glycerol (MW = 92.09, ≥ 99%), phytic acid solution (PA, 50% in H₂O), I₂ (MW = 253.81, ≥ 99%), KI (MW = 166, ≥ 99%) were purchased from Sigma-Aldrich. The graphite electrodes were purchased from Suzhou TANFENG graphene Tech Co., Ltd (China). All materials were used as received without further purification.

2.2 Preparation of the PCPG

The hydrogel containing polyvinyl alcohol (PVA), κ-carrageenan, phytic acid, and glycerol was prepared by a one-pot method. In brief, 16 g of glycerol–water binary solvent (1 : 4) and 4 g of phytic acid aqueous solution were first mixed well, and then 2.22 g of PVA and 0.3 g of κ-carrageenan were added to the mixture in turn while stirring. The obtained suspension was transferred into an oil bath at 95 °C until the powders dissolved and the suspension became a clear liquid. Second, I₂ (0.203 g) and KI (0.266 g) were added to the mixed solution. After being stirred at 95 °C for 2 h, the solution was poured into a polytetrafluoroethylene (PTFE) mold. Third, the sample was frozen at −20 °C for 18 h and thawed at 20 °C for gelation. For simplification, the hydrogel was represented as PCPG hydrogel. The hydrogels with different compositions were prepared in the same procedure. Furthermore, 2 g of gelatin was added to 16 g of glycerol–water binary solvent (1 : 1), stirred at 60 °C for 1 h, then poured into a PTFE mold and refrigerated for 18 h. Finally, the PCPG hydrogel and the graphite electrodes as well as copper wires attached to its surface are encapsulated by a gelatin film and use the thermogalvanic effect to generate electrical signals. The through-holes architecture of the hydrogel was fabricated via accurate laser cutting technology, generating a kirigami-mimetic configuration.

2.3 Characterization

The morphology and chemical compositions of the hydrogels were characterized using scanning electron microscope (SEM) technique (Hitachi SU8010) and FTIR (Spectrum 100). A thermogravimetric analyzer, TGA-601 from HUIC, was employed for thermogravimetric analysis under an air atmosphere, with a heating rate of 20 °C min⁻¹. Mechanical properties, both tensile and compressive, of the PCPG were evaluated at room temperature using a universal mechanical test machine (QT-1196). For the tensile test, rectangular samples measuring 25 × 5 × 1 mm³ were tested at a speed of 100 mm min⁻¹. The compressive test involved cylindrical samples with a height of 20 mm and a diameter of 10 mm, conducted at a rate of 5 mm min⁻¹.

2.4 Thermoelectric measurement

For the thermoelectric measurements, all samples were prepared with dimensions of 20 × 20 × 8 mm³. The samples leveraged in the stability tests had dimensions of 20 mm in length, 4 mm in width, and 2 mm in thickness. A Keithley 2400 instrument was employed to measure the output voltage and current. The temperature of the two Peltier chips was managed by a direct current source, and thermocouples (NAPUI TR230X) were used to monitor the temperature. Electrochemical impedance spectroscopy (EIS) and cyclic voltammetry (CV) of the hydrogels were assessed using an electrochemical workstation (CHI660e) from Shanghai Chenhua Instrument Co. Ltd The ionic conductivity of the PCPG was determined using the formula σ = d/R × A, where d is the thickness, R is the impedance value, and A is the cross-sectional area of the PCPG. Hydrogels in a rectangular shape of 30 × 10 × 5 mm were subjected to solvent retention and long-term performance testing in an oven at 298 K and 30% relative humidity. The weight and thermoelectric performance of these samples were documented at various intervals.

2.5 Deep-learning-based data analysis and mental monitoring

Our facial expression recognition system is designed as an auxiliary tool for mental health assessment. 10 volunteers were recruited to capture six basic facial expressions (happy, sad, surprise, fear, anger, and disgust) under standardized conditions, resulting in 3000 high-quality samples (10 participants × 6 expressions × 50 repetitions each). Data collection was conducted using a thermoelectric hydrogel array connected to a flexible wireless multichannel sensing module to monitor current signals generated by facial expressions. To ensure the accuracy of the data labels for the six facial expressions, we implemented a rigorous validation process. Each expression was recorded as a video segment and independently labeled by a panel of five trained annotators. Only segments with at least four annotators in agreement were retained, ensuring high inter-rater reliability. Ambiguous or unclear segments were excluded, guaranteeing the consistency and accuracy of the labeled dataset. 75% of facial expression datasets were randomly selected from the database to serve as training samples, while the remaining 25% were designated as testing samples. A deep learning architecture was established using the PyTorch framework, which included an input layer, several convolutional neural network layers, and an output layer. To assess the classification capabilities of the built Resnet-34 neural network, the t-SNE technique was leveraged to project complex data into a simplified 2D feature space for visualization. Data preprocessing for facial expression recognition was carried out using a Python script that encompassed filtering, sampling, integration, and normalization steps. An intermittent measurement approach for mental monitoring was employed, with data collected every 2 hours over an 8 hours period to ensure participant comfort. Ethical guidelines were strictly adhered to throughout the process to safeguard participant health and privacy.

All experiments were performed in compliance with the relevant guidelines. All human subjects provided written informed consent.

3. Results and discussion

3.1 Design of the skin-integrated thermogalvanic hydrogel array

Fig. 1a illustrates the thermogalvanic hydrogel patch with a sandwich structure, which is composed of five layers: the top and bottom layers act as the breathable encapsulation films for reliably adhering to the skin; the sensing hydrogel layer is located at the middle layer of the patch for acquiring high-fidelity electrical signals about facial expression in a self-powered manner; the other layers are the flexible graphite electrodes with high conductivity. The thermogalvanic hydrogels with double polymer networks and electrolyte ions are fabricated via a simple one-pot process (Fig. 1b). PVA with hydroxyl groups and carrageenan (CG) with sulfate groups can serve as the first and second network, respectively.²⁷ Phytic acid, possessing six phosphoric acid molecules, is beneficial for forming dynamic hydrogen bond interactions with polymer networks to further strengthen the mechanical performance of the hydrogels.²⁸ Moreover, an abundance of free hydrogen ions can be obtained because of the strong acidity of the PA, which greatly enhances the conductivity of the hydrogels. This strategy will provide a new avenue for achieving the inherent trade-off between ion conductivity and mechanical strength.²⁹ In addition, the non-drying properties of the hydrogels are effectively improved in the presence of PA and glycerol (Gly).³⁰ To achieve the self-powered sensing, the I⁻/I₃⁻ is introduced into the hydrogels which generate a temperature-dependent voltage using the temperature gradient between the skin and environment.³¹ As shown in Fig. 1c and d, the array is directly fixed at five sites on the human face to detect the muscle movement of facial expression simultaneously in a self-powered manner.³² Subsequently, the expression information acquired from the hydrogel array is transferred into a pretrained deep learning model for human emotion classification (Fig. 1e–g). Thanks to the through-holes structure design, the hydrogel array not only offers good air permeability but also allows long-term mental monitoring by evaluating the ratio of positive to negative emotions (Fig. 1h).

Fig. 1 Design of the skin-integrated thermogalvanic hydrogel array. (a) Structure design of the thermogalvanic hydrogel patch. (b) Schematic illustration of the molecular structure of the PCPG. (c) The layout scheme for the PCPG assembly on the face, and (d) relation to the sensory stimulus of facial strain. (e–h) Applications of the PCPG in mental monitoring based on facial expression recognition.

3.2 Mechanical performance of the PCPG

The preparation procedure of the PCPG is depicted in Fig. 2a. A precursor solution containing polymer chains and solute molecules is obtained through the one-pot method, accompanied by the freezing-thawing process. Abundant hydrogen bonds are formed between PVA chains, CG, Gly, PA, and water molecules, which are demonstrated by the shifting of hydroxyl groups of PVA (3328 cm⁻¹) to a lower frequency of (3234 cm⁻¹) for the hydrogels (Fig. 2b and c). We further regulated the mechanical performance of the PCPG hydrogels with different contents of CG and PVA. As depicted in Fig. S1a,† the hydrogel demonstrates a large strain of 533% and stress of 1.07 MPa, significantly higher than the samples without CG. Additionally, the Young's modulus (108 kPa) of the hydrogels with 1.5 wt% of CG is well matched to that of the human face skin, ensuring a high sensing accuracy and good wearing comfort (Fig. 2d). Furthermore, the PVA content is tuned to produce an optimum hydrogel with a fixed CG concentration (1.5 wt%) (Fig. 2e and S1b†). Results show that the 10 wt% of PVA yields a stronger hydrogel and the specific stress–strain values are concluded in Fig. 2f. Therefore, the hydrogels with 1.5 wt% of CG and 10 wt% of PVA are employed in the following characterizations unless otherwise noted.

Fig. 2 Mechanical property of the PCPG. (a) Schematic structure of the crosslinked network of the PCPG hydrogel in freeze–thawing processes. (b) Intermolecular interactions in hydrogels after freezing-thawing. (c) FTIR spectra from bottom to top are, respectively, PVA, CG, PVA/CG, PVA/CG/Gly, PVA/CG/PA/Gly (PCPG). (d) Young's modulus and toughness of the PCPG with different carrageenan contents. (e) Tensile stress–strain curves with different PVA contents. (f) Stress–strain behavior of PVA and CG as a function of concentration. (g) Photographs showing the PCPG exhibits the ability to lift a weight of 500 g. Scale bar 1 cm. (h) Top: Tensile loading–unloading curves of hydrogels without cracks under different strains and schematic diagram. Bottom: Cyclic stress–time curve of hydrogels with cracks under different strains and schematic diagram. (i) The schematic structures of the PCPG hydrogel with crack blunting effects. (j) The fracture energy of the notched hydrogels with different compositions is shown from left to right as follows: CG (C), PVA (P), PVA/CG (PC), PVA/CG/Gly (PCG), and PVA/CG/PA/Gly (PCPG).

To further demonstrate the mechanical strength of the hydrogel, it was stretched to over four times of original length without obvious rupture (Fig. S2a†). Meanwhile, the well-designed polymer networks enable the hydrogel to endure various types of deformation and lift 500 g weight (Fig. 2g and S2b†). Subsequently, the elasticity and recovery performance of the hydrogels were evaluated by cyclic elongation and compression tests. Successive loading cycles of the hydrogel exhibit a large hysteresis in the first loading cycle, but the almost overlapped cycles for the following tests are observed thanks to the self-recovery of hydrogen bonds (Fig. S3a†). The distinct hysteresis loops are present in the loading–unloading curves of the hydrogels under various strains from 50% to 400%, showing that enormous energy is dissipated by breaking the hydrogen bonds during elongation (Fig. 2h). Due to the strong intermolecular interactions, the hydrogel can achieve remarkable resistance to crack under 50% strain for 100 cycles (Fig. 2h and i). Moreover, the highest fracture energy of the PCPG demonstrates the formation of the multiple hydrogen bonds in hydrogels (Fig. 2j and S4†). Not only stretchable, the hydrogels also possess excellent compressive performance and recover from a large strain of 50% (Fig. S3b†). This demonstrates the PCPG hydrogel is outstanding in terms of ultra strength, elasticity, skin-mimetic modulus, and crack blunting capability, which makes it highly suitable for facial expression recognition.

3.3 Thermoelectrical performance of the PCPG

The introduction of I⁻/I₃⁻ endows the PCPG hydrogels with heat-to-electricity conversion capability based on the thermogalvanic effect. Thus, the hydrogels can generate a temperature-dependent voltage and current characterized by the Seebeck coefficient (S_e). The redox reactions of the hydrogels, taking place at the hot and cold ends, are depicted in Fig. 3a:

Hot end: I₃⁻ + 2e⁻ → 3I⁻ (1)

Cold end: 3I⁻ − 2e⁻ → I₃⁻ (2)

Fig. 3 Thermoelectric property of the PCPG. (a) Schematic illustration of the thermoelectric conversion mechanism. (b) Temperature and potential distribution of the finite element model. (c) Cyclic voltammograms of the PCPG gels with different temperatures. (d) Peak current versus the square-rooted scan rate. (e) S_e and conductivity variations with different I⁻/I₃⁻ concentrations. (f) The output voltage–current-power curves under different temperature differences (ΔT). (g) Possible non-drying mechanism of the PCPG hydrogel. (h) Thermogravimetric characterization of the PCPG with different compositions. (i) The measured current curves at different days. ΔT = 5 K. (j) Skin condition after wearing the PCPG and PU film for 5 h. Scale bar 3 cm.

Moreover, the operation principle of the thermogalvanic hydrogels is validated by the COMSOL simulation in Fig. 3b. Next, the good reversibility of the redox reactions I⁻/I₃⁻ is evaluated using CV tests, which are demonstrated by the significantly symmetrical redox peaks of CV profiles under different temperatures (Fig. 3c). In addition, the linear relationship between the square-rooted scan rate and the peak current density of CV curves in Fig. 3d signifies that the redox reactions are limited by ionic diffusion. Benefiting from the interconnected porous structure in Fig. S5,† the hydrogels possess a good electrical performance. As depicted in Fig. 3e, the optimal thermoelectrical performance of 1.05 mV K⁻¹ and 34.4 mS cm⁻¹ is achieved in 0.04 M I⁻/I₃⁻ and 40 wt% PA. It is noteworthy that the addition of PA significantly enhances the conductivity of the hydrogels, which is 8 times higher than that of the hydrogels without PA (Fig. S6†).

The effect of temperature on the thermoelectrical performance is further investigated. The thermal conductivity of the hydrogels under different temperatures shows no obvious fluctuation, confirming a stable temperature gradient can be maintained in the hydrogels (Fig. S7†). Due to the enhanced mobility of ions, the conductivity increases as the temperature rises from 293 K to 333 K, reaching a maximum value of 40 mS cm⁻¹ (Fig. S8†). The output voltage–current-power density curves of the hydrogels under various temperature gradients (ΔT) were measured (Fig. 3f). By maintaining the hot and cold ends at 293 K respectively, the hydrogel still shows a maximum output power of 0.7 nW under a ΔT of −20 K, which is comparable to the value of 0.75 nW under a ΔT of 20 K. This is attributed to the fact that Gly and PA inhibit the crystallization of free water in hydrogels at low temperatures. As shown in Fig. S9,† the hydrogel's output voltage and current exhibit a significant linear relationship with the temperature change, highlighting its high sensitivity and responsiveness to thermal changes.

Dehydration is a key factor affecting hydrogel performance. PA and Gly can form multiple hydrogen bonds with water molecules, which can reduce water loss and retain 60% of original weight of hydrogels at room temperature (Fig. 3g and S10a†). Thermogravimetry analysis (TGA) is further performed to demonstrate that the thermal stability of the PCPG hydrogel is improved in the presence of PA and Gly (Fig. 3h). For variations in thermoelectrical performance, the voltage and current profiles were measured over different days. The thermopower of the hydrogel decreases from 1.05 to 0.81 mV K⁻¹ after ten days, corresponding to a decrease in current from 62 μA to 49 μA (Fig. 3i and S10b†). Recyclability is an appealing aspect that helps reduce e-waste and preparation costs. Due to the non-covalent interactions, the hydrogel exhibits superior recyclability and its electrical performance can be preserved at a high level after five cycles (Fig. S11†). What's more, the degradation of the PCPG hydrogel when buried in soil is demonstrated in Fig. S12,† which greatly reduces the burden of environmental pollution. The air-permeable property is necessary for the hydrogel array to achieve long-term on-face health monitoring. The micropores on the hydrogel and gelatin films significantly improve the breathability of the patch (Fig. S13†). Notably, the breathable patch has no significant effect on the human skin after 5 h wearing. In contrast, due to inhibition of the sweat evaporation, a slight irritation is observed after wearing the non-breathable PU film for the same period (Fig. 3j). For comparison, a radar plot is depicted in Fig. S14 and Table S1,† which clearly demonstrates the hydrogel possesses a relative comprehensive performance.

3.4 Self-powered strain sensing performance of the PCPG

As shown in Fig. 4a, the self-powered strain sensing mechanism leveraging the synergy of piezoresistive and thermogalvanic effect is elaborated under uniaxial elongation. When a temperature gradient is fixed between the two ends of the hydrogel, an initial thermal voltage based on the thermogalvanic effect is produced. Once the hydrogel encounters a tensile strain, the resistance (R = ρ × L/S) will swiftly increase to a plateau due to the increase of L and decline of S. Benefiting from the deformation-independent thermovolage, the current changes in a similar way to the resistance variation (Fig. 4b). Then, the conductive pathway can quickly recover after unloading the tensile strain, resulting in a reduced resistance. By increasing the tensile strain from 0% to 200%, the effective conductivity of the hydrogel significantly decreases by one third (Fig. 4c). The strain-sensitive performance of the PCPG hydrogel is further investigated. The gauge factor (GF) for evaluating the sensitivity of the hydrogel is defined as GF = (ΔI/I₀)/ε (where ΔI = I − I₀, I and I₀ signify the current with or without applied stretching, respectively, ε represents tensile strain). After being stored at room temperature for 10 days, the sensitivity remained stable, indicating that the electrical performance of the device was not significantly affected during this period (Fig. S15†). When a uniaxial elongation is applied, the relative current variation shows an obvious response, especially at the strain of smaller than 20% (Fig. 4d). As shown in Fig. 4e, the response and recovery time are determined as 230 ms and 220 ms, respectively, which are sufficient for the perception of facial muscle movement. The current response is highly reversible during loading and unloading stepwise strains (Fig. 4f). Moreover, based on the hysteresis loops under different strains, it is highly suitable for detecting small strains (Fig. S16†). The sensitivity of the strain sensing remains nearly constant across various temperature gradients (Fig. S17†). Thanks to the excellent flexibility of the hydrogel, the thermopower remains almost constant at various bending angles (Fig. 4g). Considering the long-term facial expression monitoring, the anti-fatigue performance of the hydrogels is evaluated under 500 repeated tensile/release cycles (Fig. 4h). Apparently, the amplitude of current changes has no obvious shift after each cycle, demonstrating the PCPG hydrogel's huge potential in wearable sensing applications.

Fig. 4 Sensing mechanism and performance of the PCPG. (a) Schematic diagrams of the resistance changes of the PCPG during the compressing and recovering process. (b) The voltage and current of the PCPG generated by repeated stretching. (c) Current–voltage curves of the hydrogel at different stretching strains. (d) Relative current change of the PCPG as a function of applied strain. (e) Relative variations of current of PCPG under gradient tensile strains of 0 to 50% and its response/recovery time under 50%. (f) Current values at 10 K temperature differences. (g) The variation of S_e under different bending conditions. (h) Stability and durability test over successive 500 cycles.

3.5 Deep-learning-enabled thermogalvanic hydrogel array for mental monitoring

Facial expression encodes extensive emotional and cognitive information, driven by facial muscle groups that generate diverse movements. The air-permeable thermogalvanic hydrogel array converts facial skin strain into electrical signals, enabling long-term wearing while preserving natural expression dynamics (Fig. 5a and S18†). Moreover, the ratio of positive to negative expressions exhibits a strong correlation with mental health, serving as an indicator of emotional well-being.^33–36 To effectively detect the signals for expression, the array is attached to the forehead, glabella, eyes, lips, and chin, based on the previous reports (Fig. 5b and S19a†).³² The multichannel current signals are acquired from the hydrogel array and wirelessly transferred to the computer for signal processing and feature extraction (Fig. 5c and S19b†). All expression data are calibrated and normalized to ensure that the features extracted are stable against motion artifacts. A deep learning method (Resnet-34) is subsequently leveraged for expression classification, and further mental health evaluation is conducted based on the positive ratio.

Fig. 5 Deep-learning-assisted mental monitoring. (a) Schematic of encapsulation of the hydrogel. (b) Schematic diagram of five-channel sensor assembled on the facial area. (c) System flow chart of data preprocessing, feature extraction, training and interaction. (d) Electrical signals of the PCPG-based corresponding to six expressions from five channels and (e) corresponding to confusion map. (f) The overall identification accuracy for each subject. (g and h) Monitoring volunteers with varying mental conditions over an extended period. (i) Flowchart of mental monitoring. (j and k) The real-time prediction interface for two volunteers by the hydrogel array.

As shown in Fig. 5d and S20,† the multichannel signals of six facial expressions are collected and distinguishable patterns are observed in different expressions. A dataset of 3000 patterns is obtained from six expressions, which is divided into 75% for the training set and 25% for the testing set. Thanks to the well-trained deep learning model, the recognition accuracy of testing set reaches 100% (Fig. 5e and S21†). Furthermore, the model can adapt to new expressions from new individuals and overall accuracies of over 98% are obtained (Fig. 5f). In addition to the previously demonstrated expression recognition in a short time, the breathable hydrogel array is also able to continuously capture facial muscle movement for 8 h, providing the support of the accurate mental health assessment. Compared to volunteers with a healthy mental state, there are more negative expressions present for the individual with a poor mental state (Fig. 5g and h). Therefore, the positive ratio (i.e., the ratio of positive and negative expression numbers) is set as 2.9, a threshold derived from existing literature, for evaluating the individual of mental state (Fig. 5i).^35,37 As a proof of concept, a display interface has been prepared to demonstrate the real-time mental monitoring based on facial expression recognition. When the volunteers with healthy and unhealthy mental state wear the multichannel hydrogel array, the collected signals are transmitted to the computer and input the pretrained deep learning model for expression classification monitoring. Under the prolonged monitoring, the mental health status is judged based on the positive ratio (Fig. 5j and k), demonstrating its huge potential in the field of wearable medical healthcare.

4. Conclusion

In conclusion, this study introduces a novel breathable thermogalvanic hydrogel array for continuous, real-time mental monitoring based on facial expression recognition. The hydrogel overcomes key challenges of existing wearable sensors, including poor breathability, limited electromechanical performance, and insufficient environmental stability. By incorporating the through-holes structure design, the hydrogel with high air-permeability enables long-term skin attachment without any skin irritation. The balance of high conductivity (34.4 mS cm⁻¹) and mechanical strength (1.07 MPa) is achieved by introducing the double network and phytic acid with a great number of hydrogen ions. Meanwhile, the hydrogen networks of water molecules are disrupted in the presence of PA and Gly, ensuring the reliable long-term operation for 10 days. With the assistance of deep learning algorithm, the hydrogel array enables the accurate recognition of six facial expressions with 100% accuracy. Finally, the platform of continuous mental monitoring is developed by evaluating the positive ratio. This study provides opportunities for developing a new mental monitoring platform based on facial expression recognition for non-invasive psychological health assessment and human–machine interfaces.

Data availability

The data supporting this article have been included as part of the ESI.†

Author contributions

Yu Li: conceptualization, writing – original draft. Ning Li: methodology, investigation and writing. Xinru Zhang: conceptualization and supervision. Jie Zhang: software, visualization. Lei Sun: data curation and writing. Zhiquan Huang: project administration. Hulin Zhang: conceptualization, writing – review & editing, supervision and funding acquisition.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

This work is supported by the National Natural Science Foundation of China (52475600) and Special Project of Science and Technology Cooperation and Exchange of Shanxi Province (202304041101021).

References

1. X. J. Du, H. Wang, Y. F. Wang, Z. Q. Cao, L. Y. Yang, X. H. Shi, X. X. Zhang, C. Z. He, X. D. Gu and N. Liu, Adv. Mater., 2024, 36, 2403411.
2. S. Kotz, S. Argaud, S. Delplanque, J.-F. Houvenaghel, M. Auffret, J. Duprez, M. Vérin, D. Grandjean and P. Sauleau, PLoS One, 2016, 11, e0160329.
3. J. C. Peters and C. Kemner, Biol. Psychol., 2017, 129, 1–7.
4. E. Carretier, M. Bastide, J. Lachal and M. R. Moro, Eur. Child Adolesc. Psychiatry, 2023, 32, 963–973.
5. R. Stelmach, E. L. Kocher, I. Kataria, A. M. Jackson-Morris, S. Saxena and R. Nugent, BMJ Glob. Health, 2022, 7, e007759.
6. T. Hassan, D. Seuss, J. Wollenberg, K. Weitz, M. Kunz, S. Lautenbacher, J. U. Garbas and U. Schmid, IEEE Trans. Pattern Anal. Mach. Intell., 2021, 43, 1815–1831.
7. Y. X. Ma, Y. X. Hao, M. Chen, J. C. Chen, P. Lu and A. Kosir, Inf. Fusion, 2019, 46, 184–192.
8. Y. M. Wang, H. Yu, W. H. Gao, Y. F. Xia and C. Nduka, IEEE Trans. Affect. Comput., 2024, 15, 606–619.
9. M. R. Ali, T. Myers, E. Wagner, H. Ratnu, E. R. Dorsey and E. Hoque, NPJ Digit. Med., 2023, 6, 20.
10. Y. Rahulamathavan and M. Rajarajan, IEEE Trans. Dependable Secure Comput., 2017, 14, 326–338.
11. P. A. Sanipatín-Díaz, P. D. Rosero-Montalvo and W. Hernandez, Sensors, 2024, 24, 3350.
12. Y. X. Wang, S. Qiu, D. Li, C. D. Du, B. L. Lu and H. G. He, IEEE/CAA J. Autom. Sinica, 2022, 9, 1612–1626.
13. Y. M. Chen, Z. L. Yang and J. P. Wang, Neurocomputing, 2016, 177, 671.
14. V. M. Hsu, A. M. Wes, Y. Tahiri, J. Cornman-Homonoff and I. Percec, Plast. Reconstr. Surg., 2014, 2, e211.
15. S. Rose, D. W. Xie, J. E. Streim, Q. Pan, P. L. Kwong and M. G. Stineman, BMC Health Serv. Res., 2016, 16, 537.
16. F. A. Torvik, E. Ystrom, K. Gustavson, T. H. Rosenström, J. G. Bramness, N. Gillespie, S. H. Aggen, K. S. Kendler and T. Reichborn-Kjennerud, Acta Psychiatr. Scand., 2018, 137, 54–64.
17. N. A. Ramli, A. N. Nordin and N. Z. Azlan, Microelectron. Eng., 2020, 234, 111440.
18. S. Yoo, T. Y. Yang, M. Park, H. Jeong, Y. J. Lee, D. W. Cho, J. Kim, S. S. Kwak, J. Shin, Y. Park, Y. Wang, N. Miljkovic, W. P. King and J. A. Rogers, Nat. Commun., 2023, 14, 1024.
19. H. L. Jia, Y. Y. Gao, J. K. Zhou, J. Li, C. K. Yiu, W. Park, Z. H. Yang and X. E. Yu, Nano Energy, 2024, 127, 109796.
20. X. Cui, Y. Nie, S. A. Khan, X. Bo, N. Li, X. Yang, D. Wang, R. Cheng, Z. Yuan and H. Zhang, ACS Sens., 2025, 10, 537–544.
21. J. P. Lee, H. Jang, Y. Jang, H. Song, S. Lee, P. S. Lee and J. Kim, Nat. Commun., 2024, 15, 530.
22. J. Q. Gao, H. S. Niu, Y. Y. Li and Y. Li, Adv. Funct. Mater., 2024, 34, 2418463.
23. R. A. Kishore, A. Nozariasbmarz, B. Poudel, M. Sanghadasa and S. Priya, Nat. Commun., 2019, 10, 1765.
24. L. L. Li, S. F. Zhao, W. H. Ran, Z. X. Li, Y. X. Yan, B. W. Zhong, Z. Lou, L. L. Wang and G. Z. Shen, Nat. Commun., 2022, 13, 5975.
25. W. J. Tang, S. Tang, C. J. Zhang, Q. T. Ma, Q. Xiang, Y. W. Yang and J. Y. Luo, Adv. Energy Mater., 2018, 8, 1800866.
26. H. S. Niu, H. Li, S. Gao, Y. Li, X. Wei, Y. K. Chen, W. J. Yue, W. J. Zhou and G. Z. Shen, Adv. Mater., 2022, 34, 2202622.
27. S. Radoor, D. R. Kandel, K. Park, A. Jayakumar, J. Karayil and J. Lee, Chemosphere, 2024, 350, 140990.
28. H. C. Wang, R. Z. Shang, J. W. Chen, X. Y. Jin, K. L. Chen, B. Huang, H. M. Chen and Q. L. Lu, Nano Energy, 2024, 128, 109843.
29. Y. Yang, Y. M. Ni, H. C. Wang, L. J. Chen, T. X. Zhu, Y. H. Zheng, Y. Cheng, Y. K. Lai, Y. X. Tang, W. L. Cai and J. Y. Huang, Chem. Eng. J., 2024, 482, 148847.
30. L. Han, K. Z. Liu, M. H. Wang, K. F. Wang, L. M. Fang, H. T. Chen, J. Zhou and X. Lu, Adv. Funct. Mater., 2018, 28, 1704195.
31. X. B. Li, X. Xiao, C. H. Bai, M. Mayer, X. J. Cui, K. Lin, Y. H. Li, H. L. Zhang and J. Chen, J. Mater. Chem. C, 2022, 10, 13789–13796.
32. M. Su, F. Y. Li, S. R. Chen, Z. D. Huang, M. Qin, W. B. Li, X. Y. Zhang and Y. L. Song, Adv. Mater., 2016, 28, 1369–1374.
33. S. An, L.-J. Ji, M. Marks and Z. Zhang, Front. Psychol., 2017, 8, 610.
34. Y. Hong, J.-H. Huang and J. Zhang, Front. Psychol., 2022, 13, 707961.
35. P. P. Rusu and A. A. Colomeischi, Front. Psychol., 2020, 11, 1608.
36. M. Tsujimoto, T. Saito, Y. Matsuzaki and R. Kawashima, J. Happiness Stud., 2024, 25, 25.
37. B. L. Fredrickson and M. F. Losada, Am. Psychol., 2005, 60, 678–686.

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d5ta00253b

---