Jihyeon
Park
ab,
Sinwoo
Kang
ab and
Jaeyoung
Lee
*abc
aSchool of Earth Sciences and Environmental Engineering, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, South Korea. E-mail: jaeyoung@gist.ac.kr
bInternational Future Research Center of Chemical Energy Storage and Conversion Processes, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, South Korea
cErtl Center for Electrochemistry and Catalysis, Gwangju Institute of Science and Technology, 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, South Korea
First published on 15th July 2022
Symbolic regression (SR) is the most widely used machine learning (ML) tool for determining the governing equation from a given dataset. However, a major problem associated with SR is gaps in the results (missing results) when more mathematical operations are introduced. We applied deep symbolic regression (DSR) to a dense space of overpotential formulas to reveal the scaling relations of the Gibbs free energies of the key intermediate adsorbates during the oxygen evolution reaction (OER) on FeNi surfaces in alkaline media. The highest-ranked empirical equation f(x) generated from 40000
000 hidden equations by DSR predicted an optimized electrocatalyst ratio of Fe8.7
:
Ni91.3, which resulted in a minimum overpotential of 0.368 V in the water-splitting process. Our approach provides a new perspective for understanding nonlinear dynamics in the electrochemical processes of chemical-energy conversion and storage.
The conventional trial-and-error method for synthesizing catalysts with high activities has the advantage of providing experimental data for the developed materials and catalytic performances. Thus, meaningful perspectives can be gained from practical results. However, the trial-and-error method is a time-consuming process, expensive, and difficult to control precisely.19–21 In contrast, computational strategies such as density functional theory (DFT) are very useful for the rapid screening of an enormous number of catalyst candidates. However, since most DFT calculations are based on an ideal combination of absorbates and crystal structures, the actual experimental results are difficult to correlate with the various descriptors.22,23 Therefore, if the actual OER results and various descriptors could be associated from a computational standpoint, a more effective method can be derived.
However, it is challenging to consider both the interdependence of each step in the OER and the various descriptors simultaneously. In particular, when using symbolic regression (SR), which aims to elucidate the relationships between variables,24–26 the number of possible functions increases exponentially as the number of variables and the types of mathematical operations used in the SR increase. Such a process is time-consuming and can inadvertently cause the occurrence of gaps (missing results) in the list of generated formulas.
Deep symbolic regression (DSR) is an algorithm that combines deep learning (DL) with SR to overcome the shortcomings of SR.27 DL, which has the characteristic of improved performance as learning progresses, penetrates deep into the gaps between the formulas created by SR. Therefore, in DSR, the performance in expressing the target value gradually improves as the algorithm proceeds with the increased spatial density of the equations.
In this study, we first employed the DSR algorithm to find correlations between actual experimental results and physical/calculative properties through the DSR algorithm. FeNiOxHy has the highest activity among non-precious-metal catalysts, and seven samples of this catalyst were synthesized with different contents of Fe and Ni using electrodeposition. The catalytic activity data for FeNiOxHy (needed for the DSR algorithm) was obtained from our cyclic voltammetry (CV) tests on the seven catalysts, while the physical/calculative property data was obtained from other references.
ΔG1 = ΔGOH | (1) |
ΔG2 = ΔGO − ΔGOH | (2) |
ΔG3 = ΔGOOH − ΔGO | (3) |
ΔG4 = ΔGO2 − ΔGOOH | (4) |
Eqn (5) and (6) were used to calculate max(ΔGi) and the electrochemical step symmetric index (ESSI), respectively, as follows.43
![]() | (5) |
![]() | (6) |
dataset (Xi ∈ Rn, yi ∈ R), function f:Rn → R | (7) |
The equations generated by the DSR algorithm can be expressed as hierarchical symbolic trees. The tree branch designated “node”, as shown in Fig. S2,† has three types. Mathematical operations, including +, −, ×, and ÷, which require two variables, are represented by light-yellow nodes. In contrast, mathematical operations, including , which require one variable, are represented by sky-blue nodes. The terminals whose branches no longer extend are represented by gray nodes.
The input database goes through a recurrent neural network (RNN), and the DSR algorithm performs mathematical operations step-by-step until the equation is completed. Notably, the mathematical operations sampled at each step are not randomly emitted by independent trials but depend on the probability (p) of the mathematical operations at all previous steps. In other words, the probability is directly proportional to the success of each attempt as evaluated in the previous steps. That is, the ith mathematical expression (τi) can be expressed by the probability vector (ψ), including all previous expressions (τ1:i−1) and the RNN internal parameters (θ) (eqn (8)), and the probability of the function at that time is identical to the product of the probabilities of all steps (eqn (9)). Further details can be found in ref. 27.
![]() | (8) |
![]() | (9) |
However, because the functions have a hierarchical-tree structure, there are several restrictions in the equation expression stage of the DSR algorithm. First, the expression length of the mathematical equation must not be less than 4 or more than 30 parameters. If the length was less than 4, the equation would likely be too simple to be accurate, and if the length was more than 30, the equation would become too complex to interpret intuitively. The second restriction is that the children of the operator cannot all be constraints . If this was allowed, the results would still be constrained (e.g.,
). The third restriction is related to the operator, which requires one variable and states that a child operator should not be an inverse function of its parent (e.g.,
). The last restriction is that a trigonometric operator should not have a descendant because such expressions cannot be found in nature (e.g., sin(x + cos(x))).
Fig. 1a shows a structural diagram of the DSR algorithm, which generates an equation for expressing the overpotential at a specific current density of 10 mA cm−2. The preprocessed database with the physical and calculative properties is applied to the DSR algorithm, and the algorithm constantly attempts to create an equation expressing the overpotential with these properties. In this situation, the equation is created in a hierarchical process inside the DSR algorithm, which can be expressed as a tree structure (Fig. S2†). However, this is only a schematic explaining the structure of the algorithm, and in practice, a complete equation is produced by the algorithm, as shown in eqn (10). Therefore, it is difficult to determine how a completed equation is calculated and produced by the DSR algorithm, which is also a representative feature of deep learning.
![]() | (10) |
![]() | ||
Fig. 1 (a) Hierarchical internal algorithm structure of the deep symbolic regression (DSR) that generates the completed equation with three different node types: yellow (binary), sky-blue (unary), and gray (terminal). Database is DSR input and the empirical equation is DSR output; (b) precision score graph as a function of the number of learning steps (max score is the maximum precision score of 4000 different equations per step, and best score is the best precision score in all previous steps); and (c) reformatted graph based on the final best equation with a gray solid line corresponding to the y = x ± 0.1 graph (Ref. 1: 44, Ref. 2: 45,and Ref. 3: 46). |
The DSR algorithm produces 4000 different equations at each step and evaluates them simultaneously. The normalized root-mean-square error (NRMSE) of the overpotential is calculated using the empirical overpotential (ŷ) and actual overpotential (y) at a specific current density of 10 mA cm−2 was calculated and rated using eqn (S1) and (S2),† respectively. The closer the precision score is to 1, the higher the accuracy. The blue open circles in Fig. 1b represent the maximum score among the 4000 equations for each step, and the red solid line represents the best score in all the previous steps. As the number of learning steps increases, the maximum and best scores also increase, indicating that the RNN inside the DSR algorithm conducts reinforcement learning of the previous step and gradually improves the performance to generate empirical equations with increasing number of steps. It seems that if more time is consumed, better empirical equations with higher precision scores are created. Furthermore, when a parameter was excluded from eqn (10) to confirm the parameter effect, the NRMSE score increased, and the empirical equation became extremely complicated, with a low R2 value (Table S5†).
The top 50 equations with their scores are shown in the ESI.† The top-ranked empirical equation expressing the overpotential is the same as that shown in eqn (10), with an NRMSE score of 0.7201. To precisely show the correlation, the overpotential calculated using eqn (10) and the actual overpotential at 10 mA cm−2 are shown in Fig. 1c, with an added gray graph of y = x ± 0.1. The average and standard error values of the overpotentials for each Fe:
Ni ratio in FeNiOxHy in the database are represented by blue symbols and error bars, respectively. All the blue symbols are close to the y = x graph, indicating that the empirical equation was successful in describing the data. Furthermore, the top-ranked empirical equation also fits the results from additional experiments (pink symbols), as well as from other reference data (pea-green symbols), showing a high R2 of 0.9894. Moreover, the empirical equations obtained via the DSR algorithm were also reformatted and plotted at current densities of 1, 5, and 20 mA cm−2, which were then verified by additional experiments (Fig. S3†).
Since a deep learning-based algorithm was used, the top-ranked empirical equation expressing the overpotential at 10 mA cm−2, which is a representative method for OER performance, is difficult to understand intuitively. The equation includes only two calculative properties (max(ΔGi) and ESSI); therefore, it was unclear in which direction we should drive our research to further lower the overpotential. To evaluate this instinctively, the equation was expressed as a three-dimensional (3D) graph (Fig. S4†). Although the overpotential value oscillates owing to the trigonometric function, the graph indicates that we should seek to reduce both max(ΔGi) and ESSI. However, the minimum value of the overpotential is limited to e−1 = 0.368 V due to the synthesis function of exp and sin.
For an ideal OER catalyst, all adsorption energies (ΔGi) should be identical (1.23 eV), and the sum of ΔG2 and ΔG3 should be 2.46 eV (ΔG2+3 = ΔGOOH − ΔGOH). However, in the real world, the average value of ΔG2+3 in a large group of OER catalysts is usually 3.2 ± 0.2 eV, irrespective of the overpotential, as shown in Fig. 2.43–48 Interestingly, even if most of the OER overpotentials are affected by ΔG2 or ΔG3, the minimum value of the empirical equation to break the scaling relationship could still not break the “great wall of the scaling relationship”. It is impressive to observe the natural limiting overpotential in the actual OER environment determined via artificial intelligence (eqn (11)). Furthermore, this suggests that there is no correlation between breaking the scaling relation and lowering the overpotential.10,43
e−1 = 0.368 V ≈ (3.2–2.46) eV/2e− | (11) |
In the forward scan, an oxidation peak of Ni was present until the Fe content reached 25% (Fig. S5†). We only collected results for the database in the backward scan to avoid any interference from this oxidation peak. The current density at the same potential increases until the Fe content reaches 10% and decreases as the Fe content exceeds 10%. The best Fe10Ni90 electrode retained its low overpotential with an increase of only 2.51 μV per minute (i.e., a 0.00273% overpotential increase per minute) (Fig. 3a).
The slope range of the Tafel plot in the Fe range 5–20% is 45–50 mV dec−1, exhibiting a clear difference from that in the other ratios (Fig. 3b). Therefore, to reduce the overpotential in a practical water electrolysis cell/stack, it is not necessary to use a catalyst with a specific ratio such as the ideal 8.7% as determined from Fig. 4. Instead, any FeNiOxHy catalyst with an Fe content between 5% and 20% with respect to Ni should be sufficient.
From the CV cycles of the mixed ratios of FeNiOxHy (Fig. S5†), two features were extracted as a function of Fe content: one for the previously mentioned overpotential at a current density of 10 mA cm−2, and the other for the specific current density at an overpotential of 350 mV (Fig. 4). Thus, as the Fe content is increased in the range 5–20%, the overpotential decreases and then increases, and the specific current density increases and then decreases, covering the local minimum and maximum of overpotential and current density, respectively. This result is comparable to recent works in which FexNi1−x (0 < x < 0.25) is mainly considered to be at the maximized activity.18,49
The measured overpotential ranges from 0.3 to 0.35 V, which is lower than those in the other ranges, and the measured specific current density is 4–9 times higher than those in the other ranges (also see Fig. S6†). To determine the ideal ratio, we used the DSR algorithm again, but in terms of specific current density. Notably, the variables and target values for the algorithm were the Fe content and specific current density (at 350 mV), respectively. The detailed conditions and equations are listed in Table S1 and eqn (S3),† respectively. From the differential value of the empirical equation, the ideal Fe content was calculated to be 8.7% (gold star in Fig. 4 and eqn (S4)†), which is close to that obtained by density functional theory (DFT) calculations, as reported by the Goddard III group.38 They confirmed that the overpotential for the OER was higher when the bulk Fe content in FeNi was approximately 1/9.
Footnote |
† Electronic supplementary information (ESI) available. See https://doi.org/10.1039/d2ta02594a |
This journal is © The Royal Society of Chemistry 2022 |