Shaojie
Zheng
,
Xu
Huang
,
Jijiang
Hu
and
Zhen
Yao
*
State Key Laboratory of Chemical Engineering, College of Chemical and Biological Engineering, Zhejiang University, Hangzhou 310030, Zhejiang, China. E-mail: yaozhen@zju.edu.cn
First published on 8th February 2024
Polypropylene in-reactor alloys present a complex structure influenced by diverse polymerization process parameters, posing challenges for traditional analysis methods in establishing a quantitative relationship between process conditions, alloy structures and mechanical properties. To address this issue, a series of polypropylene/poly(ethylene-co-propylene) alloys with varied structures were synthesized by gas-phase polymerization. Machine learning methods were employed to develop regression models for predicting flexural strength (FS), impact strength (IS) and rubber phase content. The importance of structure and process condition descriptors was further analysed to reveal the process–structure–property relationship. The FS and IS prediction models utilizing Extreme Gradient Boosting (XGB) algorithms achieved impressive R2 scores of 0.9846 and 0.9841, respectively. Notably, the significant contribution of the rubber phase content to FS and IS prediction was observed in the structure descriptors. Furthermore, process condition descriptors (flowrate and initial pressure) played crucial roles in rubber synthesis, thereby exerting a substantial impact on FS and IS. In light of the feature importance analysis, new experimental runs were designed to synthesize alloys with enhanced IS. The experimental results closely aligned with the model predictions (RMSE = 4.4751 for IS). This research provides a new approach to establish process–structure–property relationships for in-reactor alloys, providing a convenient method for designing experiments to attain desired material properties.
Machine learning methods excel at discovering the relationships among data points, making them valuable tools for handling nonlinear and multidimensional data.14 Consequently, machine learning has been widely used in structure determination, performance prediction and new material discovery.15,16 Bhowmik et al. employed Decision Tree and principal component analysis methods to successfully predict the specific heat capacity of polymers using various structural descriptors, including bonds, angles, atoms and molecular weights.17 Joo et al. developed prediction models for the physical properties of polypropylene composites, employing three machine learning methods: multiple linear regression, deep neural network and Random Forest. The study analysed the impact of components in the composite materials on model predictions, providing insights for developing new recipes to achieve specific physical properties in PP composites.18,19
Due to the prolonged polymerization time and characterization periods, the application of machine learning in polymer science and engineering has been limited, especially in smaller datasets.20 Nevertheless, certain machine learning algorithms continue to demonstrate good performance even with relatively small datasets. Cai et al. applied six machine learning techniques to model the relationship between the dynamic strength of 3D-printed PP-based composites and the associated materials and printing parameters.21 Despite having only 26 data points, the artificial neural network achieved the highest prediction accuracy, albeit with a trade-off in computational efficiency. Kamireddi et al. performed 13 microwave-assisted catalytic co-pyrolysis experiments of PP and polystyrene (PS) mixtures and used machine learning to estimate the impact of the PP and PS quantity on the oil yield, gas yield and pyrolysis time.22 Wu et al. conducted injection molding of isotactic polypropylene into 27 specimens, collecting data on the polymorphic structure and mechanical properties to develop four machine learning models.23 The XGB model has the highest prediction accuracy for polymorphic form contents and mechanical properties based on the various processing parameters. Their analysis highlighted that the injection pressure and shear rate charged a higher contribution to the polymorphic form, while the molding temperature played a crucial role in property prediction. Liu et al. conducted 30 polyoxymethylene gear durability tests and developed a machine learning model to assess fatigue reliability. This research offered valuable insights into the evaluation of polymer gear reliability under a limited dataset.24
Some studies employed algorithms to augment the original small raw dataset before modelling.25 Li et al. employed the nearest neighbour interpolation algorithm to expand a small dataset of only 23 data points and then established an XGB model to predict Akron abrasion of rubber.26 Through the analysis of feature importance, the Akron abrasion was found to be most strongly correlated with the elongation at break of polymer materials. In addition, Shen and Qian proposed a virtual sample generation method to create models for predicting the wear resistance of rubber materials.27 They expanded the training dataset, originally containing only 24 rubber composite samples, using a Gaussian mixture model. The results show that the algorithms can achieve high prediction accuracy even with a small sample size.
In this work, a series of polypropylene/poly(ethylene-co-propylene) in-reactor alloys were synthesized through homo-polymerization of propylene and copolymerization of propylene and ethylene. The alloy structure was systematically varied by adjusting the process conditions during the gas-phase copolymerization stage. Three machine learning algorithms, namely Decision Tree (DT), Random Forest (RF) and Extreme Gradient Boosting (XGB), were used to develop regression models for predicting mechanical properties based on the structural characteristics of the polymer alloy. The model with the best performance was further utilized to uncover the relationship between the process conditions and structure. The importance of structure and process condition descriptors was analysed. In light of the feature importance analysis, new experimental runs were designed to validate the model's accuracy.
The process conditions for the prepolymerization and homopolymerization stages were kept constant, while the process conditions for the gas-phase reaction were varied. These variations included changes in the ratio of ethylene (C2 in), propylene (C3 in), and hydrogen (H2 in) in the feed, as well as the flowrate of the mixture gas feed, the initial gas-phase copolymerization pressure (initial P), and the copolymerization duration (time). This variation allowed for the preparation of a series of polypropylene/ethylene–propylene copolymer alloys with different structures. The specific process conditions of gas phase polymerization are listed in Table 1 (for all datapoints, refer to ESI† Table S1). It's worth noting that “initial P” refers to the pressure in the reactor after the transfer of polypropylene particles to Reactor2, followed by the introduction of ethylene to raise the gas-phase reaction pressure to 1.4 MPa.
The crystallinity of polymer particles (Xc) was assessed via differential scanning calorimetry. Nuclear magnetic resonance and Fourier-transform infrared spectroscopy were employed to determine the content of ethylene units in the polymer chain. Additionally, the melt index (MI) of the polymer particles was measured using a melt flow indexer according to ISO 1133-1:2011. Polymer granules underwent extrusion and injection molding processes to yield specimens suitable for impact and flexural testing. The flexural strength (FS) of specimens was tested using a universal material testing machine according to ISO 178:2001, while the impact strength (IS) at room temperature was assessed using a pendulum impact tester according to ISO 180:2000.
The characterization data of the polymer alloy are shown in Table 2 (for all datapoints, refer to ESI† Table S2).
The coefficient of determination (R2) and the root mean square error (RMSE) were used as the evaluation metrics for assessing regression models. These evaluation criteria are calculated as the following equations.
Through comparative analysis, the most optimal model among these three was identified. In addition, an in-depth investigation into the interrelationship between the structure and properties of polypropylene alloys was conducted. Furthermore, employing the most suitable algorithm, machine learning models were developed to elucidate the correlations between process conditions and structural characteristics, as well as process conditions and mechanical properties. In this phase of research, the dataset was created using the process conditions (seen in Table S1†), polymer structure and mechanical properties. Input features included C2 in, C3 in, H2 in, initial P and time, while the rubber phase content or mechanical properties FS and IS were taken as the output variables.
After optimizing the hyperparameters, both the training and testing datasets were employed to assess the model's fitting performance and generalization capability. Additionally, the SHapley Additive exPlanations (SHAP) technique was applied to explicate the output of machine learning models, providing insights into the importance of individual input features in predicting a specific model outcome.33–35
Fig. 3 presents the R2 score of the three models on the training and testing data. It is obvious that the DT model presents typical overfitting, characterized by a lower R2 score for the testing set compared to the training set, particularly when predicting flexural strength.
This phenomenon could be attributed to the work process of the tree model, as illustrated in Fig. 4. This picture visualized the decision-making process of the DT model when predicting flexural strength. At the root node of the tree, the initial criterion is whether the value of crystallinity (Xc) is greater than 43.69, which divides the training dataset into two distinct subsets with significantly different flexural strengths. One subset contains 14 samples with a lower mean FS value of 15.38, while the other comprises 8 samples with a higher mean FS value of 25.54. This implies that higher crystallinity corresponds to higher polymer flexural strength. The DT model then continues to dichotomize the subset according to different ranges of polymer structure characteristics, following a strategy of minimizing differences in FS within the subset samples until the stopping condition is reached. The stopping condition is determined by hyperparameters, such as the maximum depth, which governs the growth of the decision tree. A greater depth leads to a more detailed classification, which may result in overfitting. The situation is exacerbated when the dataset is small in size.
The limitations associated with a single decision tree can be mitigated by using ensemble models. RF is a classic ensemble machine learning technique that combines multiple decision trees in parallel. These trees are created using bootstrapped subsets of the training data. The term “Random” refers to the practice of randomly sampling data with replacement when creating each tree, and during tree splitting, a random subset of features is considered for decision-making. Each tree is trained independently and the output of each tree is taken into account. For regression problems, the final output of the “Forest” is determined by averaging the output of all trees. Consequently, Random Forest is less prone to overfitting. As depicted in Fig. 3, compared to the DT model, the RF model displays a narrower difference in R2 score between the training and testing datasets.
XGB is also an ensemble model based on tree models, integrated using the boosting method. Different from the RF model, the XGB model creates multiple trees sequentially, as opposed to in parallel. Each tree in XGB is built based on the performance of all the preceding trees, aiming to minimize the prediction error through multiple iterations. This is achieved by adjusting the structure of the next tree using the gradient boosting algorithm. Fig. 5 displays three trees selected from the ensemble of 29 trees in the XGB model for predicting FS. Note that the values in each leaf of the next tree are generally lower than those in the previous tree, while the tree structure becomes more complex. The final output of XGB is determined by summing the predicted values for the corresponding leaf nodes across all trees. This implies that the predictions are continually adjusted as each tree is added. In addition, the XGB algorithm introduces regularization terms L1 and L2 into the model to mitigate the risk of overfitting, while reducing model complexity and consequently accelerating the solving process. XGB performed slightly better than RF with lower RMSE and a higher R2 score when predicting FS and IS.
The SHAP values for Run1 and Run2 in the XGB model predicting FS are visualized in Fig. 6. The SHAP values reflect the local contribution of each feature from the sample mean to the model's predicted value. Among all the features, Rubber has the highest absolute SHAP value, with a value of 3.58 in Fig. 6(a), while it registers a negative value of −2.25 in Fig. 6(b). The larger the absolute value of SHAP, the higher the importance of the feature. A positive value indicates that the feature has a positive impact on the outcome, while a negative value indicates that the feature has a negative impact on the outcome. This implies that the rubber phase content significantly influences the flexural strength for Run1 and Run2. Specifically, when the rubber phase content is at 16.23 wt%, it contributes positively to the flexural strength, leading to an increase. Conversely, when the rubber phase content is at 31.75 wt%, it contributes negatively, resulting in a decrease in flexural strength.
The average absolute SHAP values for the entire training dataset and the SHAP values for each individual data point are depicted in Fig. 7. In this figure, (a) and (b) correspond to FS, while (c) and (d) pertain to IS. When considering the entire training dataset, the average absolute SHAP values provide insights into the importance of each feature. In the context of flexural strength (Fig. 7(a)), it is evident that the rubber phase content has the most substantial impact. The features F1, F2, Xc, and Ethylene exhibit relatively similar effects, all of which are less influential than rubber. Conversely, both MI and F3 have minimal influence on flexural strength.
In Fig. 7(b), each dot represents the contribution of a specific feature to flexural strength. The colour of these dots corresponds to the values of the respective feature, with red indicating higher values and blue denoting lower values. Clearly, the rubber content has a significant impact on flexural strength, both at high and low levels. However, as the rubber content increases from low (blue dots) to high (red dots), its contribution to flexural strength changes from positive to negative. Additionally, when the dots for F1, F2, and Ethylene are red, and the dots for Xc are blue, the SHAP values are negative. This suggests that higher contents of ethylene–propylene random copolymer, ethylene–propylene segmented copolymer, and ethylene units in the polymer chain, as well as lower values of crystallinity, have a detrimental effect on the flexural strength.
For the impact strength, based on the SHAP value, Rubber is identified as the most influential feature, followed by MI, which has a slightly lower influence than Rubber. Next in order of influence are Xc, F1, F2, F3, and Ethylene features. In contrast to flexural strength, an increase in Rubber and a decrease in MI lead to an improvement in impact strength. As shown in Fig. 7(d), the SHAP values shift from negative to positive as the dots for Rubber transition from blue to red and the dots for MI from red to blue. Indeed, a low value of melt index, corresponding to a high positive SHAP value, indicates that a higher molecular weight is favourable for improving impact strength. An increase in Xc, resulting in a negative contribution, implies that high crystallinity in the polymer alloy has an adverse effect on the impact strength. Besides, higher contents of ethylene–propylene random copolymer, ethylene–propylene segmented copolymer, and ethylene units are advantageous for enhancing the impact strength of polymer alloys.
In Fig. 8(a), the mean absolute SHAP values of each process condition are depicted. It is evident that the initial pressure (initial P) and the feed flowrate of mixture gas have the highest SHAP values, followed by the ratio of ethylene (C2 in), copolymerization time (time), and the ratio of propylene (C3 in) and hydrogen (H2 in). This suggests that the pressure at the initial gas copolymerization stage and the feed flowrate during the copolymerization process have a significant impact on the synthesis of the rubber phase in the alloy. Additionally, the ratio of ethylene and propylene in the mixture gas feed and the reaction time also play important roles in determining the rubber phase content. In essence, the initial pressure also governs the ratio of ethylene and propylene in the gas reactor at the initial period of copolymerization. When the initial pressure is low, more ethylene needs to be initially fed into the reactor to reach the desired reaction pressure. This leads to a higher initial proportion of ethylene in the copolymerization reaction. Additionally, the feed flowrate impacts the rate at which gas concentrations are refreshed inside the reactor, with a higher feed flowrate resulting in faster updates of gas concentrations. These factors have significant implications for the synthesis of the rubber phase content in the alloy.
Fig. 8 (a) Mean absolute SHAP values of process conditions in XGB predicting the rubber phase content; (b) SHAP value for the initial P vs. initial P; (c) SHAP value for the flowrate vs. flowrate. |
In Fig. 8(b), the SHAP values for various individuals from the training dataset show how the importance attributed to the feature “initial P” changes as its value varies. The shadow represents the number of points in that interval. It indicates that when the initial pressure is at a moderate level, the SHAP value is high, while it decreases for both smaller and larger initial pressure values. This indicates that maintaining an appropriate initial pressure is beneficial for increasing the rubber phase content in the alloy. In Fig. 8(c), as the flowrate value increases, the SHAP value for the flowrate increases and then stabilizes, suggesting that a larger feed flowrate of mixture gas can enhance the rubber phase content.
Fig. 9 Parity plots of XGB using process conditions to predict two mechanical properties: (a) FS and (b) IS. |
The SHAP values for each process condition are presented in Fig. 10. It is clear that the flowrate and initial P have the most significant influence on both FS and IS. A low mixed gas feed flowrate and a higher initial pressure have a positive contribution to FS, while a larger mixed gas feed flowrate and a moderate initial pressure have proven beneficial for IS. These observed effects can be attributed to the changes induced by these specific process conditions, resulting in variation in the rubber phase content in the alloy and subsequently influencing the physical properties. This is consistent with the established correlation between the polymer structure and properties, and the impact of process conditions on structural characteristics, as previously elucidated.
To further validate the accuracy of the established model in predicting mechanical properties, five experimental runs were designed, each involving various process conditions. In light of the previous feature importance analysis, particular attention was paid to the variation of the flowrate. The flowrate was varied over a wide range with five different values to ensure that these experimental protocols are as distinct as possible from the training and testing data mentioned previously. In addition, the values of flowrate are all higher than 2.5 kg h−1 because a high flowrate is beneficial for impact strength, which is the primary goal of PP alloys. The XGB model was also employed to generate predicted values for these conditions. The predictions for FS and IS and the corresponding experimental values are listed in Table 3.
Entry | C2 in | C3 in | H2 in | Initial P (MPa) | Flowrate (kg h−1) | Time (min) | Predicted FS (MPa) | Experimental FS (MPa) | Predicted IS (kJ m−2) | Experimental IS (kJ m−2) |
---|---|---|---|---|---|---|---|---|---|---|
Run 28 | 0.398 | 0.595 | 0.007 | 1.12 | 2.93 | 60 | 17.08 | 15.70 | 52.69 | 45.00 |
Run 29 | 0.395 | 0.598 | 0.007 | 1.12 | 2.56 | 75 | 13.13 | 14.71 | 52.09 | 47.47 |
Run 30 | 0.396 | 0.599 | 0.005 | 1.12 | 2.61 | 75 | 14.86 | 14.39 | 55.95 | 52.22 |
Run 31 | 0.405 | 0.590 | 0.005 | 0.93 | 3.18 | 75 | 15.35 | 13.24 | 55.51 | 55.89 |
Run 32 | 0.403 | 0.592 | 0.005 | 0.93 | 3.66 | 75 | 15.35 | 15.86 | 55.51 | 53.14 |
The experimental results closely aligned with the model's prediction for both flexural strength and impact strength, yielding RMSE values of 1.3656 and 4.4751, respectively. Although these RMSE values are slightly elevated compared to the model's performance on the training and testing datasets, it is worth noting that the XGB model still delivers satisfactory prediction for the uncharted experimental conditions. Furthermore, all newly designed experimental protocols yielded satisfactory impact strength values.
Footnote |
† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d3re00504f |
This journal is © The Royal Society of Chemistry 2024 |