From the journal Environmental Science: Atmospheres Peer review history

06-Aug-2022

Dear Dr Bryant:

Manuscript ID: EA-ART-06-2022-000074
TITLE: Overcoming the lack of authentic standards for the quantification of biogenic secondary organic aerosol markers

Thank you for your submission to Environmental Science: Atmospheres, published by the Royal Society of Chemistry. I sent your manuscript to reviewers and I have now received their reports which are copied below.

I have carefully evaluated your manuscript and the reviewers’ reports, and the reports indicate that major revisions are necessary.

Please submit a revised manuscript which addresses all of the reviewers’ comments. Further peer review of your revised manuscript may be needed. When you submit your revised manuscript please include a point by point response to the reviewers’ comments and highlight the changes you have made. Full details of the files you need to submit are listed at the end of this email.

Please submit your revised manuscript as soon as possible using this link:

*** PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm. ***

https://mc.manuscriptcentral.com/esatmos?link_removed

(This link goes straight to your account, without the need to log on to the system. For your account security you should not share this link with others.)

Alternatively, you can login to your account (https://mc.manuscriptcentral.com/esatmos) where you will need your case-sensitive USER ID and password.

You should submit your revised manuscript as soon as possible; please note you will receive a series of automatic reminders. If your revisions will take a significant length of time, please contact me. If I do not hear from you, I may withdraw your manuscript from consideration and you will have to resubmit. Any resubmission will receive a new submission date.

The Royal Society of Chemistry requires all submitting authors to provide their ORCID iD when they submit a revised manuscript. This is quick and easy to do as part of the revised manuscript submission process. We will publish this information with the article, and you may choose to have your ORCID record updated automatically with details of the publication.

Please also encourage your co-authors to sign up for their own ORCID account and associate it with their account on our manuscript submission system. For further information see: https://www.rsc.org/journals-books-databases/journal-authors-reviewers/processes-policies/#attribution-id

Environmental Science: Atmospheres strongly encourages authors of research articles to include an ‘Author contributions’ section in their manuscript, for publication in the final article. This should appear immediately above the ‘Conflict of interest’ and ‘Acknowledgement’ sections. I strongly recommend you use CRediT (the Contributor Roles Taxonomy from CASRAI, https://casrai.org/credit/) for standardised contribution descriptions. All authors should have agreed to their individual contributions ahead of submission and these should accurately reflect contributions to the work. Please refer to our general author guidelines http://www.rsc.org/journals-books-databases/journal-authors-reviewers/author-responsibilities/ for more information.

I look forward to receiving your revised manuscript.

Yours sincerely,
Dr Claudia Mohr

Associate Editor, Environmental Science: Atmospheres

************

Reviewer 1

Quantifying individual compounds in complex mixtures without authentic standards is an ongoing challenge in organic aerosol research. Bryant et al. developed a machine learning based approach to overcome this challenge, which would provide relative ionisation efficiencies for quantifying a large range of organic compounds with HPLC-ESI-MS using cis-pinonic acid as a surrogate standard. I find this work of high quality and it is definitely relevant for the audience of Environmental Science Atmospheres. The paper is generally well written but requires a few clarifications/additions before it can be accepted.

- Section 2.1. Please add capillary and auxiliary gas flow rates, and the applied voltage at the ESI capillary.
- Please update Table S1 to include information on the authentic standards such as manufacturer and purity.
- Section 2.2, line 3. Standards were prepared in 50:50 MeOH:H2O. Did you check that the higher methanol content in the samples does not affect peak shapes at low retention times, when the eluent composition is 90:10 instead?
- Section 2.4 and 2.5. Please provide information on filter material and brand, and any pre-cleaning or pre-conditioning/post-conditioning.
- Section 2.4. Please provide more details on the filter extraction procedure.
- Section 3.1, line 5. How does the retention time of malic acid compare with the dead time of your chromatographic method?
- Section 3.1, when discussing retention times of the authentic standards please add the information on cis-pinonic acid given that it is the surrogate standard of choice for all others.
- Please add the list of descriptors to the supporting information.
- Please describe the principle of “regularised random forest” and how it differs from “random forest”.
- Please add a discussion on which descriptors matter the most to calculate the RIE.
- Please discuss how the retention time impacts RIE.
- Caption of Figure 1, “linear model” is mentioned but I thought that “regularised random forest” was the chosen model. Please explain.
- Pinic acid synthesis:
Line 9 micture -> mixture.
Please explain how phase separation was achieved after the synthesis. Are there two separate phases at the end of the synthesis? This is not clear from the text.
Please provide information on “brine” composition.
Please provide more information on the “reduced pressure” evaporation. Was it conducted in a rotavap?
Please provide more information on the flash chromatography.
Please explain how you obtained a purity value for pinic acid of 75% and what the impurities are.
Please write calcd in full.

Reviewer 2

Bryant et al. address the problem of quantifying biogenic secondary organic aerosol markers detected with LC/HRMS while authentic standards are often lacking. The paper expands on the existing machine learning methods for predicting the ionization efficiency of the markers detected with LC/HRMS. While the methods of measurement and machine learning have been suggested previously, the application to organic aerosols is novel. To achieve this the authors have measured response factors for 54 new chemicals and retrained the random forest regression to predict ionization efficiency and applied the modeling to a case study.

The paper is a good starting point for implementing a more rigorous quantification of the detected compounds in organic aerosols; however, to facilitate this, the transparency/rigor of the methodology as well as the availability of the model and data should be improved. The methods and conclusions are relevant, though some confusion with the methods and the unavailability of the model hamper fully assessing the reliability of the conclusions. Both of these shortcomings are straightforward to fix during revision; therefore, I suggest accepting the manuscript after major revision.

Specific comments:

For the eq. 1 it should be mentioned that slope needs to be calculated from the peak area accounting for all isotopic peaks or the peak area of the base peak needs to be corrected with the theoretical isotope correction factor. It is also unclear if the authors used this in their computations. In the case of the chemicals used in this study, the differences from correction/no correction will not be enormous but could reach 20-30%.

The Materials and Methods part does not contain any details on the descriptor calculation or machine learning. There are some details given in the first two paragraphs of the results and Discussion; however, it would improve the readability if the details are given together in Materials and Methods.

“pKa showed a moderate correlation of R = 0.32 towards logIE” Should it be R2?

The authors claim to have used LOOCV due to a low number of chemicals. Actually, the number of chemicals is not so low and 10% or even ¼ could be left out from modeling for proper validation. It is now unclear how the stated RMSE and R2 values were obtained. Presumably, the predicted values given in Table S1 and Figure 1 correspond to the predicted values when each of the chemicals was left out from the training? Or was LOOCV used only for optimizing the hyperparameters with caret? Nevertheless, the same data should not be used for hyperparameter optimization and assessing the performance of the model as hyperparamter optimization particularly searches for values that yield the best fit in LOOCV conditions. This is quite a critical point, as the lack of a separate validation set might be overly optimistic by orders of magnitude (data scientists call it usually a test set but in chemistry, a validation set is more common). Please clarify.

The authors state that their group has previously already measured 51 chemicals (35 commons with this study). It is unclear why do the authors not combine these datasets. There is no need to train complex models on small datasets if larger ones are available. It would even be beneficial to build on the already published big dataset or retrain previously trained models. Please consider expanding and/or discussing in the manuscript.

Analysis of the obtained machine learning model could be expanded. Which descriptors were important in the model and how does it agree with the previous findings? If the space is limited, this information could be brought in SI.

It is also unclear how is the average RIE obtained. Is it a geometric mean of RIE values or is it the mean of these? As the values range over ca 4 orders of magnitude, a simple mean is dominated by compounds with high RIE. Also, the aim of using an average RIE is unclear. As the standard deviation of such an average is much larger than the value itself (in factors), it does not represent well any of the compounds. Please reconsider these calculations.

Two pairs of the reference, (14 and 33) as well as (46 and 53) are the same.

Please provide Table S1 in a shareable format such as in Zenodo or at least as an excel file. Also, please add InChI and/or SMILES to improve the usability of the data. Additionally, the benefit for the community would be much higher if the model could be shared as well.

We would like to thank the reviewers for their comments. Replies to each individual comment are given below.

Upon further reflection, we have now changed relative ionisation efficiency (RIE) to response factor (RF) to be in keeping with current terminology in recent papers: https://www.mdpi.com/1420-3049/27/3/1013/htm

A sentence has also been amended in the conclusion, to now read:
Further work is needed to develop this method to predict RF’s without the need of structure elucidation and expand to include newly synthesised organic compounds and the range of functional groups and gas phase precursors.

Reviewer 1

Section 2.1. Please add capillary and auxiliary gas flow rates, and the applied voltage at the ESI capillary.
Additional text has been added:
The sheath and auxiliary gas flow rates were 45 arb. and 10 arb respectively. The spray voltage was set to 4 kV.

Please update Table S1 to include information on the authentic standards such as manufacturer and purity.

All standards had a purity of 95 % or higher. Some standards were borrowed from different groups within the chemistry department. As such, manufacturer and exact purity are not available for all standards. Table S1 has been updated to include purity and manufacturer where available. Where exact purity is not available, > 95 % has been added, and the manufacturer left blank.

Section 2.2 line 3 Standards were prepared in 50:50 MeOH:H2O. Did you check that the higher methanol content in the samples does not affect peak shapes at low retention times, when the eluent composition is 90:10 instead?

The peak shapes for early eluting species was checked by running standards at 90:10 (H2O:MeOH), with no appreciable difference. 50:50 MeOH:H2O was chosen due to the high amounts of semi-polar species within the ambient samples.

Section 2.4 and 2.5. Please provide information on filter material and brand, and any pre-cleaning or pre-conditioning/post-conditioning.
Additional text has been added:

Quartz filters (Whatman QMA, 10” by 8”) were pre-baked at 500 oC for 5 hours and wrapped in foil before use.
Section 2.4. Please provide more details on the filter extraction procedure.

It is not clear what additional information the reviewer would like to be included. The current manuscript contains a full description of the process as outlined below.
A 38.44 cm2 cutting was taken from the filter and cut into roughly 1 cm2 pieces. 8 mL of MeOH (Optima LC-MS grade) was then added to the filter pieces and sonicated for 45 mins under ice. The extract was then removed and filtered through a 0.22 μm syringe filter (Millipore) into a new vial. 2 mL (2 * 1mL) of MeOH was then added to the filter pieces and extracted through the 0.22 μm filter and combined with the rest of the extract. The combined extract was then reduced to near dryness using a solvent evaporator, before being reconstituted in 50: 50 MeOH:H2O. Triplicate recovery tests showed an almost complete recovery of cis-pinonic acid (99 ± 15.6 %, n =3) from the filter.

Section 3.1, line 5. How does the retention time of malic acid compare with the dead time of your chromatographic method?
The deadtime of our method is around 0.5 min, meaning malic acid elutes just after.

Section 3.1, when discussing retention times of the authentic standards please add the information on cis-pinonic acid given that it is the surrogate standard of choice for all others.

Additional text has been added:
cis-pinonic acid, the reference compound eluted around 8 minutes.

Please add the list of descriptors to the supporting information.

A table of descriptors has been added to the supplementary (Table S2).

Please describe the principle of “regularised random forest” and how it differs from “random forest”.

Additional text has been added:
Regularised random forest models work in the same way as random forest models but reduce model complexity by disregarding features that share information.

Please add a discussion on which descriptors matter the most to calculate the RIE.

Additional text has been added:
As shown in Table 2, the 18 descriptors for model development were those of structural descriptors surrounding acidity and polarisation. Of the 18 descriptors, the most influential descriptors were MLFER_A and SpMAD_Dzp. MLFER_A provides a description of the overall solute hydrogen bond acidity and SpMAD_Dzp is a measure of a compound’s polarizability. These specific descriptors were not identified as important in Liigand et al., 2020, but other descriptors for acidity/basicity were.

Please discuss how the retention time impacts RIE.
Rather surprisingly, we found no correlation between RIE and retention time. The discussion of the average RIEs at different RT zones was for completeness. We cannot comment further on the effects of RT on RIE.

Caption of Figure 1, “linear model” is mentioned but I thought that “regularised random forest” was the chosen model. Please explain.

This is a typo. The figure caption has been changed to:
Figure 1. Comparison between measured logRIE (logRIEM) and predicted logRIE (logRIEp) produced by a RRF model. All logRIEM and logRIEP values are given in table S1 for the standards used in this study. The solid black line is 1:1 i.e would represent perfect predictions of the measured values. The blue dotted lines represent 2 x RMSE from the 1:1 line. The grey vertical lines represent predicted logRIE ± RMSE.

Pinic acid synthesis:
Line 9 micture -> mixture.
Changed

Please explain how phase separation was achieved after the synthesis. Are there two separate phases at the end of the synthesis? This is not clear from the text.

Additional text added:
The reaction mixture was washed with CH2Cl2 (3 x 100 mL) to removed any organic compounds (1,4-dioxane and water are miscible and therefore stay in the same phase). The CH2Cl2 organic layer was discarded and the remaining aqueous layer was acidified to pH1 using concentrated HCl (37%).

Please provide information on “brine” composition.

Additional text added:
Brine was produced by adding NaCl to deionised water until no further solids cold be dissolved. Solid NaCl is left in the bottom of the Winchester in order to maintain the saturation.

Please provide more information on the “reduced pressure” evaporation. Was it conducted in a rotavap?
Additional text added:
via rotary evaporator

Please provide more information on the flash chromatography.

Sufficient information is already provided for the flash chromatography in line with organic synthesis papers.

Please explain how you obtained a purity value for pinic acid of 75% and what the impurities are.

This was a writing error. The reaction yield was 75 % as stated in the synthesis procedure, however the purity was stated to be 75 % in the main text. The pinic acid purity is at least 99 % based off the elemental composition provided. This has now been updated throughout the manuscript.

text added: Purity > 99 %.

Please write calcd in full.
The elemental composition is provided in full; we do not feel that a detailed calculation is required.

reviewer 2

For the eq. 1 it should be mentioned that slope needs to be calculated from the peak area accounting for all isotopic peaks or the peak area of the base peak needs to be corrected with the theoretical isotope correction factor. It is also unclear if the authors used this in their computations. In the case of the chemicals used in this study, the differences from correction/no correction will not be enormous but could reach 20-30%.

This was included in the Tracefinder methodology.

The Materials and Methods part does not contain any details on the descriptor calculation or machine learning. There are some details given in the first two paragraphs of the results and Discussion; however, it would improve the readability if the details are given together in Materials and Methods.

This has been moved to the methodology.

pKa showed a moderate correlation of R = 0.32 towards logIE” Should it be R2?

The text is correct, the correlation between logIE and pKa was 0.32.

The authors claim to have used LOOCV due to a low number of chemicals. Actually, the number of chemicals is not so low and 10% or even ¼ could be left out from modeling for proper validation. It is now unclear how the stated RMSE and R2 values were obtained. Presumably, the predicted values given in Table S1 and Figure 1 correspond to the predicted values when each of the chemicals was left out from the training? Or was LOOCV used only for optimizing the hyperparameters with caret? Nevertheless, the same data should not be used for hyperparameter optimization and assessing the performance of the model as hyperparamter optimization particularly searches for values that yield the best fit in LOOCV conditions. This is quite a critical point, as the lack of a separate validation set might be overly optimistic by orders of magnitude (data scientists call it usually a test set but in chemistry, a validation set is more common). Please clarify.

We clarify that the reviewer is right in that the values given in Table S1 and Figure 1 correspond to the predicted values for each test compound where all of the other compounds were used to train the model each time. Given the large number of potential molecules in complex ambient conditions, and the diversity of organic compounds we wish to sample, we did not initially consider the number of compounds to be 'large'. Whilst this can be subjective, and should be informed by the modelling evaluation, we felt the LOOCV approach allowed us to more easily interpret the relationship between model performance and diversity of compounds we have in this study. Indeed, we found minimal variability in RMSE and R2 using LOOCV suggesting minimal bias due to an imbalanced distribution of compound types.

The authors state that their group has previously already measured 51 chemicals (35 commons with this study). It is unclear why do the authors not combine these datasets. There is no need to train complex models on small datasets if larger ones are available. It would even be beneficial to build on the already published big dataset or retrain previously trained models. Please consider expanding and/or discussing in the manuscript.

These datasets could not be combined due to the difference in methodologies. Our previous paper used direct injection rather the LC. As such, the datasets could not be combined. However, a high correlation between the RIE’s measured by the different methods was observed.

Analysis of the obtained machine learning model could be expanded. Which descriptors were important in the model and how does it agree with the previous findings? If the space is limited, this information could be brought in SI.
Additional text has been added:

As shown in Table 2, the 18 descriptors for model development were those of structural descriptors surrounding acidity and polarisation. Of the 18 descriptors, the most influential descriptors were MLFER_A and SpMAD_Dzp. MLFER_A provides a description of the overall solute hydrogen bond acidity and SpMAD_Dzp is a measure of a compound’s polarizability. These specific descriptors were not identified as important in Liigand et al., 2020, but other descriptors for acidity/basicity were.

It is also unclear how is the average RIE obtained. Is it a geometric mean of RIE values or is it the mean of these? As the values range over ca 4 orders of magnitude, a simple mean is dominated by compounds with high RIE. Also, the aim of using an average RIE is unclear. As the standard deviation of such an average is much larger than the value itself (in factors), it does not represent well any of the compounds. Please reconsider these calculations.

We have considered the reviewers comments and agree that a mean RIE value may not give a good representation. We calculated a simple geometric mean here and so it will be influenced significantly by the species with high RIE. One of the reasons for this section of the paper was to compare the difference between using a single standard response factor estimated as an average of all compounds with using only cis-pinonic acid. Here we show that cis-pinonic acid is one of the most poorly ionising species and thus in its previous use as a proxy standard in the literature is likely to overestimate the concentration of many species.
We have removed the discussion of the use of an average RF factor to correct concentrations. We have now included a discussion of this in the conclusion. The following text has been added:

A geometric mean average RF value was calculated to be 4.2 ± 3.9, highlighting the large variability in the predicted RFs, and therefore its lack of reliability if used as a generalised RF. We feel it is important to highlight the issues when assuming a single response factor, whether that be from cis-pinonic acid, or a “corrected” RF.

Two pairs of the reference, (14 and 33) as well as (46 and 53) are the same.

This has been amended

Please provide Table S1 in a shareable format such as in Zenodo or at least as an excel file. Also, please add InChI and/or SMILES to improve the usability of the data. Additionally, the benefit for the community would be much higher if the model could be shared as well.
The SMILE formula has been added to Table S1, and the excel file has been made available for download.


The model code has been added to GitHub. Additional text has been added to the manuscript.
Additional text has been added:
Full details of the model development and the dataset containing the predicted descriptors can be found at https://github.com/djb96/Response_factor_model.

27-Oct-2022

Dear Dr Bryant:

Manuscript ID: EA-ART-06-2022-000074.R1
TITLE: Overcoming the lack of authentic standards for the quantification of biogenic secondary organic aerosol markers

Thank you for your submission to Environmental Science: Atmospheres, published by the Royal Society of Chemistry. I sent your manuscript to reviewers and I have now received their reports which are copied below.

After careful evaluation of your manuscript and the reviewers’ reports, I will be pleased to accept your manuscript for publication after revisions.

Please revise your manuscript to fully address the reviewers’ comments. When you submit your revised manuscript please include a point by point response to the reviewers’ comments and highlight the changes you have made. Full details of the files you need to submit are listed at the end of this email.

Please submit your revised manuscript as soon as possible using this link :

*** PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm. ***

https://mc.manuscriptcentral.com/esatmos?link_removed

(This link goes straight to your account, without the need to log in to the system. For your account security you should not share this link with others.)

Alternatively, you can login to your account (https://mc.manuscriptcentral.com/esatmos) where you will need your case-sensitive USER ID and password.

You should submit your revised manuscript as soon as possible; please note you will receive a series of automatic reminders. If your revisions will take a significant length of time, please contact me. If I do not hear from you, I may withdraw your manuscript from consideration and you will have to resubmit. Any resubmission will receive a new submission date.

The Royal Society of Chemistry requires all submitting authors to provide their ORCID iD when they submit a revised manuscript. This is quick and easy to do as part of the revised manuscript submission process. We will publish this information with the article, and you may choose to have your ORCID record updated automatically with details of the publication.

Please also encourage your co-authors to sign up for their own ORCID account and associate it with their account on our manuscript submission system. For further information see: https://www.rsc.org/journals-books-databases/journal-authors-reviewers/processes-policies/#attribution-id

Environmental Science: Atmospheres strongly encourages authors of research articles to include an ‘Author contributions’ section in their manuscript, for publication in the final article. This should appear immediately above the ‘Conflict of interest’ and ‘Acknowledgement’ sections. I strongly recommend you use CRediT (the Contributor Roles Taxonomy, https://credit.niso.org/) for standardised contribution descriptions. All authors should have agreed to their individual contributions ahead of submission and these should accurately reflect contributions to the work. Please refer to our general author guidelines https://www.rsc.org/journals-books-databases/author-and-reviewer-hub/authors-information/responsibilities/ for more information.

I look forward to receiving your revised manuscript.

Yours sincerely,
Dr Claudia Mohr

Associate Editor, Environmental Science: Atmospheres

************

Reviewer 1

The authors made almost all the requested changes suggested by both reviewers and I am happy to suggest the manuscript for publications. I have two very minor suggestions, clarifications from my previous suggestions:
- concerning the flash chromatography, given that the elution has been achieved through a gradient from 100% hexane to 40% EtOAc (this is what I understand from the text) it would be good to add a sentence to describe the elution gradient so that it can be reproduced by others
- concerning "calcd", my suggestion was to spell the word in full as this is not a defined abbreviation

Reviewer 2

The authors have address majority of the comments of the reviewers and the manuscript has improved; however, some of the comments regarding the computation of R2, RMSE values, predicted RF values is only explained in the “responses to reviewers” and no changes have been made in the manuscript. This means that the reader will still be left with the same confusion as the reviewer was on the first reading. The way these values were calculated is quite critical to clarify how trustworthily the results can be interpreted. Equation or a few sentences in the Materials and Methods would be really helpful and would improve the data quality of the paper.
Please also incorporate in the manuscript that isotope correction was done by Tracefinder.
Last but not least, R of 0.32 is R2 of 0.10 and calling it moderate is questionable.

Reviewer 1
Concerning the flash chromatography, given that the elution has been achieved through a gradient from 100% hexane to 40% EtOAc (this is what I understand from the text) it would be good to add a sentence to describe the elution gradient so that it can be reproduced by others

An additional sentence has been added to the SI:

“The reaction mixture was purified by flash column chromatography (silica, n-hexane 100% for 6 min, then a gradient to 20% EtOAc in nhexane for 5 min, then 20% EtOAc in nhexane for 10 min, then a gradient to n-hexane 100% to 40% EtOAc in hexane…”

concerning "calcd", my suggestion was to spell the word in full as this is not a defined abbreviation

This has now been changed in the SI.

Reviewer 2
Addition text has been added or changed in the text:

“All isotopic peaks were corrected with the theoretical isotope correction factor within the software.”

“The pKa had a correlation of R = 0.32”

“The RMSE and R2 values were calculated by default by the built-in functionality of the Caret R package. The final model was chosen based on minimising the RMSE.”

27-Nov-2022

Dear Dr Bryant:

Manuscript ID: EA-ART-06-2022-000074.R2
TITLE: Overcoming the lack of authentic standards for the quantification of biogenic secondary organic aerosol markers

Thank you for submitting your revised manuscript to Environmental Science: Atmospheres. I am pleased to accept your manuscript for publication in its current form. I have copied any final comments from the reviewer(s) below.

You will shortly receive a separate email from us requesting you to submit a licence to publish for your article, so that we can proceed with the preparation and publication of your manuscript.

You can highlight your article and the work of your group on the back cover of Environmental Science: Atmospheres. If you are interested in this opportunity please contact the editorial office for more information.

Promote your research, accelerate its impact – find out more about our article promotion services here: https://rsc.li/promoteyourresearch.

We will publicise your paper on our Twitter account @EnvSciRSC – to aid our publicity of your work please fill out this form: https://form.jotform.com/211263048265047

How was your experience with us? Let us know your feedback by completing our short 5 minute survey: https://www.smartsurvey.co.uk/s/RSC-author-satisfaction-energyenvironment/

By publishing your article in Environmental Science: Atmospheres, you are supporting the Royal Society of Chemistry to help the chemical science community make the world a better place.

With best wishes,

Dr Claudia Mohr

Associate Editor, Environmental Science: Atmospheres

Reviewer 1

a