From the journal Digital Discovery Peer review history

14-Jun-2023

Dear Dr Josephson:

Manuscript ID: DD-ART-04-2023-000077
TITLE: Formalizing Chemical Theory using the Lean Theorem Prover

Thank you for your submission to Digital Discovery, published by the Royal Society of Chemistry. I sent your manuscript to reviewers and I have now received their reports which are copied below.

I have carefully evaluated your manuscript and the reviewers’ reports, and the reports indicate that major revisions are necessary.

Please submit a revised manuscript which addresses all of the reviewers’ comments. Further peer review of your revised manuscript may be needed. When you submit your revised manuscript please include a point by point response to the reviewers’ comments and highlight the changes you have made. Full details of the files you need to submit are listed at the end of this email.

Digital Discovery strongly encourages authors of research articles to include an ‘Author contributions’ section in their manuscript, for publication in the final article. This should appear immediately above the ‘Conflict of interest’ and ‘Acknowledgement’ sections. I strongly recommend you use CRediT (the Contributor Roles Taxonomy, https://credit.niso.org/) for standardised contribution descriptions. All authors should have agreed to their individual contributions ahead of submission and these should accurately reflect contributions to the work. Please refer to our general author guidelines https://www.rsc.org/journals-books-databases/author-and-reviewer-hub/authors-information/responsibilities/ for more information.

Please submit your revised manuscript as soon as possible using this link:

*** PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm. ***

https://mc.manuscriptcentral.com/dd?link_removed

(This link goes straight to your account, without the need to log on to the system. For your account security you should not share this link with others.)

Alternatively, you can login to your account (https://mc.manuscriptcentral.com/dd) where you will need your case-sensitive USER ID and password.

You should submit your revised manuscript as soon as possible; please note you will receive a series of automatic reminders. If your revisions will take a significant length of time, please contact me. If I do not hear from you, I may withdraw your manuscript from consideration and you will have to resubmit. Any resubmission will receive a new submission date.

The Royal Society of Chemistry requires all submitting authors to provide their ORCID iD when they submit a revised manuscript. This is quick and easy to do as part of the revised manuscript submission process.   We will publish this information with the article, and you may choose to have your ORCID record updated automatically with details of the publication.

Please also encourage your co-authors to sign up for their own ORCID account and associate it with their account on our manuscript submission system. For further information see: https://www.rsc.org/journals-books-databases/journal-authors-reviewers/processes-policies/#attribution-id

I look forward to receiving your revised manuscript.

Yours sincerely,
Dr Joshua Schrier
Associate Editor, Digital Discovery

************

Reviewer 1

I was brought on to review the Lean proofs, but I also took a look at the paper itself. My review will focus on the proofs but touch on the paper as well.

I marked "no" for reproducibility because I made several efforts to check the proofs provided in your GitHub repo in Lean, and failed to get the proofs to load without errors. The authors need to provide substantially more detailed reproduction information in their repository. In formal methods venues, it is typical to provide a virtual machine image to ensure easy reproduction. Though I realize the cultural norms may differ by field, it is essential to provide the following information at the minimum, in the GitHub repo and/or in a clearly marked appendix of the paper:
- A minimum set of steps for installing Lean and appropriate tools (rather than linking to an external website which may change)
- Which exact commit to check out for reproduction
- Exact version numbers for all dependencies
- Exact commands to run in order to check the proofs
- What to look for to confirm successful/ unsuccessful reproduction
- Which checking mode (e.g. visible files vs. project files) to select in VSCode in order to get correct results
- Which operating system(s) and version numbers you have reproduced the results on.

Here is an example of an error message that I got when trying to reproduce commit c2b15363c1e0ea0e52c1ae86abd1650670ff9044 on Windows 10:
[{
"resource": "/C:/Users/rbohrer/Documents/GitHub/LeanChemicalTheories/src/thermodynamics/basic.lean",
"owner": "lean",
"severity": 8,
"message": "unknown identifier 'ring_exp'",
"source": "Lean",
"startLineNumber": 77,
"startColumn": 3,
"endLineNumber": 77,
"endColumn": 11
},{
"resource": "/C:/Users/rbohrer/Documents/GitHub/LeanChemicalTheories/src/thermodynamics/basic.lean",
"owner": "lean",
"severity": 8,
"message": "don't know how to synthesize placeholder\ncontext:\nM : thermo_system\n⊢ Type ?",
"source": "Lean",
"startLineNumber": 77,
"startColumn": 3,
"endLineNumber": 77,
"endColumn": 11
},{
"resource": "/C:/Users/rbohrer/Documents/GitHub/LeanChemicalTheories/src/thermodynamics/basic.lean",
"owner": "lean",
"severity": 8,
"message": "unknown identifier 'ring_exp'",
"source": "Lean",
"startLineNumber": 98,
"startColumn": 3,
"endLineNumber": 98,
"endColumn": 11
},{
"resource": "/C:/Users/rbohrer/Documents/GitHub/LeanChemicalTheories/src/thermodynamics/basic.lean",
"owner": "lean",
"severity": 8,
"message": "don't know how to synthesize placeholder\ncontext:\nM : thermo_system\n⊢ Type ?",
"source": "Lean",
"startLineNumber": 98,
"startColumn": 3,
"endLineNumber": 98,
"endColumn": 11
},{
"resource": "/C:/Users/rbohrer/Documents/GitHub/LeanChemicalTheories/src/thermodynamics/basic.lean",
"owner": "lean",
"severity": 8,
"message": "unknown identifier 'ring_exp'",
"source": "Lean",
"startLineNumber": 131,
"startColumn": 3,
"endLineNumber": 131,
"endColumn": 11
},{
"resource": "/C:/Users/rbohrer/Documents/GitHub/LeanChemicalTheories/src/thermodynamics/basic.lean",
"owner": "lean",
"severity": 8,
"message": "don't know how to synthesize placeholder\ncontext:\nM : ideal_gas\n⊢ Type ?",
"source": "Lean",
"startLineNumber": 131,
"startColumn": 3,
"endLineNumber": 131,
"endColumn": 11
}]

It is also important to delete the commented-out occurrence of "sorry" in kinematic_equations.lean, so that a busy reviewer does not mistake it for a real occurrence of "sorry".

Despite my strong criticisms regarding reproducibility, I suspect the issue is merely one of needing better instructions, not a critical flaw in the underlying work. For lack of a more sophisticated phrase, the proofs all "look like Lean proofs" and their length is substantial, reflecting substantial creative effort.

I marked "no" on "appropriate and balanced citations" because the paper listed at the end of this review is not cited. I am signing this review in part because it is my own work, and I am aware there is a taboo against asking authors to cite the reviewer's work. I am including this request only because I sincerely believe the work is related enough to merit a citation.

The introduction of the manuscript frames itself as the first treatment of chemistry in a theorem prover. That is not accurate. The paper below is a prior treatment of chemistry in a theorem prover*, and its related work section gives an overview on a number of prior works using other formal methods approaches such as model checking, going back decades. (*The soundness of the theorem prover used in this work has been proved in two foundational proof checkers, in my other work). This is important context to include in your paper.

That being said, I still strongly support the publication of this work once the issues are addressed. Your formalization is far more substantial than the prior work, and so you have nothing to fear from a fair comparison. Moreover, as an interdisciplinary researcher myself, I believe this paper does very important work to bridge the cultural gaps between the chemistry community and the theorem proving community. The submission does a good job of motivating the fundamentals of theorem-proving from a high level for an audience that has never seen a theorem prover before.

Additional notes:
- The manuscript contains figures which are screenshots of text. These are most likely inaccessible to readers who use screen readers, and need to be replaced with accessible figures (either by typesetting the text or by including appropriate alt text and producing a PDF/A)

- Rose Bohrer

@inproceedings{DBLP:conf/fmics/Bohrer22,
author = {Rose Bohrer},
editor = {Jan Friso Groote and
Marieke Huisman},
title = {Chemical Case Studies in KeYmaera {X}},
booktitle = {Formal Methods for Industrial Critical Systems - 27th International
Conference, {FMICS} 2022, Warsaw, Poland, September 14-15, 2022, Proceedings},
series = {Lecture Notes in Computer Science},
volume = {13487},
pages = {103--120},
publisher = {Springer},
year = {2022},
url = {https://doi.org/10.1007/978-3-031-15008-1\_8},
doi = {10.1007/978-3-031-15008-1\_8},
timestamp = {Sat, 10 Sep 2022 20:58:36 +0200},
biburl = {https://dblp.org/rec/conf/fmics/Bohrer22.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

Reviewer 2

The paper discusses the application of the Lean theorem prover to justify derivations of mathematical formulas used in physical chemistry. Four kinds of examples are given: The derivation of formulas related to Langmuir model of single molecular-layer adsorption; the derivation of formulas related to the more complex BET model of infinite layer adsorption; the derivation of formulas related to the ideal gas law; and the derivation of the formula for constant acceleration motion.

The general proposal of applying theorem verification technology beyond the well-studied applications in mathematics and computer science to science and specifically to chemistry is a very exciting one. The presentation is quite clear and well written.

It is disappointing, though, to find that the results here are extremely limited and preliminary.

The reasoning done here is purely mathematical; the physics and chemistry is all baked into a few comparatively simple mathematical formulas. And the mathematical manipulations themselves involve merely some very simple algebraic manipulation and basic calculus, well within the power of symbolic manipulation systems of fifty years ago, such as MACSYMA. (I am not absolutely sure that MACSYMA, in 1968, could have handled the infinite series in the BET derivation, but certainly symbolic manipulation systems have handled these for several decades.) A theory of chemistry with no atoms, molecules, elements, chemicals, solids, fluids, gasses, chemical reactions, or phase changes and barely any reference to time or space is a very thin one; and a theory of scientific reasoning that consists purely of mathematical symbol manipulation is a very weak argument for using a framework that is as powerful and demanding as Lean. As far as the reader can tell here, the only advantage of using Lean in these examples over using much more straightforward and simpler symbolic math systems (Mathematica, Sympy, Matlab, etc.) is that Lean forces the user to explicity rule out division by zero, and to explicitly state conditions of summability, for the BET formulas, and of continuous differentiability, for the constant acceleration problem. Lean is capable of supporting much more complex forms of argumentation, and it would be much more satisfying if something beyond mathematical symbolic manipulation were illustrated here.

All in all, I don't think that the content here is significant or deeo enough to merit publication in Digital Discovery. In a shortened form, it might be publishable at a general AI conference such as IJCAI or AAAI or at a conference on applications of proof verifiers. (I can't speak to possible chemistry venues.)

A couple of specific comments.

The proposal of using mathematical logic for scientific reasoning goes back to work in the 1930s-1960s by logicians such as Rudolf Carnap and Alfred Tarski. It would be good to make some mention of this.

It is not clear why the solutions x(t) = at^{2}/2 + v_{0}t + x_{0) of the ODE initial value problem a=dv/dt, v=dx/dt, v(0)=v_{0}, x(0)=x_{0} belongs in a paper on chemistry rather than one on elementary math or physics. In the context of a paper on chemistry, it is not clear why one should want to prove the statement over an arbitrary normed vector space over either the reals or the complex numbers; I would think that, for chemistry applications, it would suffice to prove it for the case K=R, E=R^{n}. Finally, though I am not an expert on the state of the mathematical libraries available in Lean, I find it absolute impossible to believe that some form of this basic theorem is not available in them.

Figure 1 could be made a lot more readable if it were presented as ordinary text rather than as a white-on-black screen shot.

In section 3.4, defining state variables -- pressure, volume, temperature, and so on --- as functions over the integers seems altogether artificial. The integer argument is just being used to identify different isolated systems, and the only property of the integers that is being used is that there is more than one integer. It would suffice for these purposes to define the domain of the argument as being any set with at least two elements.

Reviewer 3

This work is a demonstration of the power of using computers to rigorously check proofs in the domain of chemical physics. Its important contributions are acting as an introduction for the uninitiated, making available a (growing) library of proofs, and bringing to light a (recoverable) lapse of rigor in the original BET paper. It should be pointed out that while the latter two points would fall under what is traditionally counted as "original research", the majority of the paper is tutorial like. If the editors deem that such tutorials fall within the journal's scope, then I can recommend publishing this particular one because it is well written and I enjoyed the introduction.

As a matter of perspective, I would suggest the authors focus a little more on how the future of this may integrate with AI, since chemists are mostly happy with their informal proofs (which come nowhere near the complexity of Fermat's last theorem, where computer checking can be make or break).

I have a radical suggestion that I hope the authors give some consideration to, just for purposes of being precise. I would change "Chemical Theory" in the title and throughout the paper to "Chemical Physics". The reason is that when I think "Chemical" theory, I think Hammet parameters, electronegativity values, Woodward-Hoffmann rules, and all the other more malleable frameworks that bench chemists use to think about bonding, etc. What is here in this article is chemically relevant, but it is really just the *physics* of large numbers of particles (or not even in the kinematic case). In fact, the authors suggest Noether's theorem as a next target. In short, theorem provers of this sort will not get any closer to *chemistry* than chemical physics. When I picked up the paper, I actually expected some pie-in-the sky article about abstracting all the other things I just mentioned at some new (and likely dubious) level of rigor.

I have a few specific comments:

Most importantly:

Something needs to be done about the reference to the blog post. It is pretty critical, exactly as the audience that the post was intended for, I was "upset" (in a scientific way) about an undefined derivative being assigned as zero. After the blog post, I had made my peace with it, particularly after the last paragraph. I don't think such a critical thing should be farmed out to the volatile internet. Somehow, at least the essence of the last paragraph (and any necessary inclusions to the effect of "any theorem about one translates into a theorem about the other, so having real.div is equivalent to having mathematical division") needs to be brought into the paper somehow, or the reference needs to be changed to something permanent (and then I still recommend more about it locally).

Secondarily:

It was Mathematica that actually trained me (by example) to be careful of the conditions in my derivations, telling me that certain integrals, etc, were only valid if, say, the real part of "a" was non-negative. So the a/ab = 1/b example regarding Wolfram Alpha is a bit of a mischaracterization of the rigor missing from computer algebra systems. The rigor *can* (and has partially been) implemented. That said, the argument that all of mathematics can be built up off of 6000 lines of logic-checking is compelling. I find it believable that these 6000 lines could eventually be made to contain identically zero bugs, which I never will expect from Mathematica, etc.

Realizing that the uninitiated now live in a world with ChatGPT, the boundaries of what tasks are presently left to the computer could be more explicit. I understand that Lean (and similar) still largely just check the logic of the steps that are directed by the user, and not that utilities for autocomplete* of proofs are currently well developed (* by this I expect is meant that one can enter assertions and a theorem whose proof is desired, and an algorithm will fill in the steps).

In eqs. 1-6, none of the symbols are defined. I have not thought too much about surface chemistry in a while and I had to get to the comment after eq 3 before I was pretty sure I knew what was meant by [S]. In Fig 1 that follows, it appears that one can enter an actual subscript (as opposed to using an underscore convention). Unfortunately it is really hard to read at this resolution. I assume it is a 0, but I had to scour around to make sure it was not maybe a theta.

I don't think the description of the Frechet derivative is quite accurate. To wit, the "high school" derivative definition here seems to be working for functions of a vector. Maybe it is not even necessary to get into this finer point?

Top of page 14: its -> it's (better: it is). Kudos that I found only one such trivial error.

Third-to-last paragraph of conclusion: I think a more precise word than "properly-designed" is needed. I don't know what is meant by this. Maybe it is not even necessary to have an adjective.

Referee 1:
I was brought on to review the Lean proofs, but I also took a look at the paper itself. My review will focus on the proofs but touch on the paper as well.

Author response:
Thank you for your time and for your helpful feedback.

Reviewer comments:
I marked "no" for reproducibility because I made several efforts to check the proofs provided in your GitHub repo in Lean, and failed to get the proofs to load without errors. The authors need to provide substantially more detailed reproduction information in their repository. In formal methods venues, it is typical to provide a virtual machine image to ensure easy reproduction. Though I realize the cultural norms may differ by field, it is essential to provide the following information at the minimum, in the GitHub repo and/or in a clearly marked appendix of the paper:
- A minimum set of steps for installing Lean and appropriate tools (rather than linking to an external website which may change)
- Which exact commit to check out for reproduction
- Exact version numbers for all dependencies
- Exact commands to run in order to check the proofs
- What to look for to confirm successful/ unsuccessful reproduction
- Which checking mode (e.g. visible files vs. project files) to select in VSCode in order to get correct results
- Which operating system(s) and version numbers you have reproduced the results on.

It is also important to delete the commented-out occurrence of "sorry" in kinematic_equations.lean, so that a busy reviewer does not mistake it for a real occurrence of "sorry".

Despite my strong criticisms regarding reproducibility, I suspect the issue is merely one of needing better instructions, not a critical flaw in the underlying work. For lack of a more sophisticated phrase, the proofs all "look like Lean proofs" and their length is substantial, reflecting substantial creative effort.

Author response:
Thank you for the helpful response. We have updated the code and updated the instructions for our GitHub repository, to help others download Lean and run our code. We also verified installation with the help of a third party. To make the manuscript text consistent with the latest version of the code, we have updated Method section to read:
“In Section 3 we outline the proofs formalized using Lean version 3.51.1. We host proofs on a website that provides a semi-interactive platform connecting to the Lean codes in our GitHub repository. An extended methods section introducing Lean is in the Supporting Information Section 5.1.”
We have updated the installation instructions in the README section of our GitHub repository to include the steps necessary for installing Lean and getting this project to work on various computer systems. GitHub repository link: https://github.com/ATOMSLab/LeanChemicalTheories

Reviewer comment:
I marked "no" on "appropriate and balanced citations" because the paper listed at the end of this review is not cited. I am signing this review in part because it is my own work, and I am aware there is a taboo against asking authors to cite the reviewer's work. I am including this request only because I sincerely believe the work is related enough to merit a citation.

The introduction of the manuscript frames itself as the first treatment of chemistry in a theorem prover. That is not accurate. The paper below is a prior treatment of chemistry in a theorem prover*, and its related work section gives an overview on a number of prior works using other formal methods approaches such as model checking, going back decades. (*The soundness of the theorem prover used in this work has been proved in two foundational proof checkers, in my other work). This is important context to include in your paper.

Author response:
Thank you for pointing out this relevant work. Indeed, it was our oversight to not cite this work, and we agree that referencing it is appropriate and beneficial for readers. We have revised the introduction section to include this citation, as well as to more broadly address prior work in formalization that has occurred outside of proof assistants (to address other reviewers’ comments, as described below). The relevant section of the introduction now reads:
“Here we focus on formalizing fundamental theories in the chemical sciences. Progress toward axiomatizing thermodynamics began with Caratheodory in 1909, with recent developments by Lieb and Yngvason. But broadly, these questions have not been addressed using theorem provers to check the mathematics, which have seen limited use in the chemical sciences. One notable application by Bohrer uses a proof assistant that reasons about differential equations and control algorithms to describe and prove properties of chemical reactors.”

Reviewer comment:
That being said, I still strongly support the publication of this work once the issues are addressed. Your formalization is far more substantial than the prior work, and so you have nothing to fear from a fair comparison. Moreover, as an interdisciplinary researcher myself, I believe this paper does very important work to bridge the cultural gaps between the chemistry community and the theorem proving community. The submission does a good job of motivating the fundamentals of theorem-proving from a high level for an audience that has never seen a theorem prover before.

Author response:
Thank you for your support!

Reviewer comments:
Additional notes:
- The manuscript contains figures which are screenshots of text. These are most likely inaccessible to readers who use screen readers, and need to be replaced with accessible figures (either by typesetting the text or by including appropriate alt text and producing a PDF/A)

Author response:
Thank you for considering the accessibility of this paper for readers who use screen readers.
First, we weighed your suggestion to replace our figures with typeset text. This would be cleaner, but we considered the form of Figure 1 to be important - a screenshot of Lean operating in VS Code shows (sighted readers) what the software environment looks like, and we decided that this style was important to maintain. We next explored two mechanisms for adding alt-text to figures in LaTeX, and we weren’t able to technically accomplish this. We consulted with Mona Minkara, a colleague who is a blind computational chemist, and she simply said that .pdfs aren’t reliably accessible to screen readers, period.
So, instead we are providing our alt text below, and will ask Digital Discovery to incorporate these in the web version of the publication.

Alt text for images in Table 1:
A table comparing hand-written proofs and formal proofs. The left column pertains to hand-written proofs, while the right column relates to formal proofs. Features of hand-written proofs include informal syntax, intended for human readers, potential to exclude information, possible presence of mistakes, the necessity of human proofreading, and the relative ease of writing. In contrast, formal proofs are characterized by a strict, computer language syntax, machine-readability and executability, inability to miss assumptions or steps, rigorous verification by a computer, automated proof checking, and a level of difficulty in writing.

Alt text for images in Table 2:
A table referencing two distinct types of computational systems used in mathematical and logical operations. First, Interactive Theorem provers such as Lean and Isabelle, are tools that assist in the creation and checking of formal logic proofs. Secondly, Computer Algebra Systems such as Mathematica, Sympy, and the symbolic math toolbox in MATLAB, are software packages that facilitate symbolic mathematics.

Alt text for images in Figure 1:
Screenshot of Langmuir's adsorption model formalization in Lean using VSCode. The left panel displays the “Code Window” and the right panel shows the “Tactic State” displaying variables and goals. The proof progresses by applying tactics, updating hypotheses and goals at each numbered location in the “Code Window”, highlighted by the cursor. The turnstile symbol denotes the goal state. Steps 1 and 2 rewrite the adsorption and desorption equations, affirming equilibrium. Steps 3-5 rearrange the variables to match the goal state. The completion of the proof is indicated by a party emoji.

Alt text for images in Figure 2:
The figure provides a comparison between the Langmuir and BET adsorption models. Unlike Langmuir, the BET model accommodates the formation of infinite layers over previously adsorbed particles. In this context, 'θ' signifies the proportion of the surface that has been adsorbed, 'V' refers to the overall volume adsorbed, 'V0' is the volume of a complete monolayer adsorbed on a unit area, 'si' designates the surface area of each sequential layer, 's0' is the surface area of the initial layer, and 'xi' and 'C' are constants associated with the heat of adsorption for the respective molecule layers.

Alt text for images in Figure 3:
This is a description of the 'thermo_system' and 'ideal_gas' structures in the Lean theorem prover. These structures represent different types of thermodynamic systems, such as 'isobaric', 'isochoric', and 'isothermal'. They use Lean's definitions to help in proving theorems related to gas laws.

Alt text for images in Figure 4:
This is a description of how kinematics is represented in the Lean theorem prover. A structure named 'motion' is defined to depict the relationship between position, velocity, and acceleration through the use of differential equations. These relationships are formalized and proven using Lean's definitions of derivative functions.

Furthermore, we started a conversation in the Lean community (link here: https://leanprover.zulipchat.com/#narrow/stream/113488-general/topic/Screen.20Readers.20in.20VS.20Code/) inquiring about the accessibility of Lean in VS Code. In short, Lean is not very accessible to those who use screen readers, but people are interested in investing their time to improve this. Current VS Code integrations provide interactive information on types, documentation, and proof state as mouseover text or in the Lean infoview - future tools could build these to be compatible with screen readers.

Reviewer comments:
- Rose Bohrer

@inproceedings{DBLP:conf/fmics/Bohrer22,
author = {Rose Bohrer},
editor = {Jan Friso Groote and
Marieke Huisman},
title = {Chemical Case Studies in KeYmaera {X}},
booktitle = {Formal Methods for Industrial Critical Systems - 27th International
Conference, {FMICS} 2022, Warsaw, Poland, September 14-15, 2022, Proceedings},
series = {Lecture Notes in Computer Science},
volume = {13487},
pages = {103--120},
publisher = {Springer},
year = {2022},
url = {https://doi.org/10.1007/978-3-031-15008-1\_8},
doi = {10.1007/978-3-031-15008-1\_8},
timestamp = {Sat, 10 Sep 2022 20:58:36 +0200},
biburl = {https://dblp.org/rec/conf/fmics/Bohrer22.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

Author response:
Thank you for pointing out this important and relevant prior work. We have included this citation and quoted the passage in our first response.
----------------------------------------------------------------

Referee: 2
Reviewer comments:
The paper discusses the application of the Lean theorem prover to justify derivations of mathematical formulas used in physical chemistry. Four kinds of examples are given: The derivation of formulas related to Langmuir model of single molecular-layer adsorption; the derivation of formulas related to the more complex BET model of infinite layer adsorption; the derivation of formulas related to the ideal gas law; and the derivation of the formula for constant acceleration motion.

The general proposal of applying theorem verification technology beyond the well-studied applications in mathematics and computer science to science and specifically to chemistry is a very exciting one. The presentation is quite clear and well written.

Author response:
Thank you, we’re glad you also find this new direction exciting.

Reviewer comments:
It is disappointing, though, to find that the results here are extremely limited and preliminary.

The reasoning done here is purely mathematical; the physics and chemistry is all baked into a few comparatively simple mathematical formulas. And the mathematical manipulations themselves involve merely some very simple algebraic manipulation and basic calculus, well within the power of symbolic manipulation systems of fifty years ago, such as MACSYMA. (I am not absolutely sure that MACSYMA, in 1968, could have handled the infinite series in the BET derivation, but certainly symbolic manipulation systems have handled these for several decades.) A theory of chemistry with no atoms, molecules, elements, chemicals, solids, fluids, gasses, chemical reactions, or phase changes and barely any reference to time or space is a very thin one; and a theory of scientific reasoning that consists purely of mathematical symbol manipulation is a very weak argument for using a framework that is as powerful and demanding as Lean. As far as the reader can tell here, the only advantage of using Lean in these examples over using much more straightforward and simpler symbolic math systems (Mathematica, Sympy, Matlab, etc.) is that Lean forces the user to explicity rule out division by zero, and to explicitly state conditions of summability, for the BET formulas, and of continuous differentiability, for the constant acceleration problem. Lean is capable of supporting much more complex forms of argumentation, and it would be much more satisfying if something beyond mathematical symbolic manipulation were illustrated here.

Author response:
Thank you for your response. You are right that Lean is a powerful tool capable of handling the most complex of arguments. Our theme for this paper is that of an introduction to a new tool. Since this tool is developed by mathematicians, we also have to introduce a new way of using mathematics, in a way scientists aren’t used to. So, while our examples could’ve been ones where we define co-tangent bundles to represent a phase space when talking about an ensemble of atoms, or used fderiv and measure spaces to define quantum mechanics, we believed it better to pick more “elementary” examples, where we don’t have to spend pages explaining the mathematical concepts or assume reader proficiency in the math.

However, we also don’t believe that the examples we did show are simple. While the derivation of these examples in an informal sense (i.e. by hand as college students would do), is a simple exercise for scientists taking less than half a page, a large part of this paper is dedicated to the task of answering the question, “How do we convert this informal language into a formal one?” We structure the paper in a way where the language becomes more formal as it progresses. While the first proof we show, the derivation of the Langmuir Isotherm, is no more than the symbolic manipulation of an equation of real numbers, the rest of the proofs progress beyond this notion of simple symbolic manipulation. The Langmuir proof is left in that stage to introduce what Lean looks like and some basic concepts. However, we use recursion to define the BET sequence and structures to define systems and fields of theory. We go beyond the idea of simple symbolic manipulation by defining objects and reasoning about them. In the thermodynamics proofs, for instance, we are able to define system descriptions (isothermal, isobaric, etc.), gas laws, equations of state, and prove theorems about how they relate and logically follow from each other.

We especially appreciate your comment, “A theory of chemistry with no atoms, molecules, elements, chemicals, solids, fluids, gasses, chemical reactions, or phase changes and barely any reference to time or space is a very thin one” and we consider it in tension with Reviewer 3’s perspective, who anticipated our paper to propose a “dubious” formalization of the breadth of chemical theory. In response, we are accepting Reviewer 3’s suggestion to rename our paper to “Formalizing Chemical Physics” to narrow the scope of what this paper includes.

Nonetheless, your comment invites many exciting questions: Could we, indeed, formally define these concepts using rigorous mathematical reasoning? How would we describe all salient properties of the relationship between a molecule and its constituent atoms? How do we formalize chemical bonding? Etc. These may well lead us into “dubious” territory, as many of these concepts are qualitatively understood by the community, but they will be fun to explore in the future.

Reviewer comments:
All in all, I don't think that the content here is significant or deeo enough to merit publication in Digital Discovery. In a shortened form, it might be publishable at a general AI conference such as IJCAI or AAAI or at a conference on applications of proof verifiers. (I can't speak to possible chemistry venues.)

Author response:
We want to reach the diverse and interdisciplinary readership of Digital Discovery because our approach is applicable to areas of science and engineering beyond chemistry. Formal methods are not yet prevalent in the science and engineering community, and we hope to encourage their more widespread adoption.


Reviewer comment:
A couple of specific comments.

The proposal of using mathematical logic for scientific reasoning goes back to work in the 1930s-1960s by logicians such as Rudolf Carnap and Alfred Tarski. It would be good to make some mention of this.

Author response:
Thank you for the suggestion! We originally focused our introduction on the role of *computer-assisted* formalization (which is much more limited), but didn’t discuss the “by hand” formalization efforts in science. We have now modified/added the following four paragraphs in the introduction:

“Historically, interactive theorem provers have been used to logically connect advanced math theorems to the foundational axioms of mathematics. Before computers, this ``axiomatization'' of mathematics was developed by hand, in works like Principia Mathematica by Alfred North Whitehead and Bertrand Russell -- the aim is to write down a minimal list of fundamental assumptions (axioms), and then systematically derive all of mathematics from those axioms. Computers play a key role in modern formalization efforts because they can store and verify massive libraries of interconnected theorems collaboratively written by hundreds of mathematicians.

An analogous program to “axiomatize” physics was famously articulated as Hilbert's sixth problem. Recent reviews have discussed progress and unsolved questions on this ``endless road'' to describe how all of physics can be derived from a minimal set of axioms. Our vision is somewhat distinct from this -- we are inspired by Paleo’s ideas for formalizing physics theories as a collection of proofs, instead of aiming to represent science as a single edifice emerging from one set of axioms (though this structure may emerge in the future). In particular, we ask “How can we formally represent a collection of proofs/derivations using an interactive theorem prover?'”

Theorem provers have previously been used to formalize derivations in physics: theorems from Newton's Principia, versions of relativity theory, electromagnetic optics, and geometrical optics have been described and proved using proof assistants. Artificial intelligence tools for scientific discovery have also used theorem provers in designing optical quantum experiments, as well as for rediscovering and deriving scientific equations from data and background theory.

Here we focus on formalizing fundamental theories in the chemical sciences. Progress toward axiomatizing thermodynamics began with Caratheodory in 1909, with recent developments by Lieb and Yngvason. But broadly, these questions have not been addressed using theorem provers to check the mathematics, which have seen limited use in the chemical sciences. One notable application by Bohrer uses a proof assistant that reasons about differential equations and control algorithms to describe and prove properties of chemical reactors.”


And in the Conclusions and Outlook, we now mention Tarski and Carnap in this new paragraph:

“Importantly, we emphasize that while our proofs are verified to be mathematically correct, this verification does not extend to the external world. This distinction between syntax (logical relationships among words and arguments in a language) and semantics (whether words are meaningful or arguments are true, according to external reality) in scientific reasoning has been emphasized by logicians such as Alfred Tarski and Rudolf Carnap. For scientists and engineers, whether a theory is true or meaningful is first and foremost about whether observational data support it -- logical correctness of the derivation is required, of course, but this is typically assumed. Indeed, when one of us described our BET proof to an experimentalist in adsorption, their reply was, "but BET isn't accurate." They knew that BET theory does not semantically match experiments in many contexts (in fact, much literature has discussed when BET analysis should \emph{not} be applied, for instance, with certain metal–organic frameworks (MOFs)). BET theory has been a useful conceptual model for the field, but nonetheless relies on approximations that often drift far from reality. In this work, we only claim to rigorously establish the \emph{syntax} of the theories we describe. Nonetheless, Lean operating with input/output functions can receive data from the external world, which may open possibilities for semantically grounding its logical conclusions in certain contexts, as well.”

Reviewer comment:
It is not clear why the solutions x(t) = at^{2}/2 + v_{0}t + x_{0) of the ODE initial value problem a=dv/dt, v=dx/dt, v(0)=v_{0}, x(0)=x_{0} belongs in a paper on chemistry rather than one on elementary math or physics. In the context of a paper on chemistry, it is not clear why one should want to prove the statement over an arbitrary normed vector space over either the reals or the complex numbers; I would think that, for chemistry applications, it would suffice to prove it for the case K=R, E=R^{n}. Finally, though I am not an expert on the state of the mathematical libraries available in Lean, I find it absolute impossible to believe that some form of this basic theorem is not available in them.

Author response:
At the time of writing, these theorems were not yet in mathlib. We would have greatly appreciated the convenience of being able to import them, as it would have indeed saved us effort. Here's the link to a discussion we had at the time with Lean community users about antiderivatives - https://leanprover.zulipchat.com/#narrow/stream/113488-general/topic/Antiderivative.20in.20Lean. They couldn’t point us to existing theorems in mathlib (indeed, one person was surprised these were not available, like you were), and instead provided suggestions on how to rewrite and generalize our new proofs. We have taken a similar approach with BET proof as well - imported the theorems when they were available and proved them ourselves when they were not.

Part of the issue of mathlib’s coverage for science and engineering mathematics is that mathematicians have been formalizing what interests them most, and this doesn’t necessarily lead to thorough coverage of math that’s useful for scientists and engineers. We hope that bringing more scientists and engineers will help broaden the coverage in this direction.

Reviewer comment:
Figure 1 could be made a lot more readable if it were presented as ordinary text rather than as a white-on-black screen shot.

Author response:
See our response to Reviewer 1’s comment. We have changed the screenshot to black-on-white in light of your comment.

Reviewer comment:
In section 3.4, defining state variables -- pressure, volume, temperature, and so on --- as functions over the integers seems altogether artificial. The integer argument is just being used to identify different isolated systems, and the only property of the integers that is being used is that there is more than one integer. It would suffice for these purposes to define the domain of the argument as being any set with at least two elements.

Author response:
Thank you for your comment. We have updated our code to implement this suggestion and revised the paragraph about the functions to explain what the general type is and how it can be substituted for natural numbers for states or real numbers for time.

The paper now reads, starting at the end of page 9:
“...We formalize the concept of “thermodynamic system” by defining a Lean structure called thermo_system over the real numbers, with thermodynamic properties (e.g. pressure, volume, etc.) defined as functions from a type to the real numbers α → R. Here, α is meant to represent a general indexing type. It could be the natural numbers if we wanted to use those to represent states of the system, real numbers to represent time, or anything else. The only requirement is that α is nonempty, meaning it has at least two different elements…”

---------------------------------------------------------------------------
Referee: 3
Reviewer comment:
This work is a demonstration of the power of using computers to rigorously check proofs in the domain of chemical physics. Its important contributions are acting as an introduction for the uninitiated, making available a (growing) library of proofs, and bringing to light a (recoverable) lapse of rigor in the original BET paper. It should be pointed out that while the latter two points would fall under what is traditionally counted as "original research", the majority of the paper is tutorial like. If the editors deem that such tutorials fall within the journal's scope, then I can recommend publishing this particular one because it is well written and I enjoyed the introduction.

As a matter of perspective, I would suggest the authors focus a little more on how the future of this may integrate with AI, since chemists are mostly happy with their informal proofs (which come nowhere near the complexity of Fermat's last theorem, where computer checking can be make or break).

Author response:
Thank you for the suggestion - we have expanded our discussion of AI to provide a brief survey of AI and automation in theorem proving for mathematics, and suggest how parallel developments could support automating theorem proving in science.

Recent approaches have leveraged machine learning to expand the capabilities of automated theorem proving. Theorem proving can be framed as a reinforcement learning problem, in which an agent is to learn an effective theorem proving policy via rewards from successfully proving theorems. “Autoformalization” refers to the translation of informal proofs into formal proofs, akin to translating text from one language to another (but with extremely strict requirements on the formal side). Theorem proving can also be framed as a next-word-prediction problem (“auto-complete” for math proofs) in which a database of formal math proofs is used to train a language model to predict the next word in the proof. Large language models (LLMs) like ChatGPT have some emergent reasoning abilities but often make mistakes and cannot be trusted. By connecting language models with ITPs to provide feedback, training them on proof databases like \texttt{mathlib}, and deploying them as part of traditional search algorithms, progress has been made toward automating proofs in Lean, even to the point of generating correct solutions to International Math Olympiad problems.

This interplay between creative but unreliable generative algorithms and the strict logic of a proof-checking system may be a model for future AI-driven discovery in science, especially for discovering new theories. An early example of this is AI-Descartes, in which a symbolic regression algorithm generates equations to match experimental data, which is then combined with an automated theorem prover to establish the equations' “derivability” with respect to a scientific theory. However, in this work, each theory required human expertise to be expressed in formal language, and reliance on an automated theorem prover limited the scope of theories to those expressible in first-order logic. AI tools that can autoformalize the informal scientific literature, generate novel theories, and auto-complete complex proofs could open new avenues for automating theory discovery. LLMs have demonstrated capabilities in solving chemistry problems, as well as answering scientific question-and-answer problems invoking quantitative reasoning. However, LLMs are unreliable - they famously “hallucinate” (generate falsehoods) and are biased or unreliable evaluators of their own outputs. Pairing them with external tools improves their capabilities; theorem provers could play a role like that. How will these models be trained? We suggest two avenues: training on human-written databases of formal proofs in science and engineering (which are yet to be written) and leveraging interactive feedback from Lean through tools like. Beyond being formally grounded in axiomatic mathematics, formal proofs in science and engineering are machine-readable instances of correct mathematical logic that could serve as a foundation for artificial intelligences aiming to learn, reason, and discover in science.

Reviewer comments:
I have a radical suggestion that I hope the authors give some consideration to, just for purposes of being precise. I would change "Chemical Theory" in the title and throughout the paper to "Chemical Physics". The reason is that when I think "Chemical" theory, I think Hammet parameters, electronegativity values, Woodward-Hoffmann rules, and all the other more malleable frameworks that bench chemists use to think about bonding, etc. What is here in this article is chemically relevant, but it is really just the *physics* of large numbers of particles (or not even in the kinematic case). In fact, the authors suggest Noether's theorem as a next target. In short, theorem provers of this sort will not get any closer to *chemistry* than chemical physics. When I picked up the paper, I actually expected some pie-in-the sky article about abstracting all the other things I just mentioned at some new (and likely dubious) level of rigor.

Author response:
We accept the suggestion, and appreciate how it brings focus. We have updated the title, and also emphasized the focus on chemical physics in the abstract, by clarifying that we are illustrating “theories in chemical physics.”

We also appreciate your candor in expecting theorem provers to be limited to treatments of physics and chemical physics. We anticipate that more than just chemical physics can be expressed and reasoned about in theorem provers like Lean. For instance, a structure could be defined that simply stores the periodic table and various properties of the elements (including electronegativity). In this way, “data” from experimental observations can be brought in, and then reasoned about using formally-correct logical machinery (e.g. to prove that the electronegativity of F > the electronegativity of O). In cheminformatics, molecules are often represented as graphs; mathlib has theorems and definitions in graph theory and topology, which might be used to describe stereochemistry, and in connection with a periodic table, functional groups. Indeed, we’re speculating here beyond what we demonstrate in the paper, but we hope this illustrates some broader possibilities.

A further comment: before a domain of study can be formalized, it must first be mathematized, and the aspects of chemistry which have not been mathematized would need to be so before we could express and reason about them in a theorem prover.

Reviewer comments:
I have a few specific comments:

Most importantly:
Something needs to be done about the reference to the blog post. It is pretty critical, exactly as the audience that the post was intended for, I was "upset" (in a scientific way) about an undefined derivative being assigned as zero. After the blog post, I had made my peace with it, particularly after the last paragraph. I don't think such a critical thing should be farmed out to the volatile internet. Somehow, at least the essence of the last paragraph (and any necessary inclusions to the effect of "any theorem about one translates into a theorem about the other, so having real.div is equivalent to having mathematical division") needs to be brought into the paper somehow, or the reference needs to be changed to something permanent (and then I still recommend more about it locally).

Author response:
Thank you for the helpful feedback. We anticipate other readers will wrestle with this as you (and we) have, and appreciate the opportunity to cover this properly. We have saved the blog on the Wayback Machine and added the link as a citation as well. You can access it here: https://web.archive.org/web/20230719150030/https://xenaproject.wordpress.com/2020/07/05/division-by-zero-in-type-theory-a-faq/

Our footnote has been updated and now states:

“Likewise, division by zero is defined to return zero instead of something like "undefined" or "NaN”. In Lean and other theorem provers, the symbol / is not used for mathematical division, but instead points to a function called real.div. This function returns x/y if y is not zero, and 0 if y equals 0. Another case is the square root function (real.sqrt), which outputs a real number for any input, even negatives, since it is defined as R → R. These conventions may be unfamiliar to scientists and engineers, but they are used for convenience and won't lead to contradictions in a proof. Any invalid step in a proof involving these conventions will be caught when invoking a theorem not true for its definition. We wrestled with this convention for some time, before finding clarity in this blog post archived for reference [citation].”

Reviewer comments:
Secondarily:

It was Mathematica that actually trained me (by example) to be careful of the conditions in my derivations, telling me that certain integrals, etc, were only valid if, say, the real part of "a" was non-negative. So the a/ab = 1/b example regarding Wolfram Alpha is a bit of a mischaracterization of the rigor missing from computer algebra systems. The rigor *can* (and has partially been) implemented. That said, the argument that all of mathematics can be built up off of 6000 lines of logic-checking is compelling. I find it believable that these 6000 lines could eventually be made to contain identically zero bugs, which I never will expect from Mathematica, etc.

Author response:
Thank you for your comment. The example given here was meant to show a time where CAS, even ones like Mathematica which have partially implemented a sort of rigor, still miss things at times. However, we have removed the example because we agree that it does mischaracterize CAS, especially ones as sophisticated as Mathematica. Instead, we just focus on your second suggestion where Lean only having 6000 lines of code for the kernel and automated checking beyond that gives a higher possibility of trust than CAS.

The paragraph at page 1 now reads:
“Scientists are generally familiar with computer algebra systems (CAS) that can symbolically manipulate mathematical expressions (see Table 2). These systems include SymPy [ 11] and Mathematica. These systems are used frequently for scientific applications but come at the cost of being unsound, meaning they can have false conclusions.”

Reviewer comments:
Realizing that the uninitiated now live in a world with ChatGPT, the boundaries of what tasks are presently left to the computer could be more explicit. I understand that Lean (and similar) still largely just check the logic of the steps that are directed by the user, and not that utilities for autocomplete* of proofs are currently well developed (* by this I expect is meant that one can enter assertions and a theorem whose proof is desired, and an algorithm will fill in the steps).

Author response:
Thank you for the comment. Your understanding is correct - interactive theorem provers indeed check the logic of steps directed by the user (though by using tactics, a single “step” from the user may constitute a large number of automated steps “behind the scenes”). Automated theorem provers fill in all steps, as you describe, but such algorithms are generally limited in scope, being only applicable to problems described by first-order logic, for example. Sometimes, lines blur even further - for example, the “Sledgehammer” tactic for the Coq theorem prover attempts to convert a Coq problem statement first-order logic and then passes the new formulation into an optimized automated theorem prover, and the “Magnushammer” tactic calls a pre-trained LLM to assist in the “translation to first-order logic” step. To clarify these points for readers, we have added the following sentence when we introduce our first proof:

Lean is an interactive theorem prover, meaning that the user is primarily responsible for setting up the theorem and writing the proof steps, while Lean continuously checks the work and provides feedback to the user.

We also added a paragraph in the discussion about automation:

While learning Lean and writing the proofs appearing here, we routinely asked ourselves, “How do I close this goal? I wish there was a way to automate this.'' In fact, the first vision for computer-assisted proofs in the 1950s and 60s was to fully automate the process; interactive theorem provers that “merely" check human-written proofs didn't appear until later. But historically, automated theorem provers (ATPs) made progress on narrow classes of problems (e.g. statements in first-order logic), but couldn't address proofs in advanced math. In short, theorem proving is like searching for a path from premises to conjecture, but in a realm with an “infinite action space'' -- traditional algorithms have been inadequate. For complex proofs, interactive theorem provers (ITPs) have been more successful, because they facilitate human creativity in writing proofs, while leveraging the rigor of the computer for checking them and providing feedback to the user. Modern ITPs also use the computer for small-scale automation via tactics; the human provides strategy while the computer executes tactics. Complex tactics sometimes blur the line between automated and interactive theorem proving. For example, Isabelle (an ITP) has the Sledgehammer tactic, which takes the current proof state and attempts to transform it into an equivalent problem in first-order logic, which can then be efficiently solved using an ATP.”

Reviewer comments:
In eqs. 1-6, none of the symbols are defined. I have not thought too much about surface chemistry in a while and I had to get to the comment after eq 3 before I was pretty sure I knew what was meant by [S]. In Fig 1 that follows, it appears that one can enter an actual subscript (as opposed to using an underscore convention). Unfortunately it is really hard to read at this resolution. I assume it is a 0, but I had to scour around to make sure it was not maybe a theta.

Author response:
We’ve modified the presentation of the informal derivation of Langmuir so that we define all symbols.

Indeed! Lean’s syntax allows local and global notations for convenient use of UNICODE symbols in executable code. This means Lean code need not use an underscore convention for variable names with subscripts. We added a sentence about this in our caption for Figure 1:

“Lean allows the use of UNICODE symbols, so we use “S0” to represent total concentration of adsorption sites without needing underscores.”

Another reader of our arXiv submission found a typo in our Langmuir derivation: In Equation 5, S should have been S0. We fixed this, and also added a footnote to tell this relevant anecdote:
“The manuscript we first submitted for peer review included a typo in Eq. 5, with S0 appearing as S. Neither the authors nor the peer reviewers detected this; it was identified by a community member who accessed the paper on arXiv. Of course, Lean catches such typos immediately.”

Reviewer comments:
I don't think the description of the Frechet derivative is quite accurate. To wit, the "high school" derivative definition here seems to be working for functions of a vector. Maybe it is not even necessary to get into this finer point?

Author response:
Thank you for your comment. After review, we have removed the discussion comparing the Frechet derivative and deriv. While understanding the differences is important when dealing with multivariable functions, upon review, we agree that its not necessary to discuss in this paper. Also, the example we show, has a function which is a vector of a single variable, which is what allows the high school derivative to work, however it wouldn’t work if this vector was a function of multiple variables.


Reviewer comment:
Top of page 14: its -> it's (better: it is). Kudos that I found only one such trivial error.

Author response:
Thanks for catching this! It is fixed in the revision.

Reviewer comment:
Third-to-last paragraph of conclusion: I think a more precise word than "properly-designed" is needed. I don't know what is meant by this. Maybe it is not even necessary to have an adjective.

Author response:
We have removed this adjective and clarified this difference in the following way:
“While Lean 3 was designed for theorem proving and management of large-scale proof libraries, the new version, Lean 4, is a functional programming language for writing proofs and programs, as well as proofs about programs.”

22-Aug-2023

Dear Dr Josephson:

Manuscript ID: DD-ART-04-2023-000077.R1
TITLE: Formalizing Chemical Physics using the Lean Theorem Prover

Thank you for your submission to Digital Discovery, published by the Royal Society of Chemistry. I sent your manuscript to reviewers and I have now received their reports which are copied below.

After careful evaluation of your manuscript and the reviewers’ reports, I will be pleased to accept your manuscript for publication after very minor revisions.

Please revise your manuscript to fully address the reviewers’ comments. When you submit your revised manuscript please include a point by point response to the reviewers’ comments and highlight the changes you have made. Full details of the files you need to submit are listed at the end of this email.

Digital Discovery strongly encourages authors of research articles to include an ‘Author contributions’ section in their manuscript, for publication in the final article. This should appear immediately above the ‘Conflict of interest’ and ‘Acknowledgement’ sections. I strongly recommend you use CRediT (the Contributor Roles Taxonomy, https://credit.niso.org/) for standardised contribution descriptions. All authors should have agreed to their individual contributions ahead of submission and these should accurately reflect contributions to the work. Please refer to our general author guidelines https://www.rsc.org/journals-books-databases/author-and-reviewer-hub/authors-information/responsibilities/ for more information.

Please submit your revised manuscript as soon as possible using this link :

*** PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm. ***

https://mc.manuscriptcentral.com/dd?link_removed

(This link goes straight to your account, without the need to log in to the system. For your account security you should not share this link with others.)

Alternatively, you can login to your account (https://mc.manuscriptcentral.com/dd) where you will need your case-sensitive USER ID and password.

You should submit your revised manuscript as soon as possible; please note you will receive a series of automatic reminders. If your revisions will take a significant length of time, please contact me. If I do not hear from you, I may withdraw your manuscript from consideration and you will have to resubmit. Any resubmission will receive a new submission date.

The Royal Society of Chemistry requires all submitting authors to provide their ORCID iD when they submit a revised manuscript. This is quick and easy to do as part of the revised manuscript submission process.   We will publish this information with the article, and you may choose to have your ORCID record updated automatically with details of the publication.

Please also encourage your co-authors to sign up for their own ORCID account and associate it with their account on our manuscript submission system. For further information see: https://www.rsc.org/journals-books-databases/journal-authors-reviewers/processes-policies/#attribution-id

I look forward to receiving your revised manuscript.

Yours sincerely,
Dr Joshua Schrier
Associate Editor, Digital Discovery

************

Reviewer 2

The revised title, the new extensive discussion, and the technical edits in the revised version of this paper are certainly major improvements. Moreover, on further thought, I feel more enthusiastic about the technical accomplishments here than I did on first reading. So I'm now happy to recommend it for publication. I don't have any significant further edits to recommend.
On the question, debated between reviewer 3 and the authors, of whether theorem provers could be used for chemistry other than physical chemistry, I side with the authors and expect that they can and eventually they will be.

A few minor points:
p. 27 ""$\alpha$ is nonempty, mening it has at least two different elements".
Change "nonempty" to (I presume) "nontrivial".

p. 34. "Automatic theorem provers ... couldn't address proofs in advanced mathematics". There is AFAIK a single significant exception to that viz. the Robbins conjecture, proven by William McCune an ATP in 1996 --- as far as I know, still, 27 years later, the only mathematical theorem first proved by an ATP.
https://en.wikipedia.org/wiki/Robbins_algebra
But it's up to the authors whether they consider this worth mentioning.

Some of the references need work. References like "Alfred North Whitehead and Bertrand Russell. Principia mathematica by alfred north whitehead and bertrand
russell. 1963."; "David Hilbert. Mathematical problems. 1902." "Stanislas Polu ... Formal
mathematics statement curriculum learning, 2022" are really not adequate.

Reviewer 1

The revision has adequately addressed my concerns:
- The authors have provided substantially improved instructions for reproducing their results. I can now easily reproduce them
- The introduction and conclusion have been adequately revised to contextualize the history of theorem proving
- The related work has been appropriately updated
- I enjoyed the discussion of LLMs as was recommended by one of the other referees. Though I do not work with LLMs, this exact discussion has become a hot topic within the area and the thoughts in your discussion align well with thoughts that my collaborators have expressed in their papers.

I can only speak as an expert in theorem-proving, not an expert in chemistry, but from this perspective the motivations provided for pursuing the work are convincing. It is entirely possible that many chemists are interested in practical concerns far away from the foundations of chemical physics discussed in this work, but formalization of foundations is widely recognized in other fields (e.g. mathematics, computer science, philosophy) to be an important area of basic research. It deepens our knowledge of classic results, paves the way for verification of practical systems, and guides the development of verification tools so that they might be suitable for that work. When formalizing established theories, the results can appear simple in hindsight, but only in hindsight. Many underlying connections have to be identified before this simple picture can emerge, and that's where the intellectual contribution lies.

Minor comment:
pg22: The typical spelling is Unicode rather than UNICODE.

Reviewer 3

All of my concerns have been adequately addressed, except for, perhaps, the dangling reference to a (now archived) blog post, but this is in the hands of the editor, per journal practice. Under no circumstance should this materially be expunged; the question is whether that material should be brought in (albeit in the authors' own words) as an appendix.

Reviewer 2:

p. 27 ""$\alpha$ is nonempty, mening it has at least two different elements".
Change "nonempty" to (I presume) "nontrivial".

Thank you for catching that. The paper now reads: “The only requirement is that α is nontrivial, meaning it has at least two different elements.” at the bottom of page 10

p. 34. "Automatic theorem provers ... couldn't address proofs in advanced mathematics". There is AFAIK a single significant exception to that viz. the Robbins conjecture, proven by William McCune an ATP in 1996 --- as far as I know, still, 27 years later, the only mathematical theorem first proved by an ATP.
https://en.wikipedia.org/wiki/Robbins_algebra
But it's up to the authors whether they consider this worth mentioning.

Thank you for the suggestion. This is worth mentioning. This sentence now reads: “But historically, automated theorem provers (ATPs) made progress on narrow classes of problems (e.g., problems in first-order logic [78 ]) but couldn’t address proofs in advanced math (except when problems are described in such simple terms, like the Robbins Conjecture [79]).“ p. 34



Some of the references need work. References like "Alfred North Whitehead and Bertrand Russell. Principia mathematica by alfred north whitehead and bertrand
russell. 1963."; "David Hilbert. Mathematical problems. 1902." "Stanislas Polu ... Formal
mathematics statement curriculum learning, 2022" are really not adequate.

Thank you for catching these. The three citations have been updated.
The first reference now reads: Alfred North Whitehead and Bertrand Russell. Principia Mathematica. Cambridge University Press, 1997.
The second reference now reads: David Hilbert. Mathematical problems. Bulletin of the American Mathematical Society, 8(10):437–479, 1902.
The third reference now reads: Stanislas Polu, Jessenbsp; Michael Han, Kunhao Zheng, Mantas Baksys, Igor Babuschkin, and Ilya Sutskever. Formal mathematics statement curriculum learning. In 2023 International Conference on Learning Representations.

We cleaned up several other references, as well.


Reviewer 1:

pg22: The typical spelling is Unicode rather than UNICODE.
Thank you for catching that. The word has been changed to Unicode

08-Nov-2023

Dear Dr Josephson:

Manuscript ID: DD-ART-04-2023-000077.R2
TITLE: Formalizing Chemical Physics using the Lean Theorem Prover

Thank you for submitting your revised manuscript to Digital Discovery. I am pleased to accept your manuscript for publication in its current form.

You will shortly receive a separate email from us requesting you to submit a licence to publish for your article, so that we can proceed with the preparation and publication of your manuscript.

You can highlight your article and the work of your group on the back cover of Digital Discovery. If you are interested in this opportunity please contact the editorial office for more information.

Promote your research, accelerate its impact – find out more about our article promotion services here: https://rsc.li/promoteyourresearch.

If you would like us to promote your article on our Twitter account @digital_rsc please fill out this form: https://form.jotform.com/213544038469056.

By publishing your article in Digital Discovery, you are supporting the Royal Society of Chemistry to help the chemical science community make the world a better place.

With best wishes,

Dr Joshua Schrier
Associate Editor, Digital Discovery


******
******

Please contact the journal at digitaldiscovery@rsc.org

************************************

DISCLAIMER:

This communication is from The Royal Society of Chemistry, a company incorporated in England by Royal Charter (registered number RC000524) and a charity registered in England and Wales (charity number 207890). Registered office: Burlington House, Piccadilly, London W1J 0BA. Telephone: +44 (0) 20 7437 8656.

The content of this communication (including any attachments) is confidential, and may be privileged or contain copyright material. It may not be relied upon or disclosed to any person other than the intended recipient(s) without the consent of The Royal Society of Chemistry. If you are not the intended recipient(s), please (1) notify us immediately by replying to this email, (2) delete all copies from your system, and (3) note that disclosure, distribution, copying or use of this communication is strictly prohibited.

Any advice given by The Royal Society of Chemistry has been carefully formulated but is based on the information available to it. The Royal Society of Chemistry cannot be held responsible for accuracy or completeness of this communication or any attachment. Any views or opinions presented in this email are solely those of the author and do not represent those of The Royal Society of Chemistry. The views expressed in this communication are personal to the sender and unless specifically stated, this e-mail does not constitute any part of an offer or contract. The Royal Society of Chemistry shall not be liable for any resulting damage or loss as a result of the use of this email and/or attachments, or for the consequences of any actions taken on the basis of the information provided. The Royal Society of Chemistry does not warrant that its emails or attachments are Virus-free; The Royal Society of Chemistry has taken reasonable precautions to ensure that no viruses are contained in this email, but does not accept any responsibility once this email has been transmitted. Please rely on your own screening of electronic communication.

More information on The Royal Society of Chemistry can be found on our website: www.rsc.org