Towards high-resolution X-ray scattering as a probe of electron correlation

Abstract

X-ray scattering cross sections are calculated using a range of increasingly correlated methods: Hartree–Fock (HF), complete active space self-consistent field (CASSCF), Monte Carlo configuration interaction (MCCI), and full configuration interaction (FCI). Even for the seemingly straightforward case of ground state Ne, the accuracy of the total scattering is significantly better with a more correlated wavefunction. Scanning the bond distance in ground state CO shows that the total scattering signal tracks the multireference character. We examine the convergence of the elastic, inelastic, and total scattering of O₃. Overall, the inelastic and total components are found to be the most sensitive to the strength of correlation. Our results suggest that highly accurate measurement of X-ray scattering could provide a sensitive probe of pair-wise correlation between electrons.

---

1 Introduction

X-ray and electron scattering have had great impact on our understanding of the structure of matter.^1–3 In X-ray crystallography, the relationship between the arrangement of the atoms in the unit cell and the elastic scattering measured in the Bragg peaks has enabled remarkable progress in structure determination.^4–6 The corresponding measurements in the gas phase appeared early, with important contributions made by Debye and colleagues, who established the widely-used independent atom model (IAM).^7,8 Electrons have larger cross sections compared to photons, which allowed gas-phase electron scattering to develop into an important technique for the structure determination of free molecules.⁹ Recently, the emergence of X-ray free-electron lasers (XFELs) has spurred a revival of gas-phase scattering using X-rays,^10–13 not the least in the domain of ultrafast X-ray scattering.^14–20 The high intensity and repetition rate at next-generation XFELs opens the prospect of accurate scattering measurements with excellent signal-to-noise on isolated and unperturbed targets in the gas phase.

Experiments capable of probing quantitatively the intriguing relationship between total scattering and electron correlation, originally pointed out by Bartell and Gavin^21,22 and illustrated in Fig. 1, may thus be approaching. It is therefore pertinent to develop the theoretical and computational tools to analyse such experimental data and to examine the relationship between electronic structure calculations and scattering signals, especially given the potential for new experimental benchmarks. Computational methods that calculate scattering cross sections from electronic wavefunctions can now efficiently provide elastic, inelastic, and total scattering cross sections for both electronic ground and excited states.^23–36 Notably, the accuracy of the cross section depends crucially on the quality of the underlying electronic wavefunction and therefore on the electronic structure method from which it is derived.

Fig. 1 Schematic illustration of the importance of electron correlation in X-ray scattering, with k₀ and k_s being the incoming and outgoing wave vectors of the incident and scattered X-ray photons, respectively. See Section 2.1 for details.

In the following, we examine the effect of electron correlation on scattering cross sections and demonstrate that highly correlated methods are necessary to match the anticipated accuracy of future X-ray scattering experiments. One of the tools employed for this purpose is Monte Carlo configuration interaction (MCCI), which is capable of converging close to the benchmark of full configuration interaction (FCI), but with a much more compact expansion and at significantly lower computational cost.^37,38 As a result, highly accurate total scattering cross sections can be obtained that more rigorously account for the effect of strong electron correlation in atoms and small molecules.

2 Theory

2.1 Scattering

The non-resonant scattering of hard X-rays by an atom or molecule can be described by first-order time-dependent perturbation theory and quantum electrodynamics.^39,40 If the system is in the stationary electronic ground state, the expression for the experimentally observable differential X-ray scattering cross section, dσ/dΩ, derived within this framework reads,^41,42where Ω is the solid angle, (dσ/dΩ)_Th the Thomson scattering cross section of a free electron, and |Ψ₀〉 the electronic ground state. In this treatment, we treat the nuclear degrees of freedom as frozen.^29,41,42 Moreover, is the scattering operator with q = k_s − k₀ the momentum transfer vector where the wave vectors of the incident and scattered photons are k₀ and k_s, while r_n is the coordinate of an electron with index n, and N_e is the number of electrons in the system. The matrix element in eqn (1) implies integration over all electronic coordinates and can be expressed as,where S_total(q) corresponds to the total, i.e., energy-integrated scattering, |Ψ_f〉 are the energetically accessible electronic states of the system, and where f = 0 gives the elastic and f ≠ 0 the inelastic X-ray scattering components. The quantity S_total(q) can be obtained via a Fourier transform of the ground-state two-electron density, Γ₀(r₁,r₂),The two-electron density, Γ₀(r₁,r₂), carries information about the probability of the correlated occurrence of a pair of electrons at coordinates r₁ and r₂ and eqn (3) thus implies that electron–electron correlation is important for total scattering.^21,22,43 Correspondingly, the elastic component of the scattering signal, S_el(q), is proportional to the |〈Ψ₀|L̂(q)|Ψ₀〉|² term in the sum in eqn (2), which corresponds to the absolute-squared Fourier transform of the ground-state one-electron density, ρ₀(r),where F(q) is known as the molecular form factor and r are the coordinates of an electron. It is common to define the total inelastic scattering, S_inel(q), as the difference between the total and the elastic scattering,

S_inel(q) = S_total(q) − S_el(q). (5)

In order to correctly benchmark the different components of X-ray scattering, is important to note that their limiting behaviour with respect to the norm of the momentum transfer vector, q = |q|, are F(0) = S_inel(∞) = N_e and F(∞) = S_inel(0) = 0.

As mentioned in the Introduction, a key difference between crystalline and gas-phase targets is the absence of a lattice that coherently amplifies the elastic signal in the latter. This has profound consequences. While elastic scattering in crystallography stems predominantly from the electron density of the target (eqn (4)), the total scattering measured in the gas phase with energy-integrating detectors is sensitive to static and dynamic correlation via the two-electron density (eqn (3)), which incorporates all electron–electron correlations of the system.

2.2 Multiconfigurational wavefunctions

Due to the above-mentioned importance of electron correlation, we anticipate that accurate scattering cross sections will require post Hartree–Fock (HF) methods. Multiconfigurational methods such as complete active space self-consistent field (CASSCF) are capable of capturing a large fraction of the electron correlation⁴⁴ but fall short of full configuration interaction (FCI), which is exact within the limits of a given basis. In order to quantify the effect of electron correlation on gas-phase X-ray scattering, we compare HF and CASSCF with the highly correlated MCCI method. In each case, the results are benchmarked against truncated CI or, when possible, FCI. While MCCI, alongside FCI and multireference CI (MRCI), is able to capture a great portion of the static and dynamic correlation, methods such as CASSCF often include the latter to a smaller degree, with the quality of the results strongly dependent on the selected active space.

All these methods construct the wavefunction as a linear combination of Slater determinants,

where the c_i are the configuration interaction coefficients, N_SD the number of Slater determinants of the expansion, and |Φ^SD_i〉 the Slater determinants. Each Slater determinant in eqn (6) is an anti-symmetrised Hartree product of spin–orbitals, ϕ_j(r), where r is the electron coordinate.

In contrast to single-reference methods such as HF, coupled cluster, or Møller–Plesset perturbation theory, FCI considers all possible determinants within a given basis to build the wavefunction. The number of Slater determinants therefore grows combinatorially with the size of the basis. Complete active space (CAS), restricted active space (RAS), and generalised active space (GAS) self-consistent field (SCF) circumvent this problem by restricting the number of occupied orbitals and therefore the number of Slater determinants. These methods are affordable for medium-sized systems, but the selection of a good active space requires experience and chemical intuition. In contrast, MCCI provides an automated, unbiased, and efficient truncation of the FCI expansion by stochastically selecting the smallest number of configurations needed to achieve the desired value of energy convergence and does not require extensive benchmarking to confirm that the active space produces reliable results. The number of configurations included in the MCCI expansion will depend on the cut-off value, which relates to the retention of configurations based on their coefficients. The smaller the cut-off, the more configurations are included, and the closer the result is to FCI. Overall, the flexibility, accuracy, and adaptability of MCCI permits the prediction of exceptionally accurate scattering cross sections.

2.3 One- and two-electron densities

Following eqn (3) and (4), the calculation of elastic and total scattering cross sections requires one- and two-electron densities. These can be calculated directly from the multiconfigurational wavefunction in eqn (6). The two-electron density required for the calculation of the total scattering in eqn (3) is,where γ_klmn are elements of the reduced two-electronic density matrix that are calculated using the configuration interaction coefficients from the multiconfigurational wavefunction and related to the spin–orbitals with indices k, l, m, and n.⁴⁵ The N_orb in eqn (7) is the number of occupied spin–orbitals. This expression can also be used to obtain the two-electron transition density between different electronic states for the simulation of coherent mixed scattering.^42,46

Analogously, the one-electron density required to calculate elastic scattering, eqn (4), can be constructed as,

where D_kl are elements of the reduced one-electron density matrix. They are directly related to the occupation of each spin–orbital and can be easily calculated knowing the Slater determinants involved in the wavefunction.

3 Results and discussion

To demonstrate the influence of the multireference character of the wavefunction on the scattering cross section, we consider three systems with increasing strength of correlation in their electronic ground state: Ne, CO, and O₃. We use the 6-31G* basis set for Ne and the 6-31G basis set for CO and O₃ throughout to ensure consistency across all methods, up to the computationally demanding FCI reference calculations. In each case, the active spaces for CASSCF and complete active space configuration interaction (CASCI) have been chosen to account for a full-valence description, closing the appropriate orbitals to match the FCI/CISDTQ references. The convergence of scattering cross sections with respect to the basis sets has been analysed elsewhere.^28,30 As mentioned before, we do not consider nuclear motion⁴² and the nuclei are hence treated as frozen, with no integration over nuclear coordinates involved.²⁹ Finally, the results are rotationally averaged, as appropriate when considering static ground-state scattering.

In the following, the computational results are presented and analysed in terms of the percent difference scattering,

where q is, again, the norm of the momentum-transfer vector, S_κ(q) is the cross section we wish to evaluate (κ is total, elastic, or inelastic), and S^ref_κ(q) is the corresponding reference cross section calculated using either FCI or truncated CI. We also evaluate the integrated absolute percent difference,where we integrate the percent difference on the interval [q_min, q_max] = [0, 20] Å⁻¹. We quantify the degree of multireference character of the wavefunction using the metric ς_MR,^47,48where N_SD is the number of Slater determinants and c_i are the configuration interaction coefficients introduced in eqn (6). ς_MR takes values in the range [0, 1), where ς_MR = 0 corresponds to a single determinant and ς_MR → 1 indicates that many configurations contribute and a multireference method may be needed.† Finally, the degree of convergence of the electronic structure calculations is quantified by the percent recovered correlation energy,where E_HF is the Hartree–Fock energy, E is the energy calculated using the method of choice, and E_corr is the correlation energy defined as E_corr = E_HF − E_ref, with E_ref the most accurate electronic energy available.

3.1 The Ne atom

We use the closed-shell ground state Ne atom to evaluate the effect of electron correlation on scattering for a system with small multireference character. Fig. 2a shows the percent difference |%ΔI_κ| for the total X-ray scattering cross section calculated at different levels of theory, with FCI(1-Frozen)/6-31G* used as the reference. The maximum percent difference for HF/6-31G* and CASSCF(8,8)/6-31G* is ∼2.1%, while for MCCI(0.001)(1-Frozen)/6-31G*‡ it is one order of magnitude more accurate with ∼0.1%. This means that even for Ne, which has weak multireference character (ς_MR = 0.08 for the FCI(1-Frozen)/6-31G* wavefunction), a highly correlated wavefunction makes a significant difference in the convergence of the total X-ray scattering cross section. We note that is also true if the coupled cluster singles and doubles (CCSD) two-electron density is used.⁴⁹ This density can be calculated with the PySCF package,⁵⁰ and has an error comparable to MCCI(0.001). The similar magnitude of the HF and CASSCF errors suggests that their deviation from the FCI reference stems primarily from their lack of dynamic correlation, well described by both CCSD and MCCI.

Fig. 2 Percent differences for total scattering, Δ%I_tot(q), calculated using eqn (9) for the following systems in their electronic ground states: (a) Ne, (b) CO (equilibrium geometry, R_eq = 2.13 a₀), (c) CO (stretched geometry, R = 4 a₀), (d) O₃ (equilibrium geometry). FCI(1-Frozen)/6-31G* and FCI(2-Frozen)/6-31G are used as the references for Ne and CO, while for O₃ we used CISDTQ(3-Frozen)/6-31G as the reference. Results are shown for HF, CASSCF, and MCCI(0.001)(1-2-3 Frozen). Further details are provided in the text.

Integrated absolute percent differences, the percent recovered correlation energy, and the numbers of configurations included in each calculation are given in the first column of Table 1. The table shows a close relation between the quality of the results, as measured by the integrated |%ΔI_tot|, and the percent recovered correlation energy %E_corr. The small number of Slater determinants included in the MCCI wavefunction (291) compared to FCI (>10⁵), demonstrates the ability of MCCI to capture a significant part of the electron correlation with a compact expansion.

Table 1 Results as a function of the level of theory for Ne, CO (at equilibrium R = 2.13 a₀ and stretched to R = 4 a₀), and O₃. The table includes the integrated absolute total scattering percent differences, |%ΔI_tot| in eqn (10), the percent recovered correlation energy, %E_corr in eqn (12), and the number of Slater determinants with a nonzero contribution N_SD in the MCCI expansion (a 10⁻¹⁵ cutoff has been used to choose these configurations), which considers 1, 2 and 3 frozen orbitals for Ne, CO and O₃ respectively. The total number of configurations has been included in parenthesis for CAS and CASCI methods. The reference is FCI(1-Frozen)/6-31G* for Ne, FCI(2-Frozen)/6-31G CO, and CISDTQ (3-Frozen)/6-31G for O₃

Method	Ne			CO (R = 2.13 a0)			CO (R = 4.00 a0)			O3
	|%ΔItot|	%Ecorr	N SD	|%ΔItot|	%Ecorr	N SD	|%ΔItot|	%Ecorr	N SD	|%ΔItot|	%Ecorr	N SD
HF	10.1	0	1	5.41	0	1	8.46	0	1	4.09	0	1
CAS	8.33	49	873(4900)	3.54	65	1223(3136)	4.18	69	1315(3136)	2.08	53	12 111(48 400)
CASCI	—	—	—	2.42	32	1307(3136)	11.6	53	1191(3136)	2.40	54	6079(48 400)
MCCI(0.001)	0.65	98	291	0.57	95	1757	0.46	96	3934	0.90	89	6160
CCSD	0.75	98	—	0.45	96	—	6.46	46	—	—	—	—
FCI/CISDTQ	—	100	125 861	—	100	4 777 056	—	100	4 777 056	—	100	6 526 866

---

3.2 The CO molecule

3.2.1 Equilibrium and stretched geometries. The ground state of CO has significantly larger multireference character than Ne. Although it can, in principle, be described using active-space based methods such as CASSCF, it exhibits non-negligible dynamic correlation, which requires highly correlated wavefunctions, especially for stretched geometries. The multireference character varies strongly with the bond length and in the following we consider the equilibrium geometry (R_eq ≈ 2.13 a₀) and a stretched geometry (R = 4.00 a₀) that is close to dissociation and where the wavefunction is most strongly correlated in this work.

Comparing the total scattering percent differences at R_eq with the FCI(2-Frozen)/6-31G reference, shown in Fig. 2b, we unsurprisingly see that the HF/6-31G calculation displays the largest maximum error at ∼1.9%. This is followed by CASSCF(10,8)/6-31G, which performs better with a maximum error of ∼1.1%, reflecting that the multiconfigurational CASSCF wavefunction recaptures a sizeable portion of the static correlation absent in the HF wavefunction. As for Ne, MCCI(0.001)(2-Frozen)/6-31G outperforms the other methods with a maximum error of ∼0.2%, getting quite close to the FCI reference with only a small fraction of the configurations of FCI (see Table 1). CCSD wavefunctions also perform well with a deviation comparable to the one seen in MCCI, reflecting the small multireference character of equilibrium CO.

Fig. 2b also includes a CASCI(10,8)/6-31G calculation where, in contrast to CASSCF, the HF orbitals have not been re-optimised. Possibly thanks to a larger number of significantly contributing Slater determinants, N_SD = 1307 compared to N_SD = 1223 for CASSCF, this calculation achieves better convergence of the scattering cross section, with a maximum error of ∼0.6%. It is notable that, although the cross section improves, the energy does not. This is to be expected, since it is the purpose of the orbital re-optimisation in CASSCF to aid the optimisation of the energy. The integrated absolute percent differences are listed in Table 1 alongside the percent recovered correlation energy and the number of Slater determinants.

Examining stretched CO, shown in Fig. 2c, we see similar overall trends as for the molecule's equilibrium structure. However, in this case, CCSD fails to account for the stronger multireference character and its error is comparable to HF. Nevertheless, the comparison between CASSCF and CASCI displays an interesting difference. As before, the orbital-optimisation decreases the energy but the orbital-frozen calculation yields worse scattering cross sections compared to CASSCF. The cause of this discrepancy is most likely the smaller number of non-zero Slater determinants in the CASCI calculation (Table 1). However, one should be cautious not to draw too general conclusions about the link between the number of Slater determinants in the wavefunction and the convergence of the scattering cross section. The latter is clearly also a question of including the correct configurations, as illustrated by the fact that the MCCI calculations manage to capture a large portion of the electron correlation and converge the scattering cross sections close to the FCI reference with significantly more compact expansions.

3.2.2 Scanning the CO bond length. To examine the influence of the strength of electron correlation on the total X-ray scattering cross sections in greater detail, calculations were carried out for a series of increasing bond lengths of CO. The multireference character increases as the bond stretches, as seen by the values of ς_MR included in Table 2. The table also contains the integrated absolute percent difference of the scattering signal, the maximum error, the number of configurations in the MCCI wavefunction, and the respective data for HF for comparison. The bond length is scanned over the range from 1.5 a₀ to 4.0 a₀, with a HF/6-31G and a MCCI(0.0005)/6-31G calculation carried out at each bond length and a FCI/6-31G calculation taken as the reference. MCCI expectedly outperforms HF, with the failure of HF increasing as the bond stretches. At the same time, MCCI is forced to include larger numbers of Slater determinants to achieve the same convergence for all bond lenghts, with N_SD increasing from 2398 at the minimum to 8059 at the maximum bond length. However, despite the increasing effort required to converge the calculations, the maximum errors and |%ΔI_tot| remain fairly constant, demonstrating the flexibility and adaptability of MCCI.

Table 2 Results for different bond lengths R of ground-state CO using MCCI(0.0005)(2-Frozen)/6-31G, with FCI(2-Frozen)/6-31G calculations taken as the reference. The table includes the number of configurations N_SD included in the MCCI expansion, the multireference character, ς_MR, defined in eqn (11), the integrated absolute percent difference for total scattering, |%I_tot|, defined in eqn (10), and the maximum percent difference thereof, max|%I_tot|. The percent recovered correlation energy (eqn (12)) is %E_corr ≈ 97% for the MCCI calculations and zero by definition for HF

R (a0)	MCCI				HF
	N SD	ς MR	|%ΔItot|	max(|%ΔItot|)	|%ΔItot|	max(|%ΔItot|)
1.5	2398	0.09	0.14	0.07	2.63	1.32
2.13	3997	0.17	0.17	0.11	2.87	1.79
2.5	5418	0.25	0.18	0.12	3.11	2.11
3.0	6797	0.43	0.19	0.10	3.62	2.80
3.5	7967	0.85	0.20	0.12	3.77	3.81
4.0	8059	0.95	0.19	0.13	5.73	4.79

---

Fig. 3 shows the percent difference as a function of q with FCI/6-31G as the reference for a sequence of increasing bond lengths that correspond to the calculations featured in Table 2. Overall, the results at different bond lengths are quite similar, with the values oscillating between 0.02% and 0.12%. The maximum errors occur for values of q between 2.0 Å⁻¹ and 4.0 Å⁻¹, where the electron correlation manifests most strongly. In this region of q, the scattering is to a large part inelastic but has not yet reached the large-q limit, S_inel(∞) = N_e. We note that the reciprocal relationship between real and momentum space causes the maxima of the scattering signal to shift to smaller values of q with increasing bond length. Also, the second peak in the percent difference relates to the appearance of a second maximum in the scattering signal when the atoms are separated by more than R = 3 a₀.

Fig. 3 Percent differences for total X-ray scattering from CO calculated at a sequence of bond lengths R ranging from 1.5 to 4.0 a₀ (solid blue lines). The cross sections are calculated using MCCI(0.0005)(2-Frozen)/6-31G with FCI(2-Frozen)/6-31G used as the reference. (To aid comparison, the result from the preceding R value is included as a faint red line in the panels for R > 1.5 a₀.)

3.3 The O₃ molecule

3.3.1 Total scattering. The final system considered is O₃. Ozone is well known for having a strong multireference character in its electronic ground state. Although it can be reasonably well described by statically correlated methods such as CASSCF,⁵¹ highly-correlated methods are expected to achieve significantly better results. Fig. 2d and Table 1 show the total X-ray scattering percent differences and other results including the notably high multireference character of ς_MR = 0.38 for the molecule in its equilibrium geometry. The curves show the same trends as in the previous examples. Again, as appears to be the case for systems with strong dynamic correlation, the HF and CASSCF results have large errors, though CASSCF and CASCI perform better. As a consequence, the cross sections obtained with these methods differ greatly from the CI singles, doubles, triples, and quadruples (CISDTQ) reference. Also, at an MCCI cut-off 0.001 the error is high compared to the other calculations presented in this article, reflecting that the large multireference character of O₃ requires more Slater determinants and thus a smaller cut-off value for the scattering cross sections to converge. In the next two sections, we use O₃ to systematically examine the convergence of scattering cross sections for the different components of X-ray scattering and as a function of the cut-off in the MCCI calculations.

3.3.2 Convergence of scattering components. We begin by examining how the degree of electron correlation affects the convergence of the elastic, inelastic, and total scattering cross sections. Fig. 4 shows the percent difference for each of these components for a range of MCCI cut-offs freezing 3 orbitals with CISDTQ(3-Frozen)/6-31G used as the reference. A comparison of the total and elastic scattering in Fig. 4a and Fig. 4b, respectively, manifests the importance of electron correlation for total scattering. The percent differences are significantly larger and the convergence with decreasing cut-offs is slower than for elastic scattering. This is further confirmed by examination of the inelastic component, calculated as the difference between the total and elastic scattering (eqn 5), shown in Fig. 4c. The inelastic cross sections clearly have the largest percent difference errors. It is thus the inelastic component that is most strongly affected by the electron correlation and correspondingly displays the slowest convergence with respect to the MCCI cut-off.

Fig. 4 Percent differences as a function of MCCI(3-Frozen) cut-offs for (a) total, (b) elastic, and (c) inelastic X-ray scattering cross sections for ground-state O₃ in its equilibrium geometry. The percent differences Δ%I_{tot/elas/inel}(q) are calculated using eqn (9) with CISDTQ(3-Frozen)/6-31G taken as the reference. The percent differences at q ∼ 0 Å⁻¹ and q ∼ 15 Å⁻¹ have been filtered out as they are highly sensitive to numerical errors. Also, the area between 0 < q < 0.5 Å⁻¹ has been shaded to indicate that numerical instabilities are likely to occur in this region. Panel (d) shows the integrated absolute percent differences |%ΔI_tot| with the corresponding ΔE in electronvolts.

We note, however, that the relative errors of the inelastic scattering cross sections at low values of q in Fig. 4c do not provide reliable information about the signal's convergence. These errors should be considered largely a numerical artefact that stems from the division by a small reference value since inelastic scattering is weak at low momentum transfer (at q = 0, it is exactly zero). This is different for elastic and total scattering, which are largest at q = 0. Nevertheless, even for elastic and total scattering, it is of limited use to evaluate the performance of an electronic structure method in the low momentum-transfer region. At q = 0, they are proportional to the squared number of the molecule's electrons, not affected by electron correlation, and thus, in principle, identical for all methods. To indicate that one should interpret the respective relative errors of the inelastic component with caution, the region of low q in Fig. 4c is shaded. The effect of electron correlation reliably manifests itself in the region of intermediate momentum transfer, roughly between 1.5 Å⁻¹ and 6.0 Å⁻¹.

3.3.3 Convergence with MCCI cut-off. We now examine the effect of the MCCI(3-Frozen) cut-off value on the cross sections, using O₃ as an example. The cut-off value defines the degree of energy convergence and implies the necessary tradeoff between accuracy and speed of the calculations. The results are displayed in Fig. 4 and in Table 3. Fig. 4a shows the percent differences for the total scattering cross sections. The error decreases dramatically for cut-offs smaller than 0.001 and the gains for 0.00005 over 0.0001 are small. Fig. 4a also demonstrates that large cut-offs (0.01 and 0.005) result in comparatively large errors, close to the CASSCF results. These trends are repeated for elastic (Fig. 4b) and inelastic (Fig. 4c) scattering. We note that the comparison of the errors of the total (Fig. 4a) and elastic (Fig. 4b) scattering shows that the elastic errors are significantly smaller, emphasising yet again the important role of electron correlation for total scattering. However, the decreases in the errors as a function of decreasing cut-off follow a similar trend for all three components of the scattering signal in Fig. 4a–c.

Table 3 Integrated percent differences |%ΔI_x| for total (tot), elastic (elas), and inelastic (inel) scattering as a function of the MCCI(3-Frozen) cut-off in ground-state equilibrium O₃. The integrated percent differences are re-normalised by their value for MCCI(0.01)(3-Frozen) for clarity. Reference values are taken from CISDTQ(3-Frozen)/6-31G. The percent recovered correlation energy %E_corr is used as a proxy for ab initio convergence

Cutoff	|%ΔItot|	|%ΔIelas|	|%ΔIinel|	%Ecorr	N SD
0.01	1.00	0.44	0.63	50	215
0.005	0.57	0.20	0.45	90	786
0.001	0.16	0.05	0.19	92	6160
0.0005	0.13	0.05	0.14	98	12 961
0.0001	0.05	0.02	0.04	99	149 850
0.00005	0.02	0.01	0.02	100	393 620
CISDTQ	—	—	—	100	6 525 866

---

Finally, we examine the relationship between the convergence of the calculated energies and the scattering signals. Fig. 4d shows the integrated absolute percent differences for total X-ray scattering (red bars) alongside the corresponding correlation energies for the MCCI calculations (blue bars) for decreasing cut-offs. The convergence of the energy and the scattering cross sections follows the same trend, as seen previously. The rate of convergence is also similar, with both quantities converging at the same rate towards a negligible energy and scattering difference beyond the 0.001 cut-off.

3.4 Further examinations of MCCI

3.4.1 Pre-optimisation. The standard variant of MCCI selects among the Slater determinants constructed from the occupied and virtual HF orbitals. The number of Slater determinants necessary to achieve a given degree of convergence can therefore be quite large and sometimes even intractable. One approach to tackle this is to pre-optimise the orbitals with a multiconfigurational SCF method such as CASSCF and then use the optimised orbitals instead of the canonical HF orbitals. This can reduce the total number of required Slater determinants and, while the orbital optimisation is expensive, the reduction in N_SD can speed up the overall calculation significantly.

The orbital optimisation is performed within a given active space and for a chosen set of electronic states. Here, we use equilibrium-geometry CO and O₃ in their electronic ground states to investigate the effect of orbital pre-optimisation on the scattering cross sections. We calculate the total scattering cross sections and compare them to FCI(3-Frozen)/6-31G and CISDTQ(3-Frozen)/6-31G references, respectively, using CAS(14,10) pre-optimised orbitals in each case. The results for a 0.0001 MCCI cut-off are shown in Fig. 5. A notable improvement is achieved in the cross sections when pre-optimised orbitals are used. The calculations of the two-electron density and of the cross sections are faster, since the pre-optimised orbital approach reduces the number of Slater determinants required. Likewise, the energies improve slightly when the pre-optimisation is used. The results are summarised in Table 4. We note, however, that this advantage is diminished by the additional computational overhead of the pre-optimisation.

Fig. 5 Percent differences for total scattering cross sections calculated using MCCI(0.0001) with and without orbital pre-optimisation for (a) CO and (b) O₃. Both molecules are in their electronic ground states and equilibrium geometries. FCI(2-Frozen)/6-31G is used as the reference for CO and CISDTQ(3-Frozen)/6-31G for O₃.

Table 4 Comparison between MCCI(0.0001) (with 2 frozen orbitals for CO and 3 for O₃) calculations using a configuration space derived from the standard HF orbitals and from pre-optimised (preopt) orbitals for the CO and O₃ in their electronic ground states and equilibrium geometries. Total X-ray scattering integrated percent differences |%ΔI_tot|, percent recovered correlation energy %E_corr, and number of configurations N_SD are shown using FCI(2-Frozen)/6-31G and CISDTQ(3-Frozen)/6-31G calculations as the references

Method	CO			O3
	|%ΔItot|	%Ecorr	N SD	|%ΔItot|	%Ecorr	N SD
MCCI(0.0001)	0.08	99.5	25960	0.28	98	149 850
MCCI(0.0001)/preopt	0.07	99.6	20495	0.20	99	111 180

---

3.4.2 MCCI stochasticity. MCCI is a stochastic method and the selected Slater determinants will vary from one calculation to another. We examine how this stochastic behaviour affects the cross sections and consider whether further statistical analysis should be prescribed as part of the MCCI procedure, e.g., carrying out multiple calculations at each cut-off and using the average and the standard deviation. We carry out this test on a set of ten MCCI(0.001)(3-Frozen) calculations for O₃ in its electronic ground state at equilibrium geometry. The results for the total cross sections are analysed statistically, particularly looking at the variability as a function of the absolute momentum transfer q.

Fig. 6 shows the variability of the MCCI cross sections at different values of q. The standard error E_st(q) is defined as,

where I_std.dev.(q) is the standard deviation for the N_s samples considered. The variability is greatest at intermediate values of q, where the accuracy of the wavefunction plays an important role, while it is negligible for very small and large values of q. However, the magnitude of the percent error is extremely small with maxima around E_st = 0.001%, confirming that it is not necessary to carry out a statistical analysis when using MCCI. Looking at the mean average integrated value of the scattering, 708.88, and the standard deviation of 0.0047, the stability of the MCCI results becomes even clearer, with the standard deviation amounting to only 0.0001% of the integrated signal. These findings are in agreement with previous studies that examined the second hyperpolarisability,⁴⁸ X-ray emission,⁵² and energies.⁵³

Fig. 6 The standard error E_st as a function of the momentum transfer vector q calculated using eqn (13). A sampling space of N_s = 10 MCCI(0.001)(3-Frozen)/6-31G calculations is used.

4 Conclusion

We have demonstrated that electron–electron correlation influences X-ray scattering cross sections, with stronger effects seen in the inelastic and total components. In order to accurately converge the total scattering signal, correlation must be accounted for even in comparably simple systems such as the Ne atom. In this context, MCCI emerges as a powerful method, capable of reproducing FCI results at a fraction of their computational cost. Notably, MCCI delivers compact and highly correlated wavefunctions that yield accurate scattering cross sections even in challenging cases. Possible future work includes examining if there are general patterns in how the approximations involved in different ab initio methods translate into scattering signals for different values of the momentum transfer q and a detailed examination of basis set effects in MCCI. However, the present results clearly demonstrate that MCCI is a strong option for calculating highly accurate X-ray scattering cross sections.

For elastic scattering, mean-field methods such as HF already offer a significant improvement over the independent atom model (IAM),^26,28,54 quite possibly delivering all the accuracy needed to reproduce accurate measurements of the elastic signals in X-ray crystallography, as in the burgeoning field of quantum crystallography.⁵⁵ Given that IAM form factors are occasionally used outside the realm of crystallography, one might consider updating tabulated non-relativistic IAM inelastic scattering functions⁵⁶ using post-HF methods, which would constitute a potential extension of the work presented in this article.

Importantly, for gas-phase experiments that measure total rather than elastic scattering, electron correlation should ideally be included via methods such as for instance MCCI or density matrix renormalization group (DMRG).^57,58 One context of growing importance is ultrafast X-ray scattering.^{42,46,59–64} As demonstrated recently, the accuracy achievable in gas-phase scattering is now sufficient to distinguish different electronic states.²⁰ Considering the progress towards high-repetition rate and high-energy modes at the European XFEL and the next-generation LCLS-II, it is reasonable to anticipate that measurements on static molecules in the gas phase could achieve large values of q_max and experimental accuracy on the order of 0.1% or less. Such measurements would provide exciting opportunities to probe the electronic structure of atoms and molecules in a manner complementary to spectroscopy and with a direct link between the observable and the two-electron density. Conceivably, future experiments may achieve such measurements on rotationally aligned molecules, further increasing the amount of information available in the experimental data.^10,65 The coming years are likely to see rapid progress towards an experimental-evidence based understanding of electron correlation.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

Funding from the Engineering and Physical Sciences Research Council UK (EPSRC) is gratefully acknowledged: EP/V006819 (A. K. and A. M. C.), EP/V049240 (A. K. and M. S.), EP/V006746 (M. J. P. and J. P. C.), EP/P001459 (M. J. P.), and EP/T021675 (M. J. P.). M. J. P. and A. K. also acknowledge support from the Leverhulme Trust (RPG-2020-208). Finally, A. K. acknowledges a Fellowship at the Swedish Collegium for Advanced Studies supported by the Erling-Persson Family Foundation and the Knut and Alice Wallenberg Foundation, as well as support by the Department of Energy, Office of Science, Basic Energy Sciences, under award number DE-SC0020276.

Notes and references

1. M. V. Laue, Nobelprize.org, Nobel Media AB 2014, 1915.
2. W. H. Bragg and W. L. Bragg, Nature, 1913, 91, 557.
3. M. H. Pirenne, The Diffraction of X-Rays and Electrons by Free Molecules, Cambridge University Press, London, 1946.
4. J. D. Watson and F. H. C. Crick, Nature, 1953, 171, 737–738.
5. M. Perutz, Proc. R. Soc. London, Ser. A, 1954, 225, 287.
6. D. C. Hodgkin, J. Kamper, M. Mackay, J. Pickworth, K. N. Trueblood and J. G. White, Nature, 1956, 178, 64 EP.
7. P. Debye, L. Bewilogua and F. Ehrhardt, Phys. Zeits., 1929, 30, 84.
8. P. Debye, Phys. Zeits., 1930, 31, 419–428.
9. I. Hargittai and M. Hargittai, Stereochemical applications of gas-phase electron diffraction: Part A The Electron Diffraction Technique, VCH, New York, 1st edn, 1988.
10. J. Küpper, S. Stern, L. Holmegaard, F. Filsinger, A. Rouzée, A. Rudenko, P. Johnsson, A. V. Martin, M. Adolph, A. Aquila, S. Bajt, A. Barty, C. Bostedt, J. Bozek, C. Caleman, R. Coffee, N. Coppola, T. Delmas, S. Epp, B. Erk, L. Foucar, T. Gorkhover, L. Gumprecht, A. Hartmann, R. Hartmann, G. Hauser, P. Holl, A. Hömke, N. Kimmel, F. Krasniqi, K.-U. Kühnel, J. Maurer, M. Messerschmidt, R. Moshammer, C. Reich, B. Rudek, R. Santra, I. Schlichting, C. Schmidt, S. Schorb, J. Schulz, H. Soltau, J. C. H. Spence, D. Starodub, L. Strüder, J. Thøgersen, M. J. J. Vrakking, G. Weidenspointner, T. A. White, C. Wunderer, G. Meijer, J. Ullrich, H. Stapelfeldt, D. Rolles and H. N. Chapman, Phys. Rev. Lett., 2014, 112, 083002.
11. M. P. Minitti, J. M. Budarz, A. Kirrander, J. Robinson, T. J. Lane, D. Ratner, K. Saita, T. Northey, B. Stankus, V. Cofer-Shabica, J. Hastings and P. M. Weber, Faraday Discuss., 2014, 171, 81.
12. M. Stefanou, K. Saita, D. V. Shalashilin and A. Kirrander, Chem. Phys. Lett., 2017, 683, 300–305.
13. L. Ma, H. Yong, J. D. Geiser, A. Moreno Carrascosa, N. Goff and P. M. Weber, Struct. Dyn., 2020, 7, 034102.
14. M. P. Minitti, J. M. Budarz, A. Kirrander, J. S. Robinson, D. Ratner, T. J. Lane, D. Zhu, J. M. Glownia, M. Kozina, H. T. Lemke, M. Sikorski, Y. Feng, S. Nelson, K. Saita, B. Stankus, T. Northey, J. B. Hastings and P. M. Weber, Phys. Rev. Lett., 2015, 114, 255501.
15. H. Yong, N. Zotev, B. Stankus, J. M. Ruddock, D. Bellshaw, S. Boutet, T. J. Lane, M. Liang, S. Carbajo, J. S. Robinson, W. Du, N. Goff, Y. Chang, J. E. Koglin, M. D. J. Waters, T. I. Sølling, M. P. Minitti, A. Kirrander and P. M. Weber, J. Phys. Chem. Lett., 2018, 9, 6556–6562.
16. B. Stankus, H. Yong, N. Zotev, J. M. Ruddock, D. Bellshaw, T. J. Lane, M. Liang, S. Boutet, S. Carbajo, J. S. Robinson, W. Du, N. Goff, Y. Chang, J. E. Koglin, M. P. Minitti, A. Kirrander and P. M. Weber, Nat. Chem., 2019, 11, 716–721.
17. J. M. Ruddock, N. Zotev, B. Stankus, H.-W. Yong, D. Bellshaw, S. Boutet, T. J. Lane, M. Liang, S. Carbajo, W. Du, A. Kirrander, M. P. Minitti and P. M. Weber, Angew. Chem., Int. Ed., 2019, 58, 6371–6375.
18. J. M. Ruddock, H. Yong, B. Stankus, W. Du, N. Goff, Y. Chang, A. Odate, A. M. Carrascosa, D. Bellshaw, N. Zotev, M. Liang, S. Carbajo, J. Koglin, J. S. Robinson, S. Boutet, A. Kirrander, M. P. Minitti and P. M. Weber, Sci. Adv., 2019, 5, eaax6625.
19. H. Yong, N. Zotev, J. M. Ruddock, B. Stankus, M. Simmermacher, A. M. Carrascosa, W. Du, N. Goff, Y. Chang, D. Bellshaw, M. Liang, S. Carbajo, J. E. Koglin, J. S. Robinson, S. Boutet, M. P. Minitti, A. Kirrander and P. M. Weber, Nat. Commun., 2020, 11, 2157.
20. H. Yong, S. M. Cavaletto and S. Mukamel, J. Phys. Chem. Lett., 2021, 12, 9800–9806.
21. L. Bartell and R. Gavin, J. Am. Chem. Soc., 1964, 86, 3493–3498.
22. L. Bartell and R. Gavin, J. Chem. Phys., 1965, 43, 856–861.
23. A. Debnarova and S. Techert, J. Chem. Phys., 2006, 125, 224101.
24. A. Debnarova, S. Techert and S. Schmatz, J. Chem. Phys., 2010, 133, 124309.
25. A. Debnarova, S. Techert and S. Schmatz, J. Chem. Phys., 2011, 134, 054302.
26. T. Northey, N. Zotev and A. Kirrander, J. Chem. Theory Comput., 2014, 10, 4911–4920.
27. H. J. Suominen and A. Kirrander, Phys. Rev. Lett., 2014, 112, 043002.
28. T. Northey, A. M. Carrascosa, S. Schäfer and A. Kirrander, J. Chem. Phys., 2016, 145, 154304.
29. A. M. Carrascosa, T. Northey and A. Kirrander, Phys. Chem. Chem. Phys., 2017, 19, 7853–7863.
30. A. M. Carrascosa and A. Kirrander, Phys. Chem. Chem. Phys., 2017, 19, 19545–19553.
31. G. Hermann, V. Pohl, J. C. Tremblay, B. Paulus, H.-C. Hege and A. Schild, J. Comput. Chem., 2016, 37, 1511–1520.
32. V. Pohl, G. Hermann and J. C. Tremblay, J. Comput. Chem., 2017, 38, 1515–1527.
33. G. Hermann, V. Pohl and J. C. Tremblay, J. Comput. Chem., 2017, 38, 2378–2387.
34. R. M. Parrish and T. J. Martínez, J. Chem. Theory Comput., 2019, 15, 1523–1537.
35. A. M. Carrascosa, H. Yong, D. L. Crittenden, P. M. Weber and A. Kirrander, J. Chem. Theory Comput., 2019, 15, 2836–2846.
36. N. Zotev, A. Moreno Carrascosa, M. Simmermacher and A. Kirrander, J. Chem. Theory Comput., 2020, 16, 2594–2605.
37. J. Greer, J. Comput. Phys., 1998, 146, 181–202.
38. L. Tong, M. Nolan, T. Cheng and J. Greer, Comput. Phys. Commun., 2000, 131, 142–163.
39. N. E. Henriksen and K. B. Møller, J. Phys. Chem. B, 2008, 112, 558.
40. G. Dixit, O. Vendrell and R. Santra, Proc. Natl. Acad. Sci. U. S. A., 2012, 109, 11636.
41. W. Schülke, Electron Dynamics by Inelastic X-Ray Scattering, Oxford Science Publications, 1st edn, 2007.
42. M. Simmermacher, A. Moreno Carrascosa, N. E. Henriksen, K. B. Møller and A. Kirrander, J. Chem. Phys., 2019, 151, 174302.
43. R. A. Bonham and M. Fink, High Energy Electron Scattering, Van Nostrand Reinhold Company, ACS Monograph, 169 edn, 1974.
44. C. J. Stein, V. von Burg and M. Reiher, J. Chem. Theory Comput., 2016, 12, 3764–3773.
45. J. P. Coe, A. M. Carrascosa, M. Simmermacher, A. Kirrander and M. J. Paterson, J. Chem. Theory Comput., 2022 , submitted.
46. M. Simmermacher, N. E. Henriksen, K. B. Møller, A. Moreno Carrascosa and A. Kirrander, Phys. Rev. Lett., 2019, 122, 073003.
47. J. P. Coe and M. J. Paterson, J. Chem. Theory Comput., 2015, 11, 4189–4196.
48. J. P. Coe and M. J. Paterson, J. Chem. Phys., 2014, 141, 124118.
49. R. J. Bartlett and M. Musiał, Rev. Mod. Phys., 2007, 79, 291–352.
50. Q. Sun, T. C. Berkelbach, N. S. Blunt, G. H. Booth, S. Guo, Z. Li, J. Liu, J. D. McClain, E. R. Sayfutyarova, S. Sharma, S. Wouters and G. K.-L. Chan, Wiley Interdiscip. Rev.: Comput. Mol. Sci., 2018, 8, e1340.
51. S. Keller, K. Boguslawski, T. Janowski, M. Reiher and P. Pulay, J. Chem. Phys., 2015, 142, 244104.
52. J. P. Coe and M. J. Paterson, Theor. Chem. Acc., 2015, 134, 58.
53. J. P. Coe, D. J. Taylor and M. J. Paterson, J. Chem. Phys., 2012, 137, 194111.
54. T. Northey and A. Kirrander, J. Phys. Chem. A, 2019, 123, 3395–3406.
55. Modern Charge-Density Analysis, ed. C. Gatti and P. Macchi, Springer, Netherlands, 1st edn, 2012.
56. J. H. Hubbell, W. J. Veigele, E. A. Briggs, R. T. Brown, D. T. Cromer and R. J. Howerton, J. Phys. Chem. Ref. Data, 1975, 4, 471–538.
57. K. H. Marti and M. Reiher, Z. Phys. Chem., 2010, 224, 583–599.
58. S. Wouters and D. V. Neck, Eur. Phys. J. D, 2014, 68, 1–20.
59. J. M. Budarz, M. P. Minitti, D. V. Cofer-Shabica, B. Stankus, A. Kirrander, J. B. Hastings and P. M. Weber, J. Phys. B, 2016, 49, 034001.
60. B. Stankus, H. Yong, J. Ruddock, L. Ma, A. Moreno Carrascosa, N. Goff, S. Boutet, X. Xu, N. Zotev, A. Kirrander, M. P. Minitti and P. M. Weber, J. Phys. B: At., Mol. Opt. Phys., 2020, 53, 234004.
61. A. Odate, A. Kirrander, P. M. Weber and M. P. Minitti, Adv. Phys. X, 2022 , under submission.
62. A. Kirrander, K. Saita and D. V. Shalashilin, J. Chem. Theory Comput., 2016, 12, 957–967.
63. A. Kirrander and P. M. Weber, Appl. Sci., 2017, 7, 534.
64. M. Simmermacher, N. E. Henriksen and K. B. Møller, Phys. Chem. Chem. Phys., 2017, 19, 19740–19749.
65. J. Yang, V. Makhija, V. Kumarappan and M. Centurion, Struct. Dyn., 2014, 1, 044101.

---

Footnotes

† For FCI the method dependence in the value of ς_MR is removed. However, the value of ς_MR may remain sensitive to the orbitals used.

‡ MCCI(0.001) indicates that the MCCI cutoff is 0.001, while ‘(N-Frozen)’ indicates that the number of frozen orbitals is N.

---