Open Access Article
Jiyuan Zhangab,
Haoyu Zhao*b,
Shuning Caib,
Xueqing Yangb,
Yabin Zhangb,
Bingkai Yuanb,
Junyong Wangb,
Jingzhe Chena,
Pengru Huang*c and
Gaolei Zhan
*bd
aDepartment of Physics, College of Sciences, Shanghai University, Shanghai 200444, P.R. China
bSuzhou Institute of Nano-Tech and Nano-Bionics of the Chinese Academy of Sciences, 398 Ruoshui Road, Suzhou, Jiangsu 215123, P. R. China. E-mail: haoyuzhao2023@sinano.ac.cn; glzhan2022@sinano.ac.cn
cInstitute for Functional Intelligent Materials (I-FIM), National University of Singapore, Singapore 117544, Singapore. E-mail: pengru@nus.edu.sg
dGuangdong Institute of Semiconductor Micro-Nano Manufacturing Technology, Foshan 528225, P.R. China
First published on 19th March 2026
By subtle control over the deposition flux and growth temperature of inorganic dielectric molecular Sb2O3, two kinetic phases and a thermodynamically stabilized phase were successfully obtained on Au(111). Phase transition processes and selective growth of molecular self-assembly were monitored by STM, backed up by DFT calculations. Furthermore, theoretical calculations elucidate the impact of molecular arrangement on the modulation of dielectric properties.
The cubic phase of antimony trioxide is a van der Waals (vdWs) molecular crystal composed of Sb4O6 cages and crystallizes in the Fd
m space group. This cubic phase is known as the α phase. Previous studies have shown that when deposited onto vdWs substrates such as highly oriented pyrolytic graphite (HOPG), MoS2, and WS2, Sb2O3 preserves this intrinsic lattice configuration due to weak molecule–substrate interactions.2,6,8,18,19 Another interesting phase is orthorhombic β-Sb2O3 with a Pccn space group. It was predicted theoretically that a transition from the indirect bandgap to the direct bandgap would occur if the α phase was transformed into the β phase.10,20 A recent experimental study revealed that the α-to-β phase transition can occur upon heat treatment at elevated temperatures, yet the α phase is reformed when cooling down to room temperature.10 Given that phase engineering of Sb2O3 plays crucial roles in material and device properties, it is therefore of great importance to reveal the underlying crystallization mechanism and selective growth routes so that a dielectric phase with tailored properties can be obtained.21–23 In addition, the contact between the dielectric layer and metal electrodes may alter the arrangement of molecules at their interface, since the selection of substrate type plays a central role in epitaxial growth.7,8,24 So far, the preparation of interface phases on non-vdWs substrates has not been explored. The main challenge lies in the direct capture of the possible metastable phases, particularly at the molecular level (Fig. 1).
Herein, we describe the use of the Au(111) substrate as an epitaxial substrate to investigate interface phases of monolayer Sb2O3. Two kinetic phases and a thermodynamically stabilized phase were successfully prepared. The combined use of molecular beam epitaxy (MBE) and scanning tunneling microscopy (STM) provides atomic insights into the epitaxial growth process and phase transition processes. The effects of temperature and molecular flux on molecular arrangement have been systematically studied. Furthermore, the dielectric properties of the experimentally observed phases were analyzed using theoretical calculations.
Antimony trioxide (Sb2O3), composed of Sb4O6 polyhedral cages, was selected for this study. We used an ultra-high vacuum MBE system equipped with a K-cell evaporator to investigate the epitaxial growth of Sb4O6 molecules on Au(111). When the substrate temperature was at 298 K for 10 min, the surface was covered with well-organized nanoarchitectures (Fig. 2a), as imaged by STM characterization. The fast Fourier transform (FFT in Fig. 2d) derived from the STM image of Fig. 2a reveals a hexagonal pattern with a lattice constant of 1.05 nm. This value is greatly larger than the lattice constant of the α phase (0.79 nm) Sb2O3 films on graphite. High-resolution STM images (Fig. 2b and c) display a porous phase (Phase 1) and a close-packed phase (Phase 1′) obtained from scanning regions outlined in red and blue squares, respectively.
Phase 1 is a porous nanostructure with an adjacent pore-to-pore distance of 1.00 ± 0.05 nm (Fig. S1). The main axes (a and b) of the adsorbed molecular lattice were rotated by an angle of ϒ = 60° ± 1°. Six bright protrusions (Fig. S2) of the unit pore can be ascribed to six Sb4O6 molecules, since the size of each protrusion is comparable to that of a Sb4O6 molecule. Noting that two Sb4O6 molecules exist within a unit cell, the molecular density of phase 1 is therefore calculated to be 2.14 nm−2. Apparent height analysis reveals a height difference of 72 pm between the molecular film and the bare gold substrate (Fig. S3), providing support for a monolayer configuration. Owing to the clearly observed herringbone feature of Au(111), the relative angle between the b axis of phase 1 and the reconstruction stripe is measured to be 16° (Fig. 2d). Given that the reconstruction stripes are oriented perpendicular to the (1
0) direction of the underlying substrate lattice, the relative angle between the b axis and the gold atomic rows is approximately 14°.
Phase 1′ shares the same lattice constant and the same azimuthal orientation of phase 1 (Fig. S4). The co-existence of two rotational domains (Fig. S5) with an angle of 28° further confirms this registry relationship, meaning that one domain is rotated by +14° with respect to the gold lattice and the other one is rotated by −14°. In contrast to phase 1, two major differences can be identified. First, the center of most six-membered pores was filled by a brighter spot, which was ascribed to an additional Sb4O6 molecule due to host–guest interactions. Apparent height analysis (Fig. S3) displays a height difference of 35 pm between the center-brighter spots and the surrounding darker spots, suggesting that the adsorbed molecules are located within the same molecular layer, instead of a bilayer configuration. In the case of phase 1′, three molecules exist within a unit cell, giving rise to a higher molecular density, i.e., 3.21 nm−2. In addition, for most regions of phase 1′, the Au(111) herringbone feature becomes invisible. Specifically, within the same STM image under the same scanning condition, the herringbone feature is visible for phase 1 yet invisible for phase 1′.
Note that each Sb4O6 molecule can adopt either an “upward” or “downward” configuration once adsorbed onto surfaces. Details of both adsorption configurations are fully presented in the SI (Fig. S6). We hypothesize that for the case of phase 1, all molecules are adsorbed onto the Au(111) surface through a “downward” configuration, whereas for phase 1′, the molecules existing inside the six-membered pores adopt an “upward” configuration. This hypothesis is supported by the fact that for STM images with different scanning conditions, there is no apparent contrast difference for phase 1, yet under positive or negative bias, the contrast difference persists for the case of phase 1′ (Fig. S7). The appearance of contrast difference is further supported by high-resolution STM images acquired for the case of the alpha phase, for which alternatively arranged “downward” and “upward” molecules can be clearly observed. According to DFT calculations, the adsorption energy of the downward configuration on Au(111) is higher than that of the upward configuration (Fig. S6), thereby supporting the structural assignment of phase 1 as a fully downward molecular assembly. Moreover, the adsorption energies of phase 1 and phase 1′ on Au(111) have been calculated to compare their relative stability, yielding values of −0.94 eV and −0.89 eV per molecule, respectively (Fig. S8).
The above-mentioned experimental observation allowed us to propose adsorption models for phase 1 (Fig. 2e) and phase 1′ (Fig. 2f), respectively. By using a matrix notion,25 the hexagonal structures of phase 1 and phase 1′ are described relative to the a1 and a2 of Au(111) as
(Fig. S9). The self-assembled network is commensurable with the Au(111) lattice and the epitaxy is of the point-on-point type. This type of epitaxy suggests that the molecule–surface interaction should be stronger than the molecule–molecule interaction. van der Waals interactions between two nearest Sb4O6 molecules separated by a distance of 0.32 nm support the formation of a 2D network (Fig. S10). The corresponding DFT-simulated STM images, displayed in the upper-right insets of Fig. 2e and f, show good consistency with the experimental STM results.
To further understand the structural evolution process, we analysed more acquired STM images and found that molecular filling inside the pores and lateral elongation of the molecular island occurred simultaneously, indicating a continuous growth process rather than a discrete phase transition. Phase 1 and phase 1′ were always observed, regardless of lower or higher coverage of adsorbed molecules. At lower molecular coverage, Sb4O6 molecules nucleate into sub-monolayer islands, in which partial filling already emerged within the porous domains (Fig. S3). Fig. 3a–f show STM images acquired from different scanning regions of the same sample, exhibiting variable filling ratios. The filling ratio was determined by manually counting the total number of pores and the number of filled pores within each STM image and calculating their ratio. The analysed STM images typically cover surface areas of approximately 25 × 25 nm2.
The reported values therefore reflect local filling statistics and are subject to uncertainties associated with finite sampling. The sequence represents the progressive transition from the pure porous phase (phase 1) to the fully close-packed phase (phase 1′) with filling ratios increasing from 0%, 1.9%, 29.8%, 61.6%, and 98.5% towards 100%. In all cases, the disappearance of the herringbone feature is accompanied by the emergence of phase 1′ with a high ratio of upward molecules, indicating that the surface reconstruction might be modified through the adsorption of Sb4O6 molecules inside the pores of phase 1. In regions of low or partial filling, the characteristic herringbone reconstruction of the Au(111) substrate remained visible, whereas in highly filled areas, the herringbone feature was not observed (Fig. S11). Similar to the adsorption of electronegative thiols or coordination molecules that lift the Au(111) herringbone reconstruction, the modification of the herringbone pattern can be explained by the local stress redistribution at the molecule–metal interface arising from the charge transfer between three oxygen atoms of Sb4O6 and the underlying gold atom.26–30
After successful preparation of the porous phase and partial filling of upward molecules (Fig. 4a), we first analyzed the temperature effect upon annealing the same sample at 373 K for 30 min. Accordingly, newly formed molecular islands emerged on the surface (Fig. 4b), whereas smaller domains of phases 1 and 1′ can still be found. The high resolution STM image revealed that the newly formed phase follows a hexagonal arrangement with a lattice constant of 0.79 ± 0.05 nm (Fig. S12), which is in perfect agreement with the α phase observed on graphite surfaces. Two molecules exist within a unit cell, providing a molecular density of 3.7 nm−2, which is the highest among the others. The newly formed α phase clearly indicates a phase transformation process, from less compact phases (phase 1 and phase 1′) to a more compact phase (α phase). This is consistent with the principle of close packing.
Furthermore, when the substrate was maintained at 473 K during growth, only the α phase was obtained, forming well-defined triangular crystalline islands (Fig. 4d), suggesting that the α phase is thermodynamically preferred. While the substrate temperature was maintained at room temperature, by increasing deposition flux (the source temperature increased from 393 K to 443 K), three phases—α phase, phase 1 and phase 1′—were found to coexist (Fig. 4c).
Although kinetically stabilized phases are generally favored at higher deposition rates, rapid molecular accumulation under such conditions leads to locally enhanced surface density and stronger intermolecular interactions, which may facilitate structural rearrangement into the more densely packed α phase. This phenomenon can also occur at lower deposition rates; however, the transformation is extremely slow and therefore difficult to observe under typical experimental conditions.31 Meanwhile, kinetic trapping remains operative in regions where molecular rearrangement is incomplete, resulting in the coexistence of kinetically stabilized and thermodynamically favored phases. This observation demonstrates that under mild annealing conditions, the α phase can also be fabricated if the molecular flux is increased to a higher value. Mild growth of α phase Sb2O3 is highly desirable for the purpose of facile growth requirements, particularly for device fabrication processes as harsh temperature conditions should be avoided.
The dielectric properties of kinetic and thermodynamic interface phases have also been evaluated through DFT calculations. The dielectric tensors corresponding to the electronic and ionic contributions were calculated using density functional perturbation theory (DFPT) and the finite-difference approach, respectively, with vdWs corrections included. To benchmark our calculations against the experiment, we also computed the static dielectric constant of the bulk cubic phase and obtained a total value of 9.96, which is in good agreement with the experimental measurement of 11.5,4 especially considering the polycrystalline nature of real samples. Fig. 5a and b display the real part of the in-plane dielectric constant (ε1) as a function of photon energy (0–25 eV), separated into electronic and ionic contributions, respectively. The out-of-plane dielectric constant is shown in Fig. S13. All three phases exhibit pronounced frequency-dispersion behavior in the low-energy region, where ionic polarization dominates, while electronic polarization governs the response at higher energies. The static (E = 0 eV) electronic dielectric constants of the α phase, phase 1′ and phase 1 were 2.25, 1.97, and 1.66, respectively. The static ionic dielectric constants of the α phase, phase 1′ and phase 1 were 4.31, 1.28, and 0.68. This leads to a total static dielectric constant (ε0 = εele1 + εion1) calculated to be 6.56, 3.25, and 2.33 for the α phase, phase 1′ and phase 1, indicating that the Pearson correlation coefficient is not applied. Consistent with the Clausius–Mossotti relation, the dielectric constant increases with number density across the three phases: phase 1 (2.14 nm−2, ε = 2.33), phase 1′ (3.21 nm−2, ε = 3.25), and α phase (3.70 nm−2, ε = 6.56), demonstrating a clear positive correlation between packing density and the dielectric response. Moreover, the highest occupied orbitals of the three phases were plotted and compared (Fig. S14). The results show that the orbital overlap in the α phase is indeed stronger than that in the other phases, which can be attributed to the closer molecular packing. According to the first-order perturbation theory, the electronic susceptibility is inversely correlated with the energy difference of the transition states, which, on average, increases with the band gap size. Therefore, a larger ionic contribution and a relatively lower electronic contribution are expected for an idea high-k dielectric material in order to reach an optimal trade-off between the dielectric constant and the band gap. In the case of phases 1 and 1′, the ratio of the electronic part to the total dielectric constants is 71% and 60%, which is greatly larger than 34% calculated for the case of the α phase. The differences in charge transfer among the various phases are found to be negligible (Fig. S15). This can be understood in light of the molecular crystal nature of the system, where the Sb4O6 molecules interact predominantly through van der Waals forces. Consequently, the different molecular arrangements in the various phases have little effect on the intramolecular charge transfer.
In summary, Sb2O3 molecules self-assemble into a previously unreported hexagonal porous phase accompanied by a hexagonal close-packed phase on Au(111) under mild growth conditions. The two kinetic phases were captured and analyzed by STM characterization. Theoretical calculations of the dielectric constant reveal that unlike the compact and thermodynamically stabilized α phase, the kinetic porous phase exhibits a lower molecular density and a lower dielectric response compared with the α phase. Upon increasing molecular coverage, “upward”-oriented Sb2O3 molecules gradually occupy the hexagonal pores without altering the original lattice parameters, forming a hexagonal close-packed phase with an enhanced dielectric constant. Moreover, modulation of the molecular beam flux and substrate temperature enables selective growth of the hexagonal porous (phase 1), hexagonal close-packed (phase 1′), and α phases on Au(111). These findings provide fundamental insights into the interface phase engineering of Sb2O3, which plays a crucial role in future electronic and optoelectronic applications.
The main data supporting the findings of this study are available within the paper and its supplementary information (SI). The supplementary information includes additional experimental methods, supplementary computational results, and supplementary STM images. See DOI: https://doi.org/10.1039/d6ma00100a.
| This journal is © The Royal Society of Chemistry 2026 |