GEOM-drugs revisited: toward more chemically accurate benchmarks for 3D molecule generation

Abstract

Deep generative models have shown significant promise in generating valid 3D molecular structures, with the GEOM-drugs dataset serving as a key benchmark. However, current evaluation protocols suffer from critical flaws, including incorrect valency definitions, bugs in bond order calculations, and reliance on force fields inconsistent with the reference data. In this work, we revisit GEOM-drugs and propose a corrected evaluation framework: we identify and fix issues in data preprocessing, construct chemically accurate valency tables, and introduce a GFN2-xTB-based geometry and energy benchmark. We retrain and re-evaluate several leading models under this framework, providing updated performance metrics and practical recommendations for future benchmarking. Our results underscore the need for chemically rigorous evaluation practices in 3D molecular generation. Our recommended evaluation methods and GEOM-drugs processing scripts are available at https://github.com/isayevlab/geom-drugs-3dgen-evaluation.