The SAMPL9 host–guest blind challenge: an overview of binding free energy predictive accuracy

We report the results of the SAMPL9 host–guest blind challenge for predicting binding free energies. The challenge focused on macrocycles from pillar[n]-arene and cyclodextrin host families, including WP6, and bCD and HbCD. A variety of methods were used by participants to submit binding free energy predictions. A machine learning approach based on molecular descriptors achieved the highest accuracy (RMSE of 2.04 kcal mol−1) among the ranked methods in the WP6 dataset. Interestingly, predictions for WP6 obtained via docking tended to outperform all methods (RMSE of 1.70 kcal mol−1), most of which are MD based and computationally more expensive. In general, methods applying force fields achieved better correlation with experiments for WP6 opposed to the machine learning and docking models. In the cyclodextrin-phenothiazine challenge, the ATM approach emerged as the top performing method with RMSE less than 1.86 kcal mol−1. Correlation metrics of ranked methods in this dataset were relatively poor compared to WP6. We also highlight several lessons learned to guide future work and help improve studies on the systems discussed. For example, WP6 may be present in other microstates other than its −12 state in the presence of certain guests. Machine learning approaches can be used to fine tune or help train force fields for certain chemistry (i.e. WP6-G4). Certain phenothiazines occupy distinct primary and secondary orientations, some of which were considered individually for accurate binding free energies. The accuracy of predictions from certain methods while starting from a single binding pose/orientation demonstrates the sensitivity of calculated binding free energies to the orientation, and in some cases the likely dominant orientation for the system. Computational and experimental results suggest that guest phenothiazine core traverses both the secondary and primary faces of the cyclodextrin hosts, a bulky cationic side chain will primarily occupy the primary face, and the phenothiazine core substituent resides at the larger secondary face.

Table S2 Summary of restraints applied for reference calculations with APR method.The parameters used on the different types of restraints.Within a host class (i.e WP6, bCD, and HbCD), the force constants (k), angle, or distance of each guest restraint were the same.If optional restraints were not applied (i.e Conformation or Wall restraints), the column is left empty.Table S3 ∆∆G error of optional WP6-G4 system for all SAMPL9 submissions The predicted binding free energy (calculated) of all methods for optional WP6-G4 system, and ∆∆G error (kcal/mol) from ∆G calculated minus ∆G experimental (-7.77kcal/mol).Submissions that did not include a WP6-G4 prediction are filled with DNS (did not submit).Each unique method has an assigned submission ID (sid), a ranked submission is denoted with an asterisk next to the method name, and a reference submission is flagged with a double asterisk.Expanded ensemble methods (sids 4, 5, 6, and 17) modeled G4 using GAFF2.11,since OpenFF2.0.0 parameters were not available.

ID
Error statistics (RMSE and ME) of ranked predictions for each host-guest system Binding free energy RMSE and ME by host-guest system, where each host-guest system category is color coded by host (WP6 in blue, bCD in yellow, and HbCD in red).The RMSE and ME for absolute binding ∆G were calculated via bootstrapping with 100,000 replacement samples, including experimental uncertainties, of all host-guest system predictions.The black error bars represent 95-percentile bootstrap confidence intervals.Optional system WP6-G4 was not included.
Error statistics (RMSE and ME) of all methods predictions for each host-guest system Binding free energy RMSE and ME by host-guest system for all methods (ranked and non-ranked combined).Each host-guest system category is color coded by host (WP6 in blue, bCD in yellow, and HbCD in red).The RMSE and ME for absolute binding ∆G were calculated via bootstrapping with 100,000 replacement samples, including experimental uncertainties, of all host-guest system predictions.The black error bars represent 95-percentile bootstrap confidence intervals.Optional system WP6-G4 was not included.The cyan dashed line connecting H2 and H3 is for clarity, to prevent the guest from being blocked.A phenylene group and its two carboxylate arms were removed for visualization purposes.The specific WP6 and guest anchor atoms selected are in Table S1 and the force constants in Table S2.
Figure S7 APR wall restraints setup for SAMPL9 WP6 dataset.Following the schematic in figure S5, shown is the WP6-G3 complex with the wall restraints used for APR binding free energy calculations for this dataset.A phenylene group and its two carboxylate arms were removed for visualization purposes.The magenta dashed lines represent the wall created via harmonic flat-bottom restraints imposed to keep the guest (via the L1 guest anchor atom) in the WP6 host cavity.The two ether oxygen atoms (O1, O31, O25, O19, 013, O7, O2, O32, O26, O20, O14, and O8) of each phenylene subunit were used to define the wall restraints.The force constant and distance criteria used for wall restraints are in Table S2.S1 and the force constants in Table S2.
Figure S9 APR wall restraints setup for SAMPL9 cyclodextrin dataset.Following the schematic in figure S5, shown is the bCD-PMT complex with the wall restraints used for APR binding free energy calculations for this dataset.A glucose subunit was removed for visualization purposes.The magenta dashed line represents the harmonic flat-bottom restraints imposed to keep the PMT guest (via the L1 anchor atom) in the bCD cavity.The oxygen atoms (O3, O33, O28, O23, O18, O13, and O8) of each glucopyranoside linker were used to define the wall restraints.The force constant and distance criteria used for wall restraints are in Table S2.
Figure S10 APR host dihedral restraint setup for SAMPL9 cyclodextrin dataset Shown is the bCD host with blue circles highlighting the atoms used to define a set of dihedral restraints on a glucopyranoside linker.One of the dihedrals is defined by atoms O14, C13, O18, and C22, and the second dihedral is defined by atoms C13, O18, C22, and C23.The PMT guest and a glucose subunit were removed for visualization purposes.The force constant and angles used are in Table S2.

Figure S3
FigureS3∆∆G error of ranked predictions for each host-guest system (calculated versus experiment) The ∆∆G error of calculated versus experiment (in kcal/mol) is represented by black dots.The distribution of the ∆∆G is highlighted for each host-guest system

Figure S5
Figure S5 Schematic representation of APR restraints setup of SAMPL9 host-guest datasets.A general schematic of the restraints used for attachpull-release (APR) binding free energy (BFE) calculations.The host is represented by the orange cylinder shape with anchor atoms (H1, H2, and H3), the guest by the gray rectangle with guest anchor atoms (L1 and L2), and dummy atoms (D1, D2, and D3) in black.The host and guest restraints required for APR simulations (shown in this schematic) are defined and setup with the help of the dummy atoms relative to the host and guest anchor atoms.Shown in cyan are Boresch-style restraints composed of three torsions (t1, t2, and t3), two angles (a1 and a2), and one distance (d1) restraints, imposed throughout the entire simulation on the host to control its translational and orientational (rotational) degrees of freedom.We note that these restraints do not perturb the host's conformational degrees of freedom.t1 is defined by D3, D2, D1, and H1.t2 is defined by D2, D1, H1, and H2.t3 is defined by D1, H1, H2, and H3.a1 is defined by D2, D1, and H1.a2 is defined by D1, H1, and H2.d1 is defined by D1 and H1.In addition, shown in magenta are three restraints (two angles (θ and β ) and one distance (r)) imposed and attached on the guest over a series of simulation windows in the attach phase, and used to translate the guest out of the host cavity by increasing r in intervals of 0.4 Å creating the pull phase simulation windows.

Figure S6
FigureS6APR host-guest restraint setup for SAMPL9 WP6 dataset.Following the schematic in figureS5, shown is the WP6-G3 complex with the host-guest restraint setup for APR binding free energy calculations for this dataset, and to give the spatial relationship between dummy atoms, and guest and host anchor atoms.The WP6 host (in beige) and guest (in gray) anchor atoms selected are shown by spheres and labeled with H1, H2, H3, L1, or L2.The boresch-style restraints imposed on the host are shown in cyan, and three restraints imposed on the guest are shown in magenta.The cyan dashed line connecting H2 and H3 is for clarity, to prevent the guest from being blocked.A phenylene group and its two carboxylate arms were removed for visualization purposes.The specific WP6 and guest anchor atoms selected are in TableS1and the force constants in TableS2.

Figure S8
FigureS8APR host-guest restraint setup for SAMPL9 cyclodextrin (bCD and HbCD) dataset.Following the schematic in figureS5, shown is the bCD-PMT complex with the host-guest restraint setup for APR binding free energy calculations for this dataset, depicting the spatial relationship between dummy atoms, and guest and host anchor atoms.The bCD host (in beige) and guest (in gray) anchor atoms selected are shown by spheres and labeled with H1, H2, H3, L1, or L2.The boresch-style restraints imposed on the bCD host are shown in cyan, and three restraints imposed on the PMT guest are shown in magenta.The cyan dashed line connecting H2 and H3 is for clarity, to prevent the guest from being blocked.A glucose subunit was removed for visualization purposes.The specific bCD, HbCD, and guest anchor atoms selected are in TableS1and the force constants in TableS2.