Longzhu Q.
Shen
a,
Richard S.
Judson
b,
Fjodor
Melnikov
a,
John
Roethle
c,
Aditya
Gudibanda
d,
Julie B.
Zimmerman
a and
Paul T.
Anastas
*a
aSchool of Forestry and Environmental Studies, New Haven, CT 06511, USA. E-mail: paul.anastas@yale.edu; Fax: +1-203-436-8574; Tel: +1-203-432-5215
bU.S. EPA, National Center for Computational Toxicology, RTP NC 27711, USA
cDepartment of Chemistry, Yale University, New Haven, CT 06511, USA
dDepartment of Computer Science, Yale University, New Haven, CT 06511, USA
First published on 5th July 2016
Toxicity is a concern with many chemicals currently in commerce, and with new chemicals that are introduced each year. The standard approach to testing chemicals is to run studies in laboratory animals (e.g. rats, mice, dogs), but because of the expense of these studies and concerns for animal welfare, few chemicals besides pharmaceuticals and pesticides are fully tested. Over the last decade there have been significant developments in the field of computational toxicology which combines in vitro tests and computational models. The ultimate goal of this field is to test all chemicals in a rapid, cost effective manner with minimal use of animals. One of the simplest measures of toxicity is provided by high-throughput in vitro cytotoxicity assays, which measure the concentration of a chemical that kills particular types of cells. Chemicals that are cytotoxic at low concentrations tend to be more toxic to animals than chemicals that are less cytotoxic. We employed molecular characteristics derived from density functional theory (DFT) and predicted values of log(octanol–water partition coefficient) (logP) to construct a design variable space, and built a predictive model for cytotoxicity based on U.S. EPA Toxicity ForeCaster (ToxCast) data tested up to 100 μM using a Näive Bayesian algorithm. External evaluation showed that the area under the curve (AUC) for the receiver operating characteristic (ROC) of the model to be 0.81. Using this model, we provide probabilistic design rules to help synthetic chemists minimize the chance that a newly synthesized chemical will be cytotoxic.
In vitro high throughput screening (HTS) methods have emerged as an efficient technology to examine how chemicals disrupt biological pathways and lead to adverse health outcomes. A paradigm shift from in vivo to in vitro and in silico testing was described by the U.S. National Research Council (NRC) in their report on Toxicity Testing in the 21st Century.3 In order to evaluate practical approaches to implement the “Tox21” vision, U.S. National Toxicological Program (NTP), the U.S. Environmental Protection Agency (EPA) and the NIH National Center for Advancing Translational Sciences (NCATS) collaboratively forged a research partnership. This Tox21 partnership is using HTS methods to test thousands of chemicals in a wide variety of cells, pathways and technologies, relevant to many aspects of chemical toxicity.4–9
Large data sets (thousands of chemicals, hundreds of measurements per chemical) are ideal for developing machine learning predictive models.10,11 While the Tox21 collaborators are primarily focused on predicting toxicity of largely-untested existing chemicals,12 these data sets can also be used in the area of green chemistry to help develop rules to apply in the design of newer, safer chemicals.13 Owing to the difference in research goals, the two fields apply separate biases in how models are developed. For computational toxicology, high predictive power for toxicity is the paramount object. A model can be as complex as is needed to achieve high accuracy, as long as the training data is not overfitted. The physical meanings of the input variables or model descriptors are often of secondary concerns. In contrast, green design13 pays heavy attention to the physical meanings of the computed molecular descriptors, their numerical accuracy, and the ability to map them to variables which synthetic chemists have control. The variables in the toxicity model are to be used as yardsticks for chemists in designing molecules. Physical observables and chemically intuitive variables are much preferable to more abstract global descriptions of chemical structure. A sufficiently accurate toxicity model that includes easy-to-interpret control variables will greatly aid in the green design process.
Current examples of green design research have modeled acute aquatic toxicity14,15 and mutagenicity/carcinogenicity16 endpoints. In this current report, we focus on in vitro cytotoxicity as a good testing ground for the development of green design rules associated with chemical toxicity. First, the data sets for cytotoxicity are larger, more uniform in protocol, and less noisy than any existing in vivo toxicity data sets. Second, cytotoxicity is on its own a valuable measure for safety assessment. It is normally performed in the early stage of drug development as a first filter.17 It can also, to certain degree, serve as an indicator of the range of doses where one might see in vivo toxicity.18,19 There exists high research interest in predictive modeling for the cytotoxicity endpoint.20–24 Therefore, it would be useful to be able to design commercial chemicals with reduced cytotoxicity potency. (Note that all chemicals are cytotoxic, and what matters is the concentration or dose that is required to cause it.)
There are three main challenging tasks in green design in the context of toxicity. The first one is to select and generate physically meaningful design variables that have an effect on the underlying mechanisms responsible for the toxicity endpoint of interest. The second one is to construct a sufficiently predictive model of toxicity based on these design variables. The third one is to devise a method to present the design-variable/toxicity mapping in a way that provides an easy way to guide safer molecular design. This current project addresses all three of these issues in the context of a particular mode of toxicity, namely cytotoxicity. We produce a probabilistic design diagram as a methodological addition to existing molecular design tools. This diagram renders the convenience of searching for solutions in the physical property space for a customerized likelihood of not causing cytotoxicity in human cells (named as benign probability hereafter). This diagram, we believe, offers advantages in its use for guiding the design of safer chemicals.
Assay name | Biological process | Organism | Tissue | Cell type |
---|---|---|---|---|
ACEA_T47D_80 h_negative | Cell proliferation | Human | Breast | T47D |
APR_HepG2_CellLoss_24 h_dn | Cell death | Human | Liver | HepG2 |
APR_HepG2_CellLoss_72 h_dn | Cell death | Human | Liver | HepG2 |
BSK_3C_proliferation_down | Cell proliferation | Human | Vascular | Umbilical vein endothelium |
BSK_3C_SRB_down | Cell death | Human | Vascular | Umbilical vein endothelium |
BSK_3C_Vis_down | Cell morphology | Human | Vascular | Umbilical vein endothelium |
BSK_4H_SRB_down | Cell death | Human | Vascular | Umbilical vein endothelium |
BSK_BE3C_SRB_down | Cell death | Human | Lung | Bronchial epithelial cell |
BSK_CASM3C_proliferation_down | Cell proliferation | Human | Vascular | Umbilical vein endothelium and coronary artery smooth muscle cells |
BSK_CASM3C_SRB_down | Cell death | Human | Vascular | Umbilical vein endothelium and coronary artery smooth muscle cells |
BSK_hDFCGF_proliferation_down | Cell proliferation | Human | Skin | Foreskin fibroblast |
BSK_hDFCGF_SRB_down | Cell death | Human | Skin | Foreskin fibroblast |
BSK_KF3CT_SRB_down | Cell death | Human | Skin | Keratinocytes and foreskin fibroblasts |
BSK_LPS_SRB_down | Cell death | Human | Vascular | Umbilical vein endothelium and peripheral blood mononuclear cells |
BSK_SAg_PBMCCytotoxicity_down | Cell death | Human | Vascular | Umbilical vein endothelium and peripheral blood mononuclear cells |
BSK_SAg_proliferation_down | Cell proliferation | Human | Vascular | Umbilical vein endothelium and peripheral blood mononuclear cells |
BSK_SAg_SRB_down | Cell death | Human | Vascular | Umbilical vein endothelium and peripheral blood mononuclear cells |
Tox21_AR_BLA_antagonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_ERa_BLA_antagonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_GR_BLA_antagonist_viability | Cell proliferation | Human | Cervix | HeLa |
Tox21_MitochondrialToxicity_viability | Cell proliferation | Human | Liver | HepG2 |
Tox21_FXR_BLA_antagonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_PPARd_BLA_antagonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_PPARg_BLA_antagonist_viability | Cell proliferation | Human | Kidney | HEK293 |
Tox21_VDR_BLA_antagonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_ARE_BLA_agonist_viability | Cell proliferation | Human | Liver | HepG2 |
Tox21_HSE_BLA_agonist_viability | Cell proliferation | Human | Cervix | HeLa |
Tox21_p53_BLA_p1_viability | Cell proliferation | Human | Intestinal | HCT116 |
Tox21_FXR_BLA_agonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_PPARd_BLA_agonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_p53_BLA_p2_viability | Cell proliferation | Human | Intestinal | HCT116 |
Tox21_p53_BLA_p3_viability | Cell proliferation | Human | Intestinal | HCT116 |
Tox21_p53_BLA_p4_viability | Cell proliferation | Human | Intestinal | HCT116 |
Tox21_p53_BLA_p5_viability | Cell proliferation | Human | Intestinal | HCT116 |
Tox21_VDR_BLA_agonist_viability | Cell proliferation | Human | Kidney | HEK293T |
Tox21_ESRE_BLA_viability | Cell proliferation | Human | Cervix | HeLa |
Tox21_NFkB_BLA_agonist_viability | Cell proliferation | Human | Cervix | ME-180 |
1. Single compound with a definite structure, excluding geometrical and optical isomers and mixtures;
2. Containing no metal elements;
3. Molecular weight < 1000.
A total of 1006 chemicals met these criteria. These chemicals were further broken down into two classes. A chemical tested to be positive in two or more cytotoxicity assays (Table 1) was labeled as “active”.26 Otherwise it was designated as “inactive”. The chemical hit rates distribution is provided in Fig. S2 in ESI.‡ Note that in these assays, we typically tested up to 100 μM, so cytotoxicity at higher concentrations would not be seen. Therefore, we are strictly modeling “cytotoxicity below 100 μM”. This classification strategy resulted in two balanced classes with a membership ratio of 0.96. All chemical data were evenly split into a training and testing set for the purpose of model cross-validation and external evaluation. Selected chemicals were first desalted using the open source chemistry toolbox OpenBabel.27 Then 3D structures with the lowest energy conformer were generated using ChemAxon Marvin calculator plugins.28
SOF = 1/(IP − EA) | (1) |
EPH = (IP + EA)2/8(IP − EA) | (2) |
ROC AUC20,33 was calculated based on the distributions of the “active” and “inactive” chemicals in the training set to advise variable selection. Fig. 1 shows that the six variables possess differentiated information in distinguishing between the “active” and “inactive” chemicals. The histogram for each design variable is provided in Fig. S3 in ESI.‡ To examine the dependency between the design variables, we computed the maximal information coefficients.34,35 Notice (see Fig. 2) that IP, EPH and EA.aq contain higher degree of mutual information compared to the rest of the matrix elements. Simultaneously, the same three variables are ranked as the weakest predictors in the ROC analysis (Fig. 1). Thus, they form a less predictive group and were therefore excluded from further consideration. The remaining three design variables SOF, PLRZ and logP were retained for the Näive Bayesian model construction.
Fig. 1 ROC AUC between “active” and “inactive” chemicals in the training set. Selected design variables were inked in purple. |
(3) |
In this study, the parameter θ represents the class identifier and X represents the design variables obtained from the previous section. Our interest is to calculate the posterior or benign probability π(θ|X), the probability for a chemical to be “inactive”. The challenge resides in estimating the likelihood function without the complete knowledge of the interactions between the Xi. The independence assumption allows one to express the likelihood function as a product of the conditional probabilities for each individual design variable. To examine the appropriateness of employing this assumption, mutual information (eqn (4)) between design variables was computed as shown in Fig. 2. The three selected variables (PLRZ, SOF and logP) showed marginal dependency between each other. Therefore, the independent assumption can be reasonably adopted.
(4) |
The model construction, data analysis and the graphical visualization in this study were coded using the Python programming language,37,38 libraries and packages.39–46
Given potential molecular mechanisms through which chemicals can invoke toxicity, the next step is to collect design variables that properly describe or control those mechanisms. For instance, the hydrophobicity of a molecule is usually an important indicator of cell membrane permeability. logP is frequently used as a simple descriptor for molecular hydrophobicity in quantitative structure–activity relationship models (QSARs) to predict acute aquatic toxicity.51 Cell membrane permeability can have two consequences. The direct one is that disruption of the membranes themselves is toxic, leading to necrosis. Alternatively, a chemical can pass through the cell membrane to enter cellular interior or interact with receptors on the membrane surface to exert a wide range of effects. These can be specific, receptor-mediated effects, which alter cell signaling, or non-specific effects which are more likely to lead to cell stress and cytotoxicity. The physical nature of these non-specific interactions is often mediated by induced electric dipole moments and dispersion forces. Polarizability is a physical quantity that describes the relative tendency of molecular electron cloud distortion under the influence of an external electric field. It quantifies the energy alteration upon molecular attractions.52 Therefore, it is reasonable to include polarizability in modeling cytotoxicity. Hard–soft acid–base (HSAB) theory estimates the tendency for chemicals to form covalent bonds, which is linked to the toxic potential of certain chemicals.53 Molecular softness built upon DFT provides a means to quantify this tendency.54 Thus, molecular softness was included in the design variable space in this study.
Note that the exact primary cause of cytotoxicity is difficult to discover. We have thoroughly analyzed the correspondence between specific molecular interactions, generalized cell-stress and cytotoxicity in this data set26 with multiple assays for each of multiple cell-stress processes (mitochondrial disruption, oxidative stress, ER stress, heat shock, apoptosis, etc.). What we almost universally see is that multiple cell stress assays are activated at roughly the same concentration (within the uncertainty of the assays themselves). We rarely ever see cytotoxicity in the absence of these more general cell stress markers, and only in the presence of some particular target activity (e.g. a receptor or enzyme). So we find (in most cases) that when cytotoxicity happens (in time and concentration), many or most of these other processes are also occurring, meaning that it is difficult to pull apart the specific cause–effect relationships between different cell-stress processes and cytotoxicity.
Precision = true positive/(true positive + false positive) | (5) |
Recall = true positive/(true positive + false negative) | (6) |
F1 score = 2(precision × recall)/(precision + recall) | (7) |
We define “inactive” chemicals as positive in this study because they represent the group of interest from the perspective of green chemical design. Therefore, type I error deserves the best attention to be avoided. Precision signifies the success rate in identifying true positives out of all predicted positives. It is a very important criterion in model evaluation according to the precautionary principles – the probability that chemical is truly safe given that it is predicted to be safe should be maximized. Recall assesses the ability of a model to make positive predictions. Both of these variables depend on the positive ratio in the data set. F1 score as the harmonic mean between precision and recall represents a balanced assessment to the model performance. ROC AUC measures the overall ability for a model to separate “inactive” chemicals from “active” ones.
The Näive Bayesian model was trained on ∼500 chemicals with 10-fold cross validation and externally tested on the remaining ∼500 chemicals. The model performance is summarized in Table 2.
Measure | Precision | F1 score | ROC AUC |
---|---|---|---|
Cross validation | 0.77 | 0.77 | 0.82 |
External evaluation | 0.78 | 0.77 | 0.81 |
It is easy to notice from Table 2 that the three performance indicators are in reasonable agreement. Also, the cross-validation and external evaluation results agree well with each other, indicating a lack of overfitting. While it would be surprising to see overfitting with only three predictor variables in such a large and diverse chemical set, it is gratifying to achieve this level of predictivity. To best of the authors’ knowledge, there are no other reports in the molecular design literature that have achieved this level of predictivity in the complete sample space. The high predictivity of this model indicates that a large majority of cytotoxicity is driven by relatively simple and non-specific molecular interactions rather than by a wide range of specific receptor-mediated mechanisms. It further suggests that the predictive model can be used for green design, at least to minimize the risk of cytotoxicity below 100 μM.
There are three advantages to using this probabilistic diagram to guide safer molecular design. Firstly, unlike the recursive partition strategy14,15 where only partial solutions were provided, this diagram-based approach reveals the complete solution in the design variable space. This feature does not only boost the elegance of the model but also broadens its utility. For instance, chemists may be subject to constraints on certain design variables due to specified functional requirements. In those scenarios, a complete solution renders a higher flexibility for the users to seek solutions in the less restricted regions in the design variable space. Secondly, instead of setting a simple yes/no criterion, this diagram estimates probabilities for a chemical to be “inactive” (not cytotoxic in this study). This is considered to be a more realistic approximation of complex systems such as biology. Probabilities can make the chemists aware of the change in cytotoxicity potential while adjusting design variable values. Thus, it provides quantitative information in the decision making process. Thirdly, the diagram can be used in two directions, estimating benign probability or seeking solutions in the design variable space. As shown in Fig. 3, a sample solution is indicated by the dotted lines. In this case, one wants to design a chemical with about 82% probability of not invoking cytotoxicity. For certain functional reasons, the polarizability has been set near 12 cm3 mol−1. Given those conditions, the designer connects two points on the probability and PLRZ axes to arrive at a specific point on R1 axis. Now one sees multiple options for the combination of logP and SOF to satisfy the two constraints above. The example on the graph illustrates one option: 2.8 for logP and 0.105 eV for SOF. The resultant molecule corresponds to propylparaben, a naturally occurring chemical that has been approved by US Food and Drug Administration (FDA) for safe use in cosmetics. Other possible solutions may exist and all can be found on the design diagram. In a word, the design diagram presents all the possible solutions in the design variable space for any particular chosen benign probability. Alternatively, it is possible to walk from the opposite direction on the graph. One can pre-assign values to all of the design variable axes and connect them to reach a resultant benign probability. This feature allows one to assess the benign probability for an existing or new chemical with defined design variable values.
Footnotes |
† EPA disclaimer: the views expressed in this paper are those of the authors and do not necessarily reflect the views or policies of the U.S. Environmental Protection Agency. |
‡ Electronic supplementary information (ESI) available. See DOI: 10.1039/c6gc01058j |
§ Numerical values of design variables used in the study are available upon request. |
This journal is © The Royal Society of Chemistry 2016 |