Donald
Weber
,
Randall
Grimes
,
Ping
Su
,
Robert
Woods
and
Patricia
Baker
*
Provista Life Sciences, 6225 N 24th Street, Suite 150, Phoenix, AZ 85016, USA. E-mail: information@provistals.com; Fax: +1 602 840 6111; Tel: +1 602 224 5500
First published on 14th May 2010
According to the literature, cytokine levels observed in serum have a dependent relationship to a patient's age. Despite the recognition of this important relationship, it has been largely overlooked as a component in cytokine-based assay modeling and development. In a 466-subject breast cancer detection assay study, we examined the impact that age-stratified analysis has on a serum-cytokine-based assay's performance. Patient samples were analyzed for 4-cytokines (i.e. Interleukin-8 and Interleukin-12 p40/p70, hepatocyte growth factor and vascular endothelial growth factor) along with carcinoembryonic antigen, all of which are putatively associated with breast cancer. Age-unstratified (baseline) and age-stratified training models were constructed using linear and logistic regression to differentiate breast cancer from controls and validated using an independent set of patient data. Age-stratified models demonstrated respective training and validation area under the receiver operating characteristic (AUROC) curve improvements over baseline of 20% and 58% for women ages 35–49; AUROC improvements of 12% and 42% for women ages 50–59; and AUROC shifts of +4% and −40% for women ages 60 and older. Predictive assay scores demonstrated similar findings. This study revealed substantive age-dependent shifts in cytokine expression measurements that were obfuscated in the age-unstratified assay modeling efforts. Such age-stratification considerations in other cytokine-based disease state detection assay development efforts could prove to be beneficial.
In this study, we examined the effect age-stratified analysis has on the development of a breast cancer detection assay's performance. The test assay, consisting of four cytokines, i.e. IL-8, IL-12 p40/p70, hepatocyte growth factor (HGF) and vascular endothelial growth factor (VEGF) along with carcinoembryonic antigen (CEA), was developed from a smaller empirical study (n = 100) of 16 markers (cytokines, chemokines and other tumor associated antigens) associated with breast cancer in the literature. Markers demonstrating the greatest differentiation in the initial study were retained in the panel while those demonstrating less were dropped.
The study was approved by the Western Institutional Review Board and written informed consent was obtained from all participants of the study. 466 female patients between the ages of 35 and 75 were prospectively enrolled from 9 community based medical facilities over a 3 year period (04/21/06 to 03/25/09). The clinical trial design consisted of a control group undergoing screening mammograms (n = 163) and a second patient group with abnormal mammograms scheduled for breast biopsy (n = 303); the latter from which the breast cancer positive cohort was derived (n = 60). All patients had blood drawn naïve to mammography or biopsies. Final invasive breast cancer positive and negative diagnostic assignments were made on the basis of the tissue histopathology. Breast cancer positive (BC+) and ductal carcinoma in situ (DCIS) were categorized as true breast cancer positive patients, and healthy screening patients (HW) and patients with biopsy negative (BC−) were categorized as true non-breast cancer controls.
Protocol specific exclusions (predicated on the potential to impact the biomarkers of interest) included women who were pregnant, nursing, receiving chemo- or radiation therapy, had known autoimmune disease(s), had taken antibiotics in the preceding two weeks or had cancer in the previous ten years. Demographic, lifestyle and breast cancer risk factor data were collected on all subjects. Select enrollment, demographic and risk characteristics can be found in Table 1.
Patient (N) | All | BC+ | BC− | HW |
---|---|---|---|---|
All | 466 | 76 | 228 | 162 |
<50 | 218 | 28 | 127 | 63 |
50≤ | 248 | 48 | 101 | 99 |
Age | All | BC+ | BC− | HW | p-Value BC+: HW |
---|---|---|---|---|---|
All | 51.8 | 54.7 | 49.5 | 54.2 | 0.93 |
<50 | 43.1 | 45.2 | 42.3 | 43.7 | 0.06 |
50≤ | 59.4 | 59.3 | 58 | 60.9 | 0.20 |
p-Value <50 : 50≤ | 0.15 | 0.67 | 0.08 | 0.56 | — |
Postmenopause | All | BC+ | BC− | HW | p-Value BC+: HW |
---|---|---|---|---|---|
All | 50% | 63% | 44% | 56% | 0.47 |
<50 | 10% | 21% | 7% | 11% | 0.20 |
50≤ | 86% | 83% | 89% | 84% | 0.94 |
p-Value <50 : 50≤ | <0.01 | <0.01 | <0.01 | <0.01 | — |
Full term pregnancies | All | BC+ | BC− | HW | p-Value BC+: HW |
---|---|---|---|---|---|
All | 2.3 | 2.3 | 2.2 | 2.4 | 0.78 |
<50 | 2.1 | 2.1 | 2.1 | 2.1 | 0.72 |
50≤ | 2.4 | 2.5 | 2.4 | 2.5 | 0.76 |
p-Value <50 : 50≤ | 0.01 | 0.23 | 0.18 | 0.05 | — |
All laboratory analyses were conducted blind to the diagnostic, patient demographic and clinical background information. IL-8, IL-12 p40/p70, HGF, VEGF and CEA were analyzed using multiplex beaded immunoassays and individual enzyme-linked immunosorbent assays (ELISA). Resultant subject data were initially stratified into three age groups: unstratified (n = 466), 49 years and younger (n = 218) and 50 years and older (n = 248). The latter group was further broken down into two cohort ages 50–59 (n = 143) and 60 years and older (n = 105). These age brackets were chosen as a close proxy for menopausal status, as shown in Table 1, and for their consistency with the age stratification used in other breast cancer studies.20–23 Within each age group, data sets were composed of all invasive breast cancer cases and, to better reflect what would be found in a clinical setting, all healthy controls and a portion of the breast cancer negative biopsy referrals equal to 10% of the number of healthy controls for the data set; cases selected by random number generation. Within each age-stratification, the data were randomly split into two sets. Seventy-five percent (75%) of the data were assigned to a training set for assay model development and the remaining twenty-five percent (25%) were held back for purposes of model validation.
Specimen samples were analyzed using two-multiplex bead-based immunoassays for IL-8, IL-12 p40/p70 and HGF, and two individual enzyme-linked immunosorbent assay (ELISA) for CEA and VEGF. A Bioplex 200™ (Bio-Rad, USA, Hercules, CA) multiplex beaded assay system and a SpectraMax 20M2e™ (Molecular Devices, USA, Sunnyvale, CA) enzyme-linked immunosorbant assay (ELISA) analytical platform were used for multiplexed and single biomarker analyses, respectively. Commercially supplied cytokine assay test kits were purchased from two separate manufacturers, i.e. IL-8, IL-12 p40/p70, HGF and VEGF from Invitrogen (Camarillo, CA) and CEA from CanAg/Fujirebio (IBL-America Distributors, Minneapolis, MN).
Sample preparation and analyses were conducted using strict federal Clinical Laboratory Improvement Amendments (CLIA) laboratory analytical quality control compliance requirements for assay method development and analysis.24 The targeted coefficients of variations (CVs) and relative percent differences (RPD) for intra-operator and intra-manufacturer performances for this study evaluation were established at ≤15%, and the inter-operator, inter-laboratory and inter-manufacturer lot performances at ≤15%.
ANOVA generalized linear regression models were developed and validated with logistic regression using SPSS® Statistics 17.0 software program, backward regression with p < 0.05 (two-sided tests) at a 95% confidence interval. Final models developed were used to generate a singular composite numeric score for each patient from which a breast cancer positive or breast cancer negative determination was made. Estimates were made of area under the receiver operating characteristic (AUROC) curve, sensitivity and specificity for all the models. All AUROC curve values were generated using Riemann summations (n = 50).
Following age-stratification, differences in individual markers were found between patients under 50 and those 50 and older. Likewise, the relationships between breast cancer and healthy patients were significantly different following stratification. While several markers rose to a level of statistical significance, in univariate analysis no singular marker proved clinically useful in differentiating breast cancers from controls. In women under 50, for example, univariate differentiation ranged from 13% (HGF) to 42% (CEA).
In contrast, multivariate analysis of the data demonstrated statistically significant differences between breast cancer positive samples and their respective controls for both the unstratified (i.e. 35–75) and age-stratified cohorts (i.e. under 50 and 50 and older) with an observed deterioration noted in the 50 and older findings. Further evaluation suggests that this deterioration may have been due to the concentration of women ages 60 and older who appear to lack differentiation between those with breast cancer and controls.
Fig. 1a shows the assay's overall analytical accuracy progression, as defined by the assay's AUROC from an age-unstratified model through two levels of age-stratification modeling of the same data (i.e. women under 50 and 50 and older, the later cohort of which was further broken down to women ages 50–59 and 60 and above). In the age-unstratified model (lower left panel: women 35–75 years of age) the training and validation datasets AUROCs were 0.81 and 0.55, respectively, which serves as the basis for the age-stratification modeling comparisons. Age-stratified models demonstrated training and validation AUROC improvements over the unstratified baseline of +20% and +58%, respectively, for ages 35–49 (upper center panel: 0.98 training, 0.87 validation); AUROC improvements of +12% and +42%, respectively, for ages 50–59 (upper right panel: 0.92 training, 0.78 validation); and AUROC shifts of +4% and −40%, respectively, for ages 60≤ (lower right panel: 0.85 training, 0.33 validation). Similar patterns were observed upon inspection of the associated sensitivity and specificity as age stratification progresses, as shown in Fig. 1b.
![]() | ||
Fig. 1 (a) Age-unstratified versus age-stratified AUROC assay performance. In the >50 and 50–59 strata, age stratification increases both the maximum training model AUROC (0.98 and 0.92 respectively) and validation stability. (b) The associated changes in sensitivity and specificity that occur as age stratification progresses. |
The improved performance was also noted by the increased stability and strengthening of the validation data's AUROC performance as measured against the training data. The difference between training versus validation age-unstratified AUROCs was 27%. However, after age-stratification modeling occurred this difference was dramatically reduced to 11% in the subsequent under 50 years of age cohort breakout and 14% in the 50–59 age cohort breakout.
Fig. 2 shows the assay's average predictive score (as generated by the relational algorithm developed for this assay) between the disease state (BC+) and non-disease state controls (nBC). These differences mirrored what is seen in Fig. 1. Of particular note was the impact of serial age-stratification on the validation data's predictive assay score. Whereby the unstratified data showed a slight difference of 0.06 between BC+ and nBC cohorts, upon the first age-stratification break-out, i.e. under 50 and 50 years of age and older, the differentiation between BC+ and nBC was immediate for the under 50 group of women (a difference of 0.38). However, for the latter age group this difference actually became smaller reflecting only a 0.01 difference in the mean score between BC+ (0.34) and the nBC control subjects (0.33). Not until the 50 years of age and older study group was broken out further into two subgroups (i.e. 50–59 and 60 years and older) did the average predictive scores demonstrate differentiations. The 50–59 cohort group showed a 0.38 difference between the mean scores of the BC+ versus the nBC controls whereas the 60 years and older subjects actually showed a reversal of −0.10 between BC+ versus nBC cohorts.
![]() | ||
Fig. 2 Performance of age-unstratified and age-stratified assay’s predictive scores. While the training set shows differentiation with all age stratification methods (mean difference between BC+ and control = 0.35), the greatest differentiation is observed in the <50 and 50–59 stratum (mean difference scores of 0.56 and 0.47 respectively). This differentiation is maintained in these strata when validated with independent data (mean difference of 0.38 in each). |
To enhance this assay's performance additional studies are planned to expand the knowledge base in three key areas—natural variance, patient specific variance and stage/subtype sensitivity. Natural variation will be addressed by expanding ethnic diversity, increasing population size and collecting longitudinal data. Patient specific variability studies will examine the contributions that other patient specific factors, such as co-morbidities, medications, demographic and lifestyle factors will have on observed cytokine variability. Finally, these future studies will incorporate postexcision diagnostic data to better determine stage and subtype sensitivity than can be currently ascertained from biopsy tissue histopathology alone.
This journal is © The Royal Society of Chemistry 2010 |