Safaa
Al Jedani
ab,
Caroline I.
Smith
a,
James
Ingham
a,
Conor A.
Whitley
a,
Barnaby G.
Ellis
a,
Asterios
Triantafyllou
c,
Philip J.
Gunning
d,
Peter
Gardner
e,
Janet M.
Risk
d,
Richard J.
Shaw
df,
Peter
Weightman
*a and
Steve D.
Barrett
a
aDepartment of Physics, University of Liverpool, L69 7ZE, UK. E-mail: peterw@liverpool.ac.uk
bDepartment of Physics, University of Jeddah, Saudi Arabia
cDepartment of Cellular Pathology, Liverpool Clinical Laboratories, University of Liverpool, Liverpool, L7 8YE, UK
dLiverpool Head and Neck Centre, Department of Molecular and Clinical Cancer Medicine, University of Liverpool, L7 8TX, UK
eManchester Institute of Biotechnology, University of Manchester, 131 Princess Street, Manchester, M1 7DN, UK
fHead and Neck Surgery, Liverpool University Foundation NHS Trust, Aintree Hospital, Liverpool, L9 7AL, UK
First published on 27th July 2023
A regression-based fusion algorithm has been used to merge hyperspectral Fourier transform infrared (FTIR) data with an H&E image of oral squamous cell carcinoma metastases in cervical lymphoid nodal tissue. This provides insight into the success of the ratio of FTIR absorbances at 1252 cm−1 and 1285 cm−1 in discriminating between these tissue types. The success is due to absorbances at these two wavenumbers being dominated by contributions from DNA and collagen, respectively. A pixel-by-pixel fit of the fused spectra to the FTIR spectra of collagen, DNA and cytokeratin reveals the contributions of these molecules to the tissue at high spatial resolution.
Earlier attempts to fuse images used Raman, rather than histology, and Fourier transform IR (FTIR) hyperspectral images using a multivariate curve resolution alternating least squares (MCR-ALS) technique.14 Later studies adopted an image super-resolution approach to enhance the IR image resolution by a training generative adversarial network (GAN) model using routine histological haematoxylin and eosin (H&E) stained sections of breast tumour biopsies as a high-resolution image source.15 The approach allowed reconstruction of the IR low-resolution image based on the H&E high-resolution image. In addition, an unsupervised curvelet-based image fusion method effected fusing images of breast and ovarian tumour biopsies acquired from IR and dark-field microscopy.16 In general, fusion algorithms differ in their performance and computational load. The most important difference between these fusion algorithms is in calculating the weight (i.e., balance the input of high-resolution and low-resolution image) which is established by solving an optimisation problem.17
Recently a regression-based fusion algorithm18 was used to merge paired, co-registered H&E and IR images of three specimens of oral squamous cell carcinoma (OSCC) metastases in cervical lymph nodes. FTIR spectral images of similar specimens have also been analysed using a machine learning algorithm (MLA),19 which revealed that a single “metric”, the ratio of image intensities at 1252 cm−1 and 1285 cm−1, was able to discriminate between metastasis and lymphoid tissue with sensitivities, specificities and precision of 98.8 ± 0.1%, 99.89 ± 0.01% and 99.78 ± 0.02% respectively. Aperture scanning near-field optical microscopy (SNOM) images of the tissue obtained at a number of key wavenumbers showed variations in the chemical composition of the tissues, but identification of corresponding cellular elements was problematic. In an attempt to rectify this, the fused images presented here allow comparison with the results of the previous MLA and SNOM studies providing further insight into the chemical composition of OSCC metastasis.
The data were pre-processed in the following order: rubber-band correction, noise reduction, scattering correction, quality test, truncation of regions most affected by paraffin (1350 cm−1–1500 cm−1 and 2835 cm−1–3000 cm−1) and lastly, vector normalisation.
Spectral cluster analysis was implemented using the MATLAB Statistics and Machine Learning Toolbox.28
In order to evaluate the quality of the fused images of this core, both SSIM and SAM were used along with peak signal to noise ratio (PSNR) and mean squared error (MSE) (ESI Fig. 1†). The SSIM scores for all the fused images ranged from ∼0.7 to ∼0.9; due to the importance of maintaining the chemical information from the FTIR image, the contrast components between optical H&E and fused images were not expected to be identical. Scores for all the fused images are higher than those for the FTIR images, which ranged from ∼0.1 to ∼0. 2. The SAM scores ranged from ∼0.001 to ∼0.2.
Fig. 2(a) shows the average FTIR spectra of lymphoid cells (59230 spectra), tumour (114970 spectra) and highly keratinised tumour areas (10265 spectra) that were obtained from a k-means cluster analysis of the datacube. It is clear that all these spectra are very similar. However, images at two specific wavenumbers14 and the ratio of absorbances at those two wavenumbers showed clear contrast between the biological moieties [Fig. 2(b), (c) and (d)].
The literature and our own previous work suggest that three key molecules, DNA and molecules from the collagen and cytokeratin families, are major contributors to OSCC and supporting stroma tissue composite.14,29 Instead of producing fused images at single wavenumbers that have been assigned to known vibrational modes in particular moieties, we considered the spectra of the three key molecules30–32 [Fig. 3(a)] and performed pixel-by-pixel least-squares fits to the fused FTIR spectra. The goodness-of-fit parameter χ2 was typically in the range 2 to 5. The contribution of each molecule was then displayed as a composite RGB image, with cytokeratin shown in the red channel, collagen in the green channel and double stranded (ds)-DNA in the blue channel [Fig. 3(b)] and any pixel that did not produce a good fit shown as black. A magnified image taken from the boundary of the metastasis/lymphoid regions [Fig. 3(c)] showed variations in the chemical composition of different topographical compartments of the microenvironment [Fig. 3(d)].
Fig. 3 (a) FTIR spectra of cytokeratin (red line), collagen (green line) and ds-DNA (blue line) adapted from ref. 30–32 (offset for clarity); (b) least-squares fit of the FTIR spectra of these molecules displayed as an RGB composite image (red = cytokeratin, green = collagen and blue = ds-DNA) with an arrow indicating the region discussed in the text; (c) magnified image taken from the white rectangle shown in (b); (d) corresponding H&E image. The individual RGB channels of image (c) are shown in ESI Fig. 2.† |
It should be noted that we lack absolute scales as ref. 30–32 did not quantify absorbance scales on which the molecular spectra were plotted. Hence, the RGB image gives the spatial distributions of these molecules but not their relative concentrations.
It is useful to compare the results of this study with a previous study of fused images of non-cancerous oral epithelium and underlying stroma.18 The two tissue types were clearly delineated in both the H&E and FTIR spectral images shown in Fig. 4 of ref. 18 and histopathologists have a good understanding of these tissue types. Consequently, the differences in contrast between the two tissue types at five different FTIR wavenumbers could be understood in terms of contributions from DNA and collagen. The analysis was supported by an FTIR image obtained at a wavenumber attributed to lipids. It is notable that the success in interpreting the spectral images of Fig. 4 of ref. 18 was obtained in the absence of knowledge of the relative intensities of the DNA and collagen FTIR spectra. This was only possible because of the clear delineation in the images of the two tissue types.
In the earlier study of lymphoid nodal tissue and metastatic OSCC, an MLA achieved excellent discrimination between the two tissue types in terms of the ratio of absorbances at 1252 cm−1 and 1285 cm−1. These two wavenumbers were attributed to contributions dominated by DNA and collagen, respectively.19 The fused images obtained in this current work are consistent with those of ref. 19 as shown in Fig. 2(d), which is in better agreement with the H&E image of Fig. 1(a) than the images obtained at the individual wavenumbers of 1252 cm−1 and 1285 cm−1. It is worth exploring why the ratio image is successful, and whether the attribution of this success solely to contributions from DNA and collagen is reasonable when it is known that a third moiety, cytokeratin, makes an important contribution to the tissue. Ignoring cytokeratin, and other moieties, for the moment we can write the ratio as:
(1) |
The fused ratio image has low values in the core of the tumour–cell aggregates, which tends to show higher differentiation in the form of large cells showing increased cytoplasm enriched with cytokeratins and lower nuclear-to-cytoplasmic ratios. The absorbance ratio is higher in the non-tumour regions, emphasising the nuclei of the lymphoid cells. This is expected as lymphocytes show little cytoplasm, which results in more closely packed nuclei in the lymphoid tissue compared with the less closely packed nuclei of the differentiated centres of tumour-cell aggregates.19 The success of eqn (1) in accounting for the distribution of DNA indicates that the numerator is dominated by the contribution from DNA and the denominator by the contribution from collagen. Thus, even in the absence of knowledge of the relative intensity of the DNA and collagen spectra, we can conclude that the collagen contributions at 1252 cm−1 and the DNA contributions at 1285 cm−1 are both relatively weak, which is consistent with the original attribution of these wavenumbers to DNA and collagen, respectively.
As previously noted, three molecular contributors that are important in OSCC and supported stroma composite are collagens, DNA and cytokeratins.29,33 In this microenvironment, cytokeratins are localised in the cytoplasm of tumour cells, collagens in the stroma and DNA in the nuclei of tumour cells and immune/inflammatory cells in the stroma. The localisation has been repeatedly confirmed by multiple histochemical and immunohistochemical investigations. Instead of producing fused images at individual wavenumbers that have been assigned to known vibrational modes in particular chemical moieties, we have included all three key molecules in the analysis of the fused FTIR images by performing a least-squares fit pixel-by-pixel to model the contributions of these molecules. This has the advantage of using all of the spectral information for each molecule and allowing sub-cellular observation of molecular contributions. Combinations of more than just these three molecules were tested by considering the addition of other molecules that might be present in the tissue (such as RNA, other proteins, glycogen) but none improved on the results discussed here.
On the larger spatial scale [Fig. 3(b)], the core of the tumour-cell aggregates appears mostly red, indicating a large cytokeratin component to the tissue, whereas the edge of the aggregates is green, indicating extracellular collagen deposits. Also, we note that the collagen was associated with keratin ‘pearls’, which is consistent with previous FTIR work.29 The largest region labelled as collagen [indicated by the arrows in Fig. 3(b) and 1(a)] appears in the H&E image to be a keratinised region, implying a possible misinterpretation or a change occurring between adjacent sections. A cytokeratin stain (ESI Fig. 3†) of an adjacent tissue section showed that this area is not keratinised. Hence, the least-squares fitting has identified the presence of a molecule in a region that is not readily apparent from the H&E alone. Thus, the FTIR data has provided information that would otherwise require staining of an additional tissue section.
At higher spatial resolution [Fig. 3(c)] it is apparent that these peripheral areas are not coloured uniformly but show overlap with other spectra, as indicated by changes in colour and intensity. The lymphoid cell nuclei are mostly shown as blue or cyan in colour, confirming the known overlap between the DNA (blue) and collagen (green) spectra.31,32 Accordingly, while areas known histologically to be rich in collagen can be delineated, the non-uniform and complex appearances at the periphery of the tumour-cell aggregates are attributable to nuclei of immune cells adjacent to the nuclei of tumour cells. The periphery of tumour-cell aggregates tends to show less differentiation in the form of smaller cells, showing little cytoplasm with decreased cytokeratins and higher nuclear-to-cytoplasmic ratios. This would result in more closely packed tumour nuclei therein.
These observations at high spatial resolution could not have been made without the fusion of the FTIR images with an H&E image. This has been demonstrated by applying the same pixel-by-pixel fit to the original (unfused) FTIR data as shown in ESI Fig. 4.†
Fitting spectra of collagen, DNA and cytokeratin to the fusion of hyperspectral FTIR data and an H&E image gives excellent agreement with histopathological analysis and reveals the distributions of these molecules at high spatial resolution. This demonstrates the potential of image fusion to both identify and locate chemical moieties in a way that is not possible with either FTIR or H&E alone.
Footnote |
† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d3an00692a |
This journal is © The Royal Society of Chemistry 2023 |