Image-Based Computational Immune Synapse Analysis
We hypothesized that immune synapses between cells can be detected based on the preferential localization of proteins at interfaces where cells contact one another. While this concept is intuitive, it has been unknown whether current spatial proteomic profiling technologies, such as imaging mass cytometry, have sufficient resolution and precision to reveal such synapses. It also remains poorly understood what fraction of cell-cell contacts will have active synapses in a given tissue sample, as this quantity will depend on the cell types of interest, cell densities, the strength and persistence of synapses, and the tissue type. In addition, synapse quantification is affected by the accuracy of cell segmentation algorithms in identifying cell-cell interfaces in 2D spatial proteomic images. Given these uncertainties, a statistical approach that not only evaluates individual cell-cell contacts, but also integrates across an image, would be valuable to quantifying synaptic activity.
To interrogate cell-cell interactions in the in situ immunological context, we defined an image-based T-cell immune synapse metric. T-cells are known to interact with antigen presenting cells (APCs), which motivated us to focus on the behavior of TCR proteins at interfaces between T-cells and APCs; however, the approach we describe here is generalizable to any cell types and synapse markers. A T-cell’s synapse strength σ is defined as the logarithm of the mean CD3 signal intensity (a proxy for TCR signal) at the membrane region in contact with an antigen presenting cell (APC) divided by the mean signal in the noncontact region (Fig. 1A). A positive synapse strength indicates T-cell synapse formation and potential functional interaction with another cell in the TME, modeling how the TCR aggregates at the synapse with an APC upon recognition of the presented antigen 16. The synapse strength σ can also be considered for each tissue sample, by averaging the values of σ for all contacts between T-cells and APCs (or specified cell type) in the sample. We also implemented a null synapse model based on random sets of contiguous T-cell membrane pixels to account for baseline CD3 aggregation (see Methods). This computational immune synapse analysis (CISA) provides a way to investigate the functional importance of cell-cell contacts from images.
To test whether CISA is capable of quantifying immune synapses in high-resolution multiplexed images, we first applied it to a published melanoma IMC dataset from Moldoveanu et al.35 This dataset contains 30 pre-treatment patient tissue microarray samples from melanoma patients that received immune checkpoint inhibition therapy. We segmented cells in each image and mapped to the cell annotations from Moldoveanu et al35. (see Methods), shown for example in Fig. 1B. We then used CISA to compute σCD3(T-CD8+, APC) for each T-cell:APC contact, i.e. the relative localization of CD3 to the contact region between a CD8 + T-cell and an adjacent APC. We also computed σCD8(T-CD8+, APC), the localization of CD8 toward the adjacent APC. We observed strong correlation (Fig. 1C) in the directional enrichment of CD3 and CD8 toward adjacent APCs, with r = 0.69 between σCD3(T-CD8+, APC) and σCD8(T-CD8+, APC), as would be expected from active T-cell:APC interactions. CD8 + T-cells can have a minor CD4 signal not expected to co-localize with CD3 in CD8 + cells, providing a control. Consistent with these expectations, the correlation of σCD3 and σCD4 (r = 0.17) in CD8 + cells is less than that between σCD3 and σCD8 (Fig. 1C). Likewise, for CD4 + T-cells, we observed a strong correlation of CD3 and CD4 toward adjacent APCs (σCD3(T-CD4+, APC) and σCD4 (T-CD4+, APC), r = 0.56), consistent with detection of TCR-mediated synapses. As expected, the correlation of σCD3(T-CD4+, APC) and σCD8(T-CD4+, APC) is lower (r = 0.26) than the CD3:CD4 correlation.
While our results show that colocalization of TCR proteins toward APCs can be detected by CISA, some protein colocalization might occur regardless of an APC contact. To evaluate the contact-independent effect, we calculated pixel-wise correlations between CD3 and CD4 (or CD8) intensity within the complete T-cell areas in each spot. CISA scores exhibited stronger correlations than the contact-naïve pixel-wise methods (Fig. 1D). This was observed for correlations of CD3 and CD8 in CD8 + T cells (t-test, p = 5.8e-9), as well as for correlations of CD3 and CD4 in CD4 + T cells (t-test, p = 3.9e-7). These results indicate that CISA captures information on contact-dependent synapse activity.
To verify the robustness of these results, we analyzed a second IMC dataset from Hoch et al.34 It consists of 167 tissue microarray (TMA) spot images from melanoma patients. This dataset showed CISA behaviors consistent with the Moldoveanu et al dataset35. CD3 and CD8 CISA correlations in CD8 + T cells were strong (r = 0.57, Fig. 1E) and higher than CD3-CD4 CISA correlations in the same cells (r = 0.09). CD3 and CD4 CISA correlations in CD4 + T cells were also strong (r = 0.43, Fig. 1E), and higher than CD3-CD8 CISA correlations in those cells (r = 0.14). As in the Moldoveanu et al dataset, CISA correlations were stronger than pixel-wise correlations (Fig. 1F), both for CD3:CD8 in Tc cells (p = 1.8e-12) and for CD3:CD4 in Th cells (p = 3.8e-12). We also analyzed the correlations of T-cell:APC CISA scores for all pairs of proteins in the IMC data (Fig. S1). These showed a rich clustering structure reflecting behaviors such as TCR co-occurrence, T-cell specificity, and APC-specificity.
To clarify the observed behaviors, example images of T-cell:APC contacts with positive or negative CISA scores are shown in Fig. 2A and B. These images show the variability in signal along membrane boundaries, supporting the importance of statistical approaches that integrate information along over entire cells and tissue images to provide robustness. For example, in addition to CD3, CD4 and CD8a, we were also able to evaluate correlations in CISA scores between CD3 and other T cell surface proteins, including ICOS (CD278) and CD7 (Fig. 2C, D). It is worth noting that ICOS has a significantly (p = 3e-9) lower expression level than CD4 in CD8 + T cells (Fig. 2E), but the correlation between σCD3(T-CD8+, APC) and σCD278(T-CD8+, APC) (Fig. 2D) is stronger than between CD3 and CD4 in those cells. This suggests CISA can detect polarization of cell membrane proteins even at a low expression level. Together, these results support CISA as a robust method to quantify immune synapses between T cells and neighboring APCs.
Whole-slide images enable regional analysis of tumor microenvironmental interactions
To further verify the applicability of CISA to other types of image data, we generated and analyzed histocytometry 37 whole slide images of metastatic melanoma 38. These WSIs are much larger than TMA spots and are based on fluorescence, providing substantial differences from IMC. We analyzed a cohort of 21 human metastatic melanoma samples from 20 patients (see Methods). A 6-marker panel was employed to characterize cell phenotypes across whole-slide sections, providing extensive data for identification of TME features and interactions. The images in our cohort averaged 67 mm2 of tissue imaged at a resolution of 663 nm per pixel.
We first analyzed regional cell type prevalence and spatial co-occurrence in these datasets, as cellular composition 39 and APC expression 38 could vary between the tumor stroma and tumor nests. The large size of WSI images makes them better than TMA spots for investigating potential region-specific T-cell interactions. After segmenting images into intratumor and stromal regions (see Methods, Fig. 3A, B), we observed that T-cells and macrophages are both abundant in the stroma, but macrophages make up the bulk of the intratumor immune infiltrate. Within the tumor, the cohort median density of T-cells is 86.3 cells per mm2, while the density of macrophages is 252.6 cells per mm2 (macrophage > T-cell: p = 2.4x10− 4, Fig. 3C). Macrophages also exhibit differential melanoma antigen loading between the intratumoral and stromal regions, where loading is defined by the presence of melanoma antigen in the cytoplasm (Fig. 3D, see Methods). In the intratumoral region, the majority of macrophages are loaded with unprocessed melanoma antigen (Fig. 3E, cohort median = 72.3%). In the stroma, a significantly lower fraction of macrophages is loaded (cohort median = 7.7%, p = 9.5x10− 7).
T-cells colocalize with macrophages in a region-dependent manner in melanoma
Because of the differential antigen-loading of macrophages between the intratumoral and stromal regions, we hypothesized that cells of the TME interact in a region-specific manner. We investigated this using a radial distribution function (RDF) analysis to identify spatial relationships between cell types in the TME (Fig. S2A,B). We applied RDF analysis to quantify the distance-dependent density of TME cells relative to T-cells (see Methods). We additionally devised a metric to quantify T-cell/TME cell colocalization relative to null expectations, ΔCDF, defined from the RDF curve as the excess of colocalization relative to a label-permuting null model – an approach that controls for variations in local cell density (Fig. S2C).
The use of RDF analysis to reveal cell-cell spatial associations is illustrated in Fig. 4 for sample Mel-512, with T-cells as the reference cell. In the intratumoral region, a peak in the RDF for loaded macrophages at approximately 12 µm indicates that T-cells and loaded macrophages are often in close proximity (Fig. 4A, solid cyan line). The observed distribution is above the cell-permuted null expectation (Fig. 4A, dashed cyan line) indicating excess colocalization. T-cell colocalization with loaded macrophages is significant when calculated over the full histocytometry cohort (\(\stackrel{-}{{\Delta }\text{C}\text{D}\text{F}}\)=4.184, 1-sample t-test p = 9.6x10− 10, Fig. 4B). Unloaded macrophage colocalization with intratumoral T-cells is also observed (Fig. 4A), though the effect is weaker than for loaded macrophages. Such colocalization is moderately significant across the cohort (\(\stackrel{-}{{\Delta }\text{C}\text{D}\text{F}}\)=0.908, 1-sample t-test p = 0.026, Fig. 4B). Tumor cells have high absolute densities near intratumoral T-cells, but do not significantly colocalize (\(\stackrel{-}{{\Delta }\text{C}\text{D}\text{F}}\)=−1.311, 1-sample t-test p = 1.0, Fig. 4A,B). Intratumor T-cell colocalization with loaded macrophages is significantly greater than with either unloaded macrophages or tumor cells (Fig. 4B, p = 1.2x10− 4 and 5.9x10− 5 respectively).
In the stromal regions, cell co-localization relationships differ from those within the tumor. For example, in the stroma of sample Mel-512, unloaded macrophages have a small RDF peak with respect to T-cells at a distance of approximately 10 µm (Fig. 4C, blue). This colocalization exceeds the null expectation by a statistically significant but small margin (\(\stackrel{-}{{\Delta }\text{C}\text{D}\text{F}}\)=2.642, 1-sample t-test p = 1.5x10− 6, Fig. 4D). On the other hand, loaded macrophages have lower colocalization with stromal T-cells than expected (Fig. 4C). Negative colocalization between these cell types is observed across the cohort (\(\stackrel{-}{{\Delta }\text{C}\text{D}\text{F}}\)=−3.567, 1-sample t-test p = 1.0, Fig. 4D). The unloaded macrophage ΔCDF is significantly greater than for loaded macrophages (p = 5.9x10-5). Meanwhile, T-cells exhibit strong colocalization to other T-cells both in the stroma and within the tumor (\(\stackrel{-}{{\Delta }\text{C}\text{D}\text{F}}\)=4.616 and 4.713, 1-sample t-test p = 1.8x10− 6 and 4.7x10− 7, stroma and intratumor respectively Fig. 4B, D; S2B).
CISA reveals region-specific T-cell/macrophage synapses in whole slide images
Given the distinct cell co-localization behaviors in the intratumor and stromal regions, we hypothesized that CISA analysis would detect regional differences in immune synapse formation within whole slide histocytometry images. Indeed, CISA analysis of intratumoral regions showed that intratumor T-cells tend to form synapses to loaded macrophages (\(\stackrel{-}{\sigma }\)=0.098, 1-sample t-test p = 3.9x10−3, Fig. 5A), consistent with the IMC results. However, this relationship was not observed in the stroma, and stromal T-cells instead tend to form synapses to unloaded macrophages (\(\stackrel{-}{\sigma }\)=0.059, 1-sample t-test p = 5.0x10−3). Both synapse strengths are significantly greater than the null model (p = 2.4x10−5 and 2.9x10−6 respectively). T-cell synapses to loaded macrophages are significantly stronger within the tumor than in the stroma, while T-cell synapses to unloaded macrophages are significantly stronger in the stroma than in the tumor (Fig. 5B, p = 6.9x10−5 for both respectively). Within the tumor, T-cell synapse strength to loaded macrophages is significantly stronger than to unloaded macrophages (Fig. 5B, p = 3.7x10−4). In the stroma, T-cells have significantly stronger synapses to unloaded macrophages than to loaded macrophages (p = 1.3x10−3). In the tumor, T-cells showed no tendency to form synapses to melanoma cells (\(\stackrel{-}{\sigma }\)=−0.101, 1-sample t-test p = 0.98), with synapse strength not significantly greater than the null model (p = 0.41, Fig. 5A). The greater T-cell synapse strength to loaded macrophages than to melanoma cells is also statistically significant (p = 6.9x10−5).
We employed synapse-focused super-resolution imaging to further verify T-cell synapses in the TME. Figure 5C shows two T-cell-macrophage contacts with different CD3 aggregation behavior via stimulated emission depletion microscopy (Fig. 5C). The left T-cell exhibits relatively homogenous CD3 along its membrane without aggregation towards its neighboring macrophage. Volumetric rendering suggests a CD3-containing T-cell protrusion towards the macrophage which may signify early synapse formation or antigen sampling (Fig. S3A), though there is not a reciprocal concentration of ICAM-1 in the macrophage that would demonstrate mature synapse formation (Fig. S3B). The right T-cell shows CD3 concentration towards the dendrite protrusion on the target macrophage (Fig. 5C, yellow box). Volumetric rendering shows not only contact between the T-cell CD3 and the macrophage dendrite but also reciprocal macrophage ICAM-1 in the contact area, as expected from mature synapse formation and functional interaction (Fig. 5D, Fig. S3B).
T-cell-macrophage interactions are associated with T-cell proliferation in metastatic melanoma
T-cells recognizing their cognate antigen in conjunction with pro-inflammatory signals undergo clonal expansion 40 typically within the lymph node, but in vivo mouse data suggest this may occur in the tumor as well 15. We therefore hypothesized that proliferating T-cells in the TME might have stronger synapse strengths than other T-cells. We extended CISA to address this by classifying T-cells as proliferating (Ki-67+) or non-proliferating (Ki-67−) from KI-67 imaging data. We found that intratumor Ki-67 + T-cells have significantly stronger synapses to loaded macrophages than Ki-67 − T-cells (Fig. 6A, p = 4.2x10− 4) and significantly greater than the Ki-67 + T-cell null model (p = 9.5x10− 7). There was no significant difference in synapse strength between Ki-67 + and Ki-67 − stromal T-cells, unlike in the intratumor region (Fig. 6B). Ki-67 + T-cell synapse strength to melanoma cells is also stronger than for Ki-67 − T-cells (p = 0.021), although not significantly greater than the null model. To further address the synapse-proliferation association, we compared the fraction of T-cells that are Ki-67 + in the synapse positive and non-positive groups. For intratumoral T-cells in contact with loaded macrophages, we observed a significantly greater fraction of Ki-67 positivity in synapse-positive T-cells compared to non-positive synapse T-cells (p = 2.8x10− 4, Fig. 6C). We observed a similar association between KI-67 and T-cell/melanoma synapses, albeit with lower statistical significance (p = 0.014).
On the other hand, T-cell synapse formation with unloaded macrophages was not associated with a proliferative response. We observed no significant difference in average synapse strength between Ki-67 + and Ki-67 − T-cells in contact with unloaded macrophages, either within the tumor or in the stroma (Fig. 6A, B). Additionally, intratumoral Ki-67 + T-cells have significantly higher average synapse strengths with loaded macrophages than with unloaded macrophages (Fig. 6A, p = 6.0x10− 5). In the stroma, T-cell Ki-67 status is not associated with synapse strength to an adjacent macrophage, regardless of whether the macrophage has an antigen load.
Synapse analysis of breast cancer imaging mass cytometry data reveal T-cell/B-cell interactions
To test the applicability of the RDF and CISA approaches to other types of tumors, we applied these methods to a cohort of breast cancer IMC images of primary tumors from 281 patients 25, of which 275 had associated survival data and clinical subtyping. Because these IMC images were derived from tissue microarrays, individual IMC spot images had far fewer cells than histocytometry images, resulting in noisy and uninformative sample-level T-cell RDF curves. Consequently, we aggregated all breast cancer IMC images to generate a single cohort RDF for each class of cell-cell co-localizations. We did not distinguish antigen-loaded from -unloaded macrophages in this analysis because loaded macrophages make up only 0.3% of macrophages in the stroma and only 11% of the total macrophage content.
RDF analysis showed no distinguishable colocalization of T-cells with tumor cells within breast tumors (ΔCDF = − 0.465, Fig. S4), similar to T-cells within melanomas. This behavior was consistent across clinical subtypes, with HER2 + tumors showing the largest (negative) deviation from the null (ΔCDF = − 2.260, Fig. S5A-C). Intratumor T-cell/macrophage associations were close to null expectations (ΔCDF = 0.509). T-cell/B-cell interactions were below null expectations, though noisy. This is related to the fact that only 2.6% of B-cells reside within tumor nests, which hinders statistical assessment of colocalization. In the stromal region, T-cells show a co-localization with both macrophages and B-cells, and these effects are of comparable magnitude (ΔCDF = 1.261 and 1.567 respectively). The T-cell co-localization with B-cells is apparent despite the presence of B-cells in only 130 of 275 images, compared to 272 patient images containing macrophages. Stromal T-cell colocalization with B-cells and macrophages is consistent across clinical subtypes with some variation in effect size (Fig. S5A-C).
We then applied CISA to the breast cancer IMC images to investigate T-cell synapses with macrophages and B-cells. In the intratumoral region (Fig. 7A), T-cells do not have significant synapse formation to macrophages (\(\stackrel{-}{\sigma }\)=0.078 1-sample t-test p = 0.28) or tumor cells (\(\stackrel{-}{\sigma }\)=−0.235, 1-sample t-test p = 1.0), consistent with the lack of T-cell/cancer synapses in melanoma. A noteworthy effect in the breast IMC images is that intratumor T-cells form strong synapses with B-cells (\(\stackrel{-}{\sigma }\)=0.839, 1-sample t-test p = 5.5x10−5). These effect sizes are larger than those for the T-cell/loaded macrophage interactions in melanoma (Fig. 5A, 5B, 6A, 6B). Remarkably, this synapse effect is highly significant despite the lack of frequent colocalization between B-cells and T-cells -- intratumor T/B-cell contacts occur in only 46 of 275 (17%) images. In the stroma, T-cell synapse behaviors are consistent with the tendency of both macrophages and B-cells to co-localize with T-cells. Stromal T-cells have significant synapse formation with macrophages (\(\stackrel{-}{\sigma }\)=0.088, 1-sample t-test p = 0.035, Fig. 7B), on par with the effect size seen in melanoma (Fig. 5A). Moreover, stromal T-cells form even stronger immune synapses with B-cells (\(\stackrel{-}{\sigma }\)=0.601, 1-sample t-test p = 6.8x10−16). Curiously, due to the paucity of B-cells, stromal T-/B-cell contacts are present in only 104 of 275 (38%) patients, compared to macrophage contacts being present in 235 patients. Synapse behaviors do not significantly differ across breast cancer subtypes (1-way ANOVA p > 0.05 for each target cell, Fig. S5D-F).
Stromal T-cell synapses with B-cells are associated with improved breast cancer survival
B-cells can hinder or help tumor growth depending on the context, and prior studies have suggested T-cells may be necessary for the beneficial B-cell effects 41. We therefore tested whether the T-cell synapses to B-cells were associated with clinical outcomes in breast cancer. We first analyzed the 173 hormone-receptor positive HER2 negative (HR + HER2−) patients. We divided these patients into two groups based on synapse formation to B-cells: 1) those with a stromal T-cell:B-cell σ > 0 (representing contact and synapse formation), vs. 2) those with either σ <= 0 (representing contact with no synapse formation) or no contact (and therefore no synapse formation) between T- and B-cells. We observed a significant difference in disease-free survival (DFS) between these groups by Kaplan-Meier (KM) estimation, with improved survival for group 1 (p = 0.016, Fig. 7C). This DFS benefit is specific to the stroma, as splitting by the intratumor σ does not result in a significant survival difference (p = 0.88, Fig. S6A), nor does splitting by the combined intratumor and stromal T-cell σ (p = 0.106, Fig. S6B). DFS differences cannot be explained simply by contact between stromal T- and B-cells: grouping patients by the presence or absence of T-cells in contact with B-cells regardless of synapse strength resulted in a non-significant DFS difference (p = 0.131, Fig. S6C). Additionally, grouping patients by the proportion of T- and B-cell infiltration was not sufficient for a significant difference in DFS (p = 0.073, Fig. 7D). We next integrated these factors into a multivariable survival regression model using Cox Proportional Hazards, regressing the aforementioned factors and patient metadata against DFS. Stromal T-cell synapse formation with B-cells exhibited a negative and significant hazard ratio (log HR = − 1.45, CI [− 2.45, − 0.45], p < 0.005, Fig. 7E) in distinction from other factors, including increased immune cell infiltration.
We next investigated the association of DFS with T-cell/B-cell interactions in other breast cancer molecular subtypes. The Jackson et al. cohort25 has 48 TNBC (HR − HER2−) and 52 HER2+ (including 29 HR + and 23 HR−) patients. Splitting patients again by stromal T-cell σ to B-cells, the survival benefit in TNBC is significant by KM survival estimation (p = 0.018, Fig. 7F). No samples had a non-positive sample-average synapse strength in the TNBC cohort, either when considering only stromal or both stromal and intratumor synapses. As such, we could not test for additional survival benefits of T-cell-B-cell contact. Survival was not significantly different when splitting patients by quantity of T- and B-cell infiltration (p = 0.377, Fig. 7G). Multivariable survival regression demonstrated T-cell synapse formation to B-cells has a strong effect on DFS after controlling for infiltration factors (log HR = − 2.45, CI [− 4.52, − 0.38], p = 0.02, Fig. 7H). In the HER2 + cohort, KM survival estimation shows no DFS benefit for stromal T-cell synapse formation to B-cells (p = 0.727, Fig. S6D) or increased T- and B-cell infiltration (p = 0.515, Fig. S6E). Rather, the presence of hormone receptors estrogen receptor (ER) and progesterone receptor (PR) are the factors influencing survival in the Cox Proportional Hazards model with opposite effects (ER: log HR = 1.56, CI [0.59, 2.72], p = 0.01, PR: log HR = − 1.72, CI [− 3.02, − 0.41], p = 0.01, Fig. S6F). Thus, T-cell synapses to B-cells have strong survival benefits in TNBC but not HER2 + patients in this cohort.