STPath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images

Avatar
Poster
Voice is AI-generated
Connected to paperThis paper is a preprint and has not been certified by peer review

STPath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images

Authors

Huang, T.; Liu, T.; Babadi, M.; Ying, R.; Jin, W.

Abstract

Spatial transcriptomics (ST) has shown remarkable promise in pathology applications, shedding light on the spatial organization of gene expression and its relationship to the tumor microenvironment. However, its clinical adoption remains limited due to the low throughput of current sequencing technologies. While recent methods attempt to infer ST from WSIs using pretrained image encoders, they remain constrained by limited gene coverage, organ-specific training, and the need for dataset-specific fine-tuning. In light of this, we introduce STPath, a generative foundation model pretrained on a large-scale collection of whole slide images (WSIs) with paired ST. Through extensive pretraining with tailored noise schedules, STPath learns representations across 38,984 genes and 17 organs, enabling accurate gene expression prediction without requiring further fine-tuning. STPath integrates different data modalities, including histological images, gene expressions, organ type, and sequencing technology information, within a geometry-aware Transformer architecture. We evaluate STPath across 5 tasks spanning 23 datasets and 9 biomarkers, including gene expression prediction, spot imputation, spatial clustering, gene mutation prediction, and survival prediction. The results demonstrate STPath\'s strong ability to infer spatially resolved gene expression and reveal crucial pathological structures within tissue samples, underscoring its promise for scalable ST-based pathology applications.

Follow Us on

0 comments

Add comment