mengflz/pasta-benchmark-rep-genes

PASTA Benchmark Models - Representative Genes

This repository contains PASTA (PAthological Super-resolution Transcriptome-guided pAradigm) models trained for predicting 100 representative gene expressions from H&E-stained pathology images.

Model Description

PASTA is a deep learning framework that can predict spatial gene expression patterns directly from histology images. These benchmark models were trained on HEST-library human data with various foundation model backbones to evaluate their performance on gene expression prediction tasks.

Model Architecture

• Base Architecture: Modified Dense Prediction Transformer (DPT)
• Task: Spatial gene expression prediction (100 representative genes)
• Input: Patched H&E-stained pathology image patches (224×224 pixels); Paired ST data(optional)
• Output: Super-resolution spatial gene expression maps

Available Models

This repository includes 16 models trained with different backbone encoders:

• PLIP_rep_genes.pt - Based on PLIP backbone
• UNI_rep_genes.pt - Based on UNI backbone
• Virchow_rep_genes.pt - Based on Virchow backbone
• Hibou-B_rep_genes.pt - Based on Hibou-B backbone
• Phikonv2_rep_genes.pt - Based on Phikon backbone
• H-optimus-1_rep_genes.pt - Based on H-optimus-1 backbone
• UNIv2_rep_genes.pt - Based on UNI backbone
• Kaiko-L_rep_genes.pt - Based on Kaiko-L backbone
• H-optimus-0_rep_genes.pt - Based on H-optimus-0 backbone
• Kaiko-B_rep_genes.pt - Based on Kaiko-B backbone
• Hibou-L_rep_genes.pt - Based on Hibou-L backbone
• Virchow2_rep_genes.pt - Based on Virchow backbone
• CONCH_rep_genes.pt - Based on CONCH backbone
• gigapath_rep_genes.pt - Based on gigapath backbone
• Phikon_rep_genes.pt - Based on Phikon backbone
• UNI_pos_emb_rep_genes.pt - Based on UNI backbone(trained with paired ST data)

Notes on variants:

• pos: Trained with coarse-grained pathway score/gene expression as positional embeddings. In these variants, you can use coarse-grained pathway scores/gene expression as an optional input.

Usage

Installation

pip install torch torchvision huggingface_hub

Loading a Model

import torch
from huggingface_hub import hf_hub_download

# Download model weights
model_path = hf_hub_download(
    repo_id="mengflz/pasta-benchmark-rep-genes",
    filename="UNI_rep_genes.pt"
)

# Load the model
checkpoint = torch.load(model_path, map_location='cpu')

Partial Inference Example

🌟 To use PASTA models for inference, please follow the code in PASTA codebase and confirm that you have the access to related backbone foundation models.

import torch
from pasta.model import PASTA
from huggingface_hub import hf_hub_download
from PIL import Image
import torchvision.transforms as transforms

# Download model checkpoint
model_path = hf_hub_download(
    repo_id="mengflz/pasta-benchmark-rep-genes",
    filename="UNI_rep_genes.pt"
)

# Initialize PASTA model
model = PASTA(
    model_name='UNI',
    pathway_dim=100,
    freq_flag=True,
    N=128,
    scale=1.0
)

# Load trained weights
checkpoint = torch.load(model_path, map_location='cpu')
model.load_state_dict(checkpoint, strict=False)
model.eval()

# Prepare input image
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], 
                       std=[0.229, 0.224, 0.225])
])

image = Image.open("path/to/histology_image.png")
input_tensor = transform(image).unsqueeze(0)

# Run inference
with torch.no_grad():
    feat, feat_mean, feat_recon = model(input_tensor)
    # feat: spatial gene expression map [B, 100, H, W]
    # feat_mean: average expression per gene [B, 100]
    # feat_recon: reconstructed image [B, 3, H, W]

Model Performance

These models were benchmarked on HEST-library for gene expression prediction performance. The evaluation metrics including PCC, SSIM and RMSE at different resolution(100µm, 64µm, 32µm, 16µm).

Training Details

Please refer to the original PASTA paper or PASTA Github repository for more details.

Citation

If you use these models in your research, please cite:

License

This project is licensed under GPL-3.0. Please note that the underlying foundation models (UNI, Virchow, etc.) have their own licenses that must be respected.

Acknowledgments

• Foundation models: UNI, UNIv2, Virchow, Virchow2, CONCH, Phikon, PhikonV2, gigapath, H-optimus, Kaiko, Hibou, PLIP
• CLAM for data processing
• DPT architecture inspiration: isl-org/DPT

Contact

For questions and issues, please open an issue on the repository or contact the authors.

Repository Structure

.
├── README.md                          # This file
├── models_config.json                 # Configuration for all models
├── UNI_rep_genes.pt                   # Model weights
├── UNIv2_rep_genes.pt
├── Virchow_rep_genes.pt
└── ... (other model files)