Additional file 2 of Zero-shot evaluation reveals limitations of single-cell foundation models

Additional file 2: Supplementary Figs. S1-S9. Fig. S1: Tissue composition of the multi-tissue datasets. The tissue composition of the collections used for pretraining is depicted in the upper panel: A Geneformer and B scGPT. Below, the multi-tissue datasets utilized in evaluations are visualized: C Immuneand D Tabula Sapiens. Fig. S2: Technology distribution across datasets. Fig. S3: Proposed single-cell foundation models fail to outperform cell embeddings derived from HVG or generated using the scVI model. Average BIO score calculated on the highly variable genesof the log normalized input data and on the embeddings extracted from scVI, scGPT, and Geneformer models. Median value annotated with a dashed line. A higher score indicates better performance in separating clusters. Fig. S4: UMAP projection of the embeddings improves cell type separation for scVI. Scores used for assessing cell types separation in cell embedding space across raw and UMAP projected embeddings. A Average silhouette widthscore, B Normalized mutual informationscore, C Adjusted randscore and D Average BIOscore - an average of the other scores. The higher the score – the better the performance of the model. Fig. S5: Size of the pretraining dataset correlates with the performance at separating the cell types in cell embedding space. Average BIO score calculated on the embeddings extracted from selected variations of the scGPT models. The dashed line marks the median score across datasets. Fig. S6: UMAP projection of the embeddings results in lower batch integration scores. Scores used for assessing batch integration in cell embedding space across raw and UMAP projected embeddings. A Batch integrations score based on Average silhouette widthfor batch and label, and B Principal component regressionscore. The higher the score – the better the performance of the model. Fig. S7: Performance of the proposed foundation models with respect to reconstructed expression binor ranking. The predicted bins as a function of input bins for scGPT GEP, scGPT GEPCand the agreement between input and output rankings for Geneformerfor A Pancreas, B PBMC, C PBMCD ImmuneE Tabula Sapiensdatasets shown. Fig. S8: Pearson’s correlation of input and Geneformer’s predicted expression rankings with respect to fraction of masked input tokens. Shown here are results for Pancreasand PBMCdatasets. Fig. S9: Pearson’s correlation of input and Geneformer’s predicted expression rankings with respect to fraction of masked input tokens. Shown here are results for PBMC, Immuneand Tabula Sapiensdatasets.