In silico perturbation as an adversarial attack: combinatorial searchover cell states in single-cell foundation models

Abstract Background: Identifying which genes to perturb in order to drive a cell towards a desired state is a common problem in biology, and the answer is often a combination of genes. Single-cell foundation models can predict the effect of forcing a gene on without an experiment, but in silico perturbation has been confined to single genes or to a few hundred enumerated combinations. In token-based models, choosing which genes to force on is formally the same optimization problem as a discrete adversarial attack on a language model. We therefore investigated whether Greedy Coordinate Gradient (GCG), a search developed for such attacks, can be used to find combinations of genes. Results: Our results suggest that GCG is useful for finding combinations of genes. In Geneformer, the effect of a set was well below the sum of the individual effects of its members, which suggests that assembling combinations from single-gene effects is not a promising approach. GCG exceeded forward-selection greedy search at equal cost in two models (Geneformer and scGPT) and at every set size. Against published reprogramming protocols, GCG recovered the reported factors at a rate well above chance and higher than the random-pool and all-gene greedy searches at equal cost, although ranking genes by their individual effect gave a higher recovery rate. Recovery corresponded to whether a search used the embedding gradient, which suggests that the gradient is useful for narrowing the search, whereas stronger optimization of the objective did not raise recovery, which suggests that the similarity induced by the model does not fully reflect the regulatory logic of genes. Conclusions: These results are obtained within the constraints of the foundation models; nevertheless, searching for combinations of factors with foundation models extends in silico perturbation beyond single genes, reveals a property of the induced objective that single-gene evaluation does not, and points to a new direction for using and evaluating the models.