The Future Is Now: The Era of LLM × Single-Cell Has Begun

· 8 min read · Analysis Notes

The Future Is Now: The Era of LLM × Single-Cell Has Begun

I'm sure everyone has been seeing it everywhere lately. On September 12th, OpenAI released its powerful new language model o1, introducing a "chain-of-thought" reasoning mode that allows LLMs to decompose complex problems and reason through them step by step — just like a human — while making the thinking process visible. o1 is reportedly capable of solving complex mathematical problems at an olympiad gold-medal level, conducting scientific research, and proposing well-reasoned research directions — performance on par with a PhD-level specialist.

Source: https://openai.com/index/introducing-openai-o1-preview/

You might have noticed that the official release page specifically highlighted one use case: "helping researchers annotate cell sequencing data." Single-cell omics, long heralded as the future of biological research, has finally caught the attention of large language models.

Source: https://openai.com/index/introducing-openai-o1-preview/

Traditional single-cell annotation has long relied on manual annotation as the gold standard. Manual annotation is a complex and painstaking process involving analysis and judgment at multiple levels. It primarily involves two key steps:

  1. Identifying differentially expressed genes (DEGs): Researchers compare gene expression profiles across cell populations to find genes significantly upregulated or downregulated.
  2. Interpreting gene expression patterns: Analyzing the expression levels and distribution of canonical marker genes across cell clusters.

Both steps demand that researchers have deep, comprehensive knowledge of the target cell types or tissues — including cell-type-specific marker genes and gene expression patterns in various cellular states (activation, differentiation, stress, etc.) — as well as the ability to recognize and filter out technical noise, such as contamination, empty droplets, doublets, and stress-induced transcriptional artifacts from experimental handling. It is precisely because manual annotation relies so heavily on expertise and experience that it is often considered an "art," one that takes years of training and practice to master.

However, as single-cell datasets have grown dramatically in size and complexity, purely manual annotation faces mounting challenges in terms of efficiency and consistency. A variety of automated and semi-automated annotation approaches have consequently been developed. Broadly speaking, these strategies fall into the following (non-exclusive) categories:

  1. Marker gene–based methods: Cell type is determined by matching or scoring against predefined marker gene sets (probability- or expression-based). Representative methods include scType and SCINA.
  2. Reference mapping methods: Cell types are inferred automatically by mapping cells onto annotated reference datasets. Representative methods include SingleR and Azimuth.
  3. Machine learning methods: Classification models are trained to learn annotation rules directly from data. Representative methods include CellTypist and ACTINN.
  4. Functional analysis methods: Gene regulatory networks or pathway enrichment analyses assist annotation. Representative methods include GSEA and SCENIC.

Each strategy has its trade-offs in speed, requirements for reference genes or datasets, computational demands, and ability to discover novel cell types. The current consensus is still to manually verify results after automated annotation or cross-validate using multiple methods — which still places substantial demands on researchers. This is exactly the kind of task where LLMs can step in.

Using GPT to assist with single-cell annotation is no longer a novelty. Back in March 2024, a paper that left me thinking "how did this make it into Nature Methods?" described how providing GPT with cluster marker genes enables cell type prediction at an average accuracy exceeding 75% (agreement with manual annotation). The approach is refreshingly simple: list the top differentially expressed genes for each cluster as marker genes, then ask GPT to predict cell types. The authors also wrote a compact R package called GPTCelltype — just one function, 50 lines of code — which accepts a Seurat object in R, calls the GPT API, and returns the predicted cell types as a vector.

Source: Hou, W., Ji, Z. Assessing GPT-4 for cell type annotation in single-cell RNA-seq analysis. Nat Methods 21, 1462–1465 (2024). https://doi.org/10.1038/s41592-024-02235-4

Other researchers have pursued the development of single-cell–specific generative foundation models. A notable example is scGPT, also published in Nature Methods, which was trained on 30 million single-cell profiles and can be applied to annotation, data integration, and perturbation response prediction. However, the barrier to entry is high — users need their own server infrastructure to deploy and fine-tune the model — and there is no ready-made R package for integration with the Seurat ecosystem.

Source: Cui, H., Wang, C., Maan, H. et al. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nat Methods 21, 1470–1480 (2024). https://doi.org/10.1038/s41592-024-02201-0

The use of LLMs in real scientific research workflows remains a subject of debate. Researchers' hesitation likely stems from the opacity of LLM reasoning and the instability of LLM outputs — problems shared by other AI models such as deep neural networks. The algorithm is a black box: you know the input and output, but not why the AI made a particular decision. Models like GPT-4 and Claude are even more black-box–like, because the exact composition of their training data is unknown. LLMs are fundamentally different from traditional machine learning or statistical/mathematical models — their outputs come without p-values, probabilities, or confidence intervals. The fact that LLMs have gained academic acceptance so quickly, in a community that values statistical significance and reproducibility, is itself a testament to their power.

There is no question that LLMs can meaningfully assist with analysis. For bioinformatics enthusiasts who aren't deep experts, leveraging off-the-shelf LLMs that require no retraining saves both time and effort. Moreover, prompt engineering and providing more contextual information can substantially improve the accuracy and reliability of LLM outputs.

Take cell annotation as an example. A bit of prompt engineering on top of GPTCelltype can yield noticeably better results. The original prompt template in GPTCelltype is:

"Identify cell types of [tissue name] cells using the following markers separately for each row. Only provide the cell type name. Do not show numbers before the name. Some can be a mixture of multiple cell types."

Here are several ways to improve it:

  • Assign the LLM a role: For example, "You are an expert in molecular and cellular biology" — this primes the model to draw on relevant domain knowledge.
  • Constrain the output format: For example, "Provide the predicted cell type in the following format: Cluster - Cell Type. No additional explanation." — this makes outputs more consistent and readable.
  • Sharpen the instructions: For example, "Do not use 'Unknown' as a cell type. Provide your best guess." — this prevents the AI from "cheating" by simply saying it doesn't know.

Providing more information also boosts accuracy. For example, in addition to upregulated genes, also supply downregulated genes. When several clusters belong to the same broad cell category but their DEG lists lack that category's canonical markers (e.g., epithelial cells all express Epcam; immune cells all express Ptprc), you can additionally supply the expression levels of key marker genes and GSEA pathway enrichment results (e.g., the top enriched pathways). You can also query the LLM multiple times or compare outputs across different models to improve robustness.

In my own experiments, combining these strategies provided a further accuracy boost. As shown in the figure below — using GSE185344 (prostate cancer patient single-cell data, ~50,000 cells) as an example — after optimizing the prompt and providing GSEA enrichment results, GPT-4 successfully distinguished smooth muscle cells from fibroblasts, and NK cells from T cells. Myeloid cells were further resolved into neutrophils and macrophages, and the final results were almost perfectly concordant with manual annotation.

Comparison of annotation methods using GSE185344 data. Top-left: original GPTCelltype (GPT-4). Top-right: GPT-4 with optimized prompt. Bottom-left: Claude 3.5 Sonnet with the same optimized prompt. Bottom-right: manual annotation. LLM performance across the board is quite impressive!

On model selection — all accessed via API — gpt-3.5-turbo is the fastest (results in 2–3 seconds) and cheapest (under $0.01 per annotation run); gpt-4o is close behind (~4–5 seconds, ~$0.02 per run); gpt-4 is the most expensive, taking ~10–15 seconds and costing $0.40–$0.50 per run — orders of magnitude pricier. Claude 3.5 Sonnet performs similarly to gpt-4o but costs ~$0.06 per run. Overall performance differences between models are modest, though gpt-4 was the only model capable of accurately annotating neutrophils.

Looking ahead, this paradigm of directly interacting with LLMs to assist analysis will very likely expand rapidly to other aspects of single-cell data analysis. By supplying LLMs with DEG lists or pathway enrichment results, they can identify conserved marker genes and pathways to facilitate multimodal or cross-species data integration, or predict cellular responses to drugs and genetic perturbations based on the most salient genes or pathways. LLMs can also interpret the biological meaning of "meta-genes" or gene modules emerging from unsupervised clustering, helping to discover conserved or condition-specific biological functions and uncover gene regulatory networks. With their continuously updated and vast knowledge bases, LLMs are uniquely well-positioned for data interpretation and prediction tasks.

These are some of my thoughts — I hope they give you something to think about. If anything is inaccurate, please feel free to reach out and discuss!

Originally posted on WeChat Official Account: bioinfo_studyroom

Bioinformatics