The research team of the Chinese Academy of Sciences released two important papers: the release of the first large-scale model of the basis of life across species, and the release of a new AI model for cell fate prediction

Author | Chinese Academy of Sciences Multidisciplinary Research Team

Editor | ScienceAI

Known as one of the three major scientific projects of mankind in the 20th century The genome project has kicked off an in-depth analysis of the mysteries of life. Due to the multi-dimensional and highly dynamic nature of life processes, it is difficult for traditional experimental research methods to systematically and accurately decipher the underlying common laws of the genetic code. It is urgent to use powerful computing technology to achieve representation modeling and knowledge discovery of genetic data.

Currently, artificial intelligence technology with large models as the core has triggered revolutions in fields such as computer vision and natural language understanding, demonstrating in-depth understanding of data and knowledge, and is expected to be applied in the field of life science research, systems To accurately decipher the underlying common laws of genetic codes

Recently, the "Xcompass Consortium" (Xcompass Consortium) composed of a multi-disciplinary interdisciplinary research team of the Chinese Academy of Sciences has made important breakthroughs in artificial intelligence empowering life science research. Successfully Constructed the world's first large-scale model of the basis of cross-species life - GeneCompass. This model integrates the transcriptome data of more than 126 million single cells of humans and mice, and integrates four types of prior knowledge including promoter sequences and gene co-expression relationships. The number of basic model parameters reaches 130 million, realizing the control of gene expression. Panoramic learning and understanding of regulatory laws simultaneously supports prediction of cell state changes and accurate analysis of various life processes, demonstrating the great potential of artificial intelligence in empowering life science research.

The study is titled "GeneCompass: Deciphering Universal Gene Regulatory Mechanisms with Knowledge-Informed Cross-Species Foundation Model" and was published on bioRxiv.

Paper link: https://www.biorxiv.org/content/10.1101/2023.09.26.559542v1

In addition, The team also simultaneously released a gene regulatory network generation model based on transfer learning, CellPolaris, which can accurately identify core factors for cell fate conversion and has the ability to simulate transcription factor perturbations.

The research is titled "CellPolaris: Decoding Cell Fate through Generalization Transfer Learning of Gene Regulatory Networks" and was published on bioRxiv.

Paper link: https://www.biorxiv.org/content/10.1101/2023.09.25.559244v1

GeneCompass: The first large-scale model of the basis of life across species

Individual mammals typically contain tens of thousands to tens of trillions of cells. Although all cells in an individual contain the same genetic sequence, the fate and function of each cell vary widely due to its unique spatiotemporal context. Such a sophisticated life process is controlled by a complex gene expression regulation system

In order to enhance the understanding of the essential laws of life and innovate the diagnosis and treatment of various major diseases, it is necessary to study the ubiquitous gene regulation mechanisms of life. Explore deeper. However, traditional research methods have low throughput and are limited to a single model organism, and cannot reveal complex gene regulatory mechanisms. In recent years, breakthroughs in single-cell omics technology have produced a large number of gene expression profile data of different types of cells. , providing a data basis for interpreting gene-gene interactions. At the same time, the development of deep learning, especially the emergence of large generative models, can comprehensively summarize the nonlinear regulation mechanism of massive data learning in different cell states, bringing unprecedented opportunities to life science research.

A large model of the basics of life across species, including 120 million cells and 130 million parameters

Currently, a single species has been obtained worldwide The scale of single-cell transcriptome data is only in the tens of millions, which is difficult to fully support the training of large models of basic life models used to analyze complex life processes.

The team collected open source single-cell transcriptome data from different species, and through pre-processing processes such as screening, cleaning, and normalization, established the largest known high-quality database, including more than 126 million cells in mice and humans. The training data set scCompass-126M adopts a deep learning architecture based on the Transformer self-attention mechanism, which can capture the long-term dynamic correlation between different genes in different cell backgrounds, and the model parameter size reaches 130 million. In order to achieve high-resolution characterization of life processes, GeneCompass dual-encodes gene numbers and expression levels for the first time, enabling effective and sensitive extraction of correlations between genes. This enables GeneCompass to provide more precise analysis of gene-gene interactions under a variety of specific conditions, such as cell types and perturbation states.

Embedding prior knowledge during pre-training can effectively improve model performance

The model effectively integrates promoter sequences, known gene regulatory networks, gene family information and gene co- Expressing the relationship between four kinds of biological prior knowledge, adding human annotation information encoding, improves the understanding of complex feature correlations between biological data. Through training and integrating data information and prior knowledge of different species, GeneCompass is expected to improve the efficiency and accuracy of traditional biological research and bring new entry points to complex life science problems that cannot yet be broken through.

GeneCompass integrates four kinds of biological prior knowledge.

Scale effect prompts model training to capture the conservative laws of biological evolution

The team found that models pre-trained on large-scale cross-species data can perform better on single-species subtasks It is consistent with the scaling law: larger-scale multi-species pre-training data can produce better pre-training representations and further improve the performance of downstream tasks. This finding shows that there are conserved gene regulation patterns between species, and that these patterns can be learned and understood by pre-trained models. At the same time, this also means that with the expansion of species and data, model performance is expected to continue to improve

Increasing the scale of cross-species data can improve model performance

Multi-task performance advantages show the powerful generalization ability of the basic large model

As the largest cross-species pre-trained basic life model with knowledge embedding to date, GeneCompass can realize multiple cross-species Transfer learning for downstream tasks, and achieving better performance than existing methods in cell type annotation, quantitative gene perturbation prediction, drug sensitivity analysis, etc. This fully demonstrates the strategic advantages of pre-training based on multi-species unlabeled big data and then using different sub-task data for model fine-tuning. It is expected to become a universal solution for analyzing and predicting various biological problems related to gene-cell characteristics.

Cell polarization: Transfer learning decodes gene regulatory networks and predicts cell fate changes

Using transfer learning to generate cells Specific gene regulatory network

The team also developed a set of gene regulatory network construction AI models based on generalized transfer learning, called CellPolaris. The model first sorts out hundreds of sets of transcriptome and chromatin accessibility data in matching cell scenarios to build a high-quality gene regulatory network, and then uses the generalized transfer learning model to generate more genes in cell scenarios using only transcriptome data. regulatory network. Then, using the generated high-confidence gene regulatory network, we developed a tool for identifying core transcription factors in cell fate transitions and a transcription factor perturbation simulation tool based on a probabilistic graphical model. This model can effectively identify the core factors of cell fate conversion and realize the simulation of transcription factor perturbation. It has important application value in the analysis of gene regulatory mechanisms and the discovery of disease-causing genes.

, Duration00:09

##Simulating the impact of transcription factor knockout on cell fate during placental development

The gene regulatory network generated by the CellPolaris model provides rich molecular interaction information and can be used as prior knowledge for large deep learning models . The low-dimensional embedding vectors generated by deep learning large models will provide important information for the analysis of gene regulatory mechanisms and the discovery of disease-causing genes.

The above two studies were completed by the "Compass Alliance" team. The "Compass Alliance" team is currently mainly composed of the Institute of Zoology of the Chinese Academy of Sciences, the Joint Computer Network Information Center, the Institute of Automation, the Institute of Computing Technology, and the Institute of Mathematics and Systems Science. Composed of research institutes and other research institutes, the goal of the alliance is to establish a new paradigm of life science research driven by digital intelligence and analyze the essential laws of life.

#artificial intelligence

× [Biological Neuroscience Mathematics Physics Chemistry Materials]

##

The above is the detailed content of The research team of the Chinese Academy of Sciences released two important papers: the release of the first large-scale model of the basis of life across species, and the release of a new AI model for cell fate prediction. For more information, please follow other related articles on the PHP Chinese website!