AI Regulatory model

Borzoi is a deep learning model from Calico that predicts cell-type-specific and tissue-specific RNA-seq coverage from DNA sequence. It scores variant effects across transcription, splicing, and polyadenylation, and extracts cis-regulatory motifs driving RNA expression. Published in Nature Genetics.

RNA-seq, gene regulation, splicing, polyadenylation, variant effect, CNN, deep learning, eQTL

Sequence-based machine-learning models trained on genomics data improve genetic variant interpretation by providing functional predictions describing their impact on the cis-regulatory code. However, current tools do not predict RNA-seq expression profiles because of modeling challenges. Here, we introduce Borzoi, a model that learns to predict cell-type-specific and tissue-specific RNA-seq coverage from DNA sequence. Using statistics derived from Borzoi's predicted coverage, we isolate and accurately score DNA variant effects across multiple layers of regulation, including transcription, splicing and polyadenylation. Evaluated on quantitative trait loci, Borzoi is competitive with and often outperforms state-of-the-art models trained on individual regulatory functions. By applying attribution methods to the derived statistics, we extract cis-regulatory motifs driving RNA expression and post-transcriptional regulation in normal tissues. The wide availability of RNA-seq data across species, conditions and assays profiling specific aspects of regulation emphasizes the potential of this approach to decipher the mapping from DNA sequence to regulatory function.

DeepSEA is a foundational deep learning model that predicts the chromatin effects of noncoding variants directly from DNA sequence, including DNase I sensitivity, histone mark profiles, and transcription factor binding. Published in Nature Methods, it was one of the first deep learning approaches for noncoding variant interpretation.

noncoding variant, chromatin, epigenetics, deep learning, DNase I, histone marks, TF binding, regulatory effect

Noncoding variants are of tremendous biological importance, and their functional interpretation is a critical challenge in genomics. Here we introduce DeepSEA, a deep learning-based sequence model that directly predicts the chromatin effects of sequence alterations at single-nucleotide sensitivity. DeepSEA captures regulatory sequence context and learns a wide range of regulatory features including DNase I sensitivity, histone mark profiles, and transcription factor binding. The model achieves state-of-the-art accuracy for predicting the functional consequences of noncoding variants and can be applied to prioritize disease-associated variants from large-scale sequencing studies.

Enformer is a DeepMind model that integrates long-range DNA interactions (up to 200 kb) using a CNN + Transformer architecture to predict gene expression, chromatin profiles, and TF binding from DNA sequence. Achieves SOTA on variant effect prediction and regulatory element annotation.

gene expression prediction, long-range interactions, CNN, Transformer, DeepMind, chromatin, variant effect, regulatory genomics

Quantitative gene expression measurements across cell types and tissues can provide a complete picture of the dynamic functions of the genome. However, gene expression is challenging to predict from sequence alone because of the enormous distances over which regulatory elements act. Enformer combines CNN and Transformer architectures to integrate information from up to 200 kb of DNA sequence, enabling accurate prediction of gene expression, chromatin state, and transcription factor binding. Enformer achieves state-of-the-art predictions across diverse genomic assays and accurately predicts the effects of genetic variants on gene expression.

ExPecto (Basenji2) is a deep learning framework that predicts the causal effects of both coding and noncoding genetic variants on gene expression levels and disease risk directly from DNA sequence, without requiring any prior knowledge of regulatory elements or annotations.

variant effect, gene expression, disease risk, ab initio, deep learning, noncoding, tissue-specific

The complexity and scale of human genetics studies present a significant challenge for interpreting the functional consequences of genetic variants. Here we introduce ExPecto, a deep learning framework that can predict the causal effects of genetic variants on gene expression levels and disease risk directly from DNA sequence. ExPecto uses a modular architecture with a core deep convolutional neural network to learn regulatory features from sequence, followed by spatial transformation and tissue-specific prediction layers. We demonstrate that ExPecto can accurately predict variant effects on expression across a wide range of tissues and can identify pathogenic variants from population-scale sequencing data. The framework enables ab initio prediction of expression and disease risk without relying on prior annotations of regulatory elements.

Flashzoi is an enhanced Borzoi model that replaces relative positional encodings with rotary positional encodings (RoPE) and uses FlashAttention-2, achieving over 3x faster training/inference and up to 2.4x reduced memory usage while maintaining or improving accuracy on RNA-seq coverage prediction, variant effects, and enhancer-promoter linking.

Borzoi, FlashAttention, RoPE, accelerated inference, regulatory genomics, variant effect, enhancer-promoter

Accurately predicting how DNA sequence drives gene regulation and how genetic variants alter gene expression is a central challenge in genomics. Borzoi, which models over ten thousand genomic assays including RNA-seq coverage from over half a megabase of sequence context alone promises to become an important foundation model in regulatory genomics, both for massively annotating variants and for further model development. However, the currently used relative positional encodings limit Borzoi's computational efficiency. We present Flashzoi, an enhanced Borzoi model that leverages rotary positional encodings and FlashAttention-2. This achieves over 3-fold faster training and inference and up to 2.4-fold reduced memory usage, while maintaining or improving accuracy in modeling various genomic assays including RNA-seq coverage, predicting variant effects, and enhancer-promoter linking. Flashzoi's improved efficiency facilitates large-scale genomic analyses and opens avenues for exploring more complex regulatory mechanisms and modeling.

Sei is a deep learning model that generates a comprehensive map of regulatory activity from DNA sequence, predicting 21,907 chromatin features across cell types and contexts. Enables interpretation of noncoding variants in terms of specific regulatory functions and tissues, providing a global atlas of human cis-regulation.

regulatory activity map, chromatin, transcriptional regulation, deep learning, sequence-to-activity, human genetics, 21,907 features

Deciphering the impact of noncoding variants on gene regulation is a major challenge in human genetics. While deep learning models can accurately predict regulatory features from DNA sequence, interpreting these predictions to understand variant effects across diverse contexts remains difficult. Here we present Sei, a deep learning model that produces a comprehensive, sequence-based global map of regulatory activity. Sei predicts 21,907 chromatin profiles encompassing a wide range of cell types and regulatory features, and organizes them into 40 tissue-agnostic regulatory activities using a hierarchical model. We demonstrate that Sei accurately predicts regulatory effects and can identify disease-relevant variants across diverse conditions, enabling a more complete understanding of human genetic variation.