Epistatic Net allows the sparse spectral regularization of deep neural networks for inferring fitness functions
Epistatic Net обеспечивает разреженную спектральную регуляризацию глубоких нейронных сетей для вывода функций приспособленности
2021-09-01
SCID: 54.1/xkvjcvn6
Discuss with AI
Epistatic Netdeep neural networksfitness function inferencesparse epistatic interactionsspectral regularization
Figures from the paper
Abstract (AI)
Abstract Despite recent advances in high-throughput combinatorial mutagenesis assays, the number of labeled sequences available to predict molecular functions has remained small for the vastness of the sequence space combined with the ruggedness of many fitness functions. Expressive models in machine learning (ML), such as deep neural networks (DNNs), can model the nonlinearities in rugged fitness functions, which manifest as high-order epistatic interactions among the mutational sites. However, in the absence of an inductive bias, DNNs overfit to the small number of labeled sequences available for training. Herein, we exploit the recent biological evidence that epistatic interactions in many fitness functions are sparse; this knowledge can be used as an inductive bias to regularize DNNs. We have developed a method for sparse epistatic regularization of DNNs, called the epistatic net (EN), which constrains the number of non-zero coefficients in the spectral representation of DNNs. For larger sequences, where finding the spectral transform becomes computationally intractable, we have developed a scalable extension of EN, which subsamples the combinatorial sequence space uniformly inducing a sparse-graph-code structure, and regularizes DNNs using the resulting greedy optimization method. Results on several biological landscapes, from bacterial to protein fitness functions, show that EN consistently improves the prediction accuracy of DNNs and enables them to outperform competing models which assume other forms of inductive biases. EN estimates all the higher-order epistatic interactions of DNNs trained on massive sequence spaces—a computational problem that takes years to solve without leveraging the epistatic sparsity in the fitness functions.
Key Findings
1
A scalable EN extension applies fast sparse-recovery algorithms informed by coding theory to regularize models for larger sequences.
2
Across several biological fitness landscapes, EN consistently improves DNN prediction accuracy and outperforms competing models based on alternative priors.
3
EN enables estimation of higher-order epistatic interactions in DNNs trained over massive sequence spaces, reducing a computation that otherwise takes years.
4
Epistatic Net (EN) introduces spectral regularization for deep neural networks by exploiting sparsity in higher-order epistatic interactions.
Research Object
deep neural network models of biological fitness functions over combinatorial sequence spaces
Research Subject
sparse higher-order epistatic interactions and their effect on fitness-function prediction accuracy
Publication Details
Publication Date
2021-09-01
Journal
Publisher
ISSN
Cited by
43
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest