New AI research offers a simple but effective structure –

Proteins, the energy of the cell, are involved in various applications, in particular material and therapeutic. They are made up of a chain of amino acids that fold into a certain shape. A significant number of new protein sequences have been discovered recently through the development of low-cost sequencing technology. Accurate and efficient methods of in silico protein function annotation are needed to bridge the current gap between sequence and function, because functional annotation of a new protein sequence is still expensive and time-consuming.

Many data-driven approaches rely on learning representations of protein structures because many protein functions are controlled by how they are folded. These representations can then be applied to tasks such as protein design, structure classification, model quality assessment, and function prediction.

The number of published protein structures is an order of magnitude lower than the number of datasets in other application areas of machine learning due to the difficulty of experimental identification of protein structure. For example, the Protein Database contains 182,000 experimentally confirmed structures, compared to 47,000 protein sequences in Pfam and 10,000 annotated images in ImageNet. Several studies have used the abundance of untagged protein sequence data to develop an appropriate representation of existing proteins to fill this representation gap. Many researchers have used self-supervised learning to pre-train protein coders on millions of sequences.

Recent developments in accurate protein structure prediction techniques based on deep learning have made it possible to efficiently and confidently predict the structures of many protein sequences. However, these techniques do not specifically capture or use protein structural information that is known to determine protein function. Many structure-based protein coders have been proposed to make better use of structural information. Unfortunately, edge interactions, which are crucial for simulating protein structure, have not yet been explicitly addressed in these models. Additionally, due to the lack of experimentally established protein structures, relatively little work has been done until recently to create pretraining techniques that take advantage of unlabeled 3D structures.

Inspired by this breakthrough, they are creating a protein encoder that can be applied to a range of property prediction applications and is pretrained on the most feasible protein structures. They suggest a simple yet effective structure-based encoder called the GeomEtry-Aware Relational Graph Neural Network, which conveys a relational message about protein residue graphs after encoding spatial information by including various structural or sequence edges. They suggest a sparse edge message passing technique to improve the protein structure encoder, which is the first effort to implement edge-level message passing on GNNs for protein structure encoding. Their idea was inspired by the design of the attention triangle by Evoformer.

They also provide a geometric pretraining approach based on the well-known contrastive learning framework to learn the protein structure encoder. They suggest innovative augmentation functions that enhance the similarity between acquired representations of substructures of the same protein while decreasing that between those of different proteins to find physiologically related protein substructures that coexist in proteins. They simultaneously suggest a set of simple baselines based on self-prediction.

They established a solid foundation for pretraining protein structure representations by comparing their pretraining methods to several downstream property prediction tasks. These pre-learning problems include masked prediction of various geometric or physico-chemical properties, such as residue types, Euclidean distances, and dihedral angles. Numerous tests using a variety of benchmarks, such as Enzyme Commission Number Prediction, Gene Ontology Term Prediction, Fold Classification, and Reaction Classification, show that GearNet enhanced with edge message passing can consistently outperform existing protein encoders on the majority of tasks in a supervised environment.

Moreover, using the suggested pre-training strategy, their model trained on less than a million samples achieves equivalent or even better results than the most advanced sequence-based encoders pre-trained on million or billion datasets. The codebase is publicly available on Github. It is written in PyTorch and Torch Drug.

CheckPaper AndGithub link. All credit for this research goes to the researchers of this project. Also, don’t forget to joinour 26k+ ML subreddit,Discord Channel,AndE-mailwhere we share the latest AI research news, cool AI projects, and more.