Machine Learning and Artificial Intelligence for the Prediction of Host–Pathogen Interactions: A Viral Case

Introduction

The causative agents of infectious diseases come in a great variety of shapes, biochemistry, and genetic makeup. They originate from a variety of population reservoirs and only cross their barriers on occasion,^1,2 due to the subtle changes in the dynamic ecological equilibrium. In the face of climate changes of as yet unseen proportions, the change in ecological balances will inevitably cause emergence of new infectious diseases.^3,4 This makes further pandemics highly plausible. To tackle the outbreaks of the future we must improve our understanding of infectious diseases caused by the interactions between a pathogen and a host. In this review we will explore how novel techniques for computational analysis of interactions between a pathogen and its host may foster such understanding. For the purpose of this review, we will focus primarily on viruses, however some aspects of interactions between viruses and their hosts may be generalized to other pathogens.

Viruses require host cells to procreate and spread their progeny. For this, they enter cells, replicate and egress in a stepwise process occurring through interactions between molecules of the host cell and the pathogen molecules. Such interactions are commonly referred to as host–pathogen interactions (HPI).⁵ HPIs include mediating various host mechanisms’ exploitation by the pathogen. Typically, this occurs through direct interactions between the molecules of the pathogen and the molecules of the host cell (Figure 1A). For example, SARS-CoV2 virus enters human host cells through the binding of its S protein to Angiotensin-converting Enzyme 2 (ACE-2) of the cells. Next, in brief, particles get endocytosed, fuse with endosomes and uncoat their genome, perform primary protein translation, and viral RNA synthesis. Consecutively, virus forms replication factories, producing progeny building blocks, and assembles progeny, which subsequently egress the infected host-cell.⁶ Needless to say, on a mechanistic level this process involves dozens of HPIs which give the virus its advantages and may be exploited as antiviral strategies. Additionally, these mechanisms vary greatly for other viruses⁷ (Figure 1B) and even other human coronaviruses (HCoV).

Figure 1 Schematic overview of host–pathogen interactions. (A) simplistic depiction of a pathogen (green hexagon in blue circle) surface protein (red circle) binding to a receptor (black Y-shape) on a host cell surface. (B) A generalized and simplified overview of a pathogen life-cycle stages involving interactions with the host cell.

For example, S proteins of the predominant seasonal HCoV-OC43 (Betacoronavirus 1) and HCoV-229E (Alphacoronavirus)^8,9 bind to 9-O-acetylsialic acids and Amino-peptidase N as primary receptors in human cells rather than ACE-2. Similarly, the primary receptor of the HCoV-HKU1 responsible for the first severe acute respiratory syndrome (SARS) outbreak is 9-O-acetylsialic acids rather than ACE-2.¹⁰ In other words, human pathogenic viruses constantly change and adapt. Therefore, understanding biological mechanisms of virus entry, replication, and egress may allow to develop general strategies for fighting infectious diseases. However, identifying HPIs on a mechanistic level is often a laborious manual experimental task. Furthermore, due to the ability of the virus to change and adapt, deciphering HPI mechanisms in a timely fashion is akin to chasing a moving target. In the currently ongoing SARS-CoV2 outbreak, identifying important mechanistic changes in the emerging virus variants fast is the question of life and death. Perhaps the most promising weapon to tackle laborious manual tasks that humanity has in its arsenal to date are machine learning (ML) and artificial intelligence (AI). While experimental validation remains a must, these computational techniques may significantly narrow down the number of experiments required for HPI identification through predicting putative interaction partners. Such HPI prediction tasks include predicting host–pathogen protein–protein interactions^11,12 (e.g., using protein sequences and infection phenotypes¹³), prediction of a putative host or a receptor for the specific pathogen.¹⁴

Historically, the field of AI originates from an attempt to create fundamental and applied basics for machines with “intelligent properties”.¹⁵ ML, often considered an AI subfield, has significantly facilitated AI by providing a tangible toolset. Conversely, the traction some ML algorithms like artificial neural networks have gained in recent years may be to a larger extent attributed to the effort to make progress in AI. Taken together, these methodologies refer to a group of computational techniques that enable computers to perform specific tasks without the use of explicit rules or instructions.¹⁶ This is usually accomplished by creating ML models from real world or computer simulated data. Typically, an ML model is trained on a data set of engineered features (e.g., extraction of cell size, signal intensity, etc.) in a user-supervised or unsupervised manner. Conventionally, a supervised data set consists of the features (X) and targets (Y), where Y corresponds to an objectively known property (i.e., the ground truth). An approach like this is well suited to automating biological data processing, where formulating a detailed and finite set of rules denoting related events is difficult.^17–19

Beyond classical ML, a subset of approaches named Deep Learning (DL) uses algorithms like deep artificial neural networks (DNN) to provide a methodology for pattern recognition with unprecedented accuracy.²⁰ This is achieved through a combination of approaches including representation learning, allowing automated feature generation,²¹ as well as, stacking multiple hidden layers (modules) of artificial neural networks. Combining various kinds of connections and linear algebra operations between individual neurons in a layer allows to construct a broad variety of DNN layers including, for example, convolutional layers,²² recurrent layers,²³ attention layers,²⁴ Sigmoid or SoftMax classification layers. Modern DNN architectures reach expressive capacity of billions or even trillions²⁵ trainable parameters.²⁶ Noteworthy, as a rule of thumb, the larger the expressive capacity of a DNN the more data points are required to train while avoiding overfitting. This bears design choice constraints in the domain specific fields like HPI analysis. To facilitate training of such advanced architectures, Practicality of DL relies on the modern parallel computing, which allows it to represent the features of the input data (e.g., micrographs or CT scans) through non-linear transformations in the so-called vector latent space. Recently, adoption of these methodologies has dramatically increased in the field of Infections Biology. Applications of AI, ML and DL span molecular and genetic data (Figure 2A), microscopy (Figure 2B) and language data (Figure 2C). Here we will review specific examples of these applications and discuss the future outlook for ML and AI for the prediction of HPIs.

Figure 2 Overview of machine learning and artificial intelligence application for host–pathogens interactions research. (A) Schematic representation of machine learning applications for genetic and molecular data. (B) Schematic representation of machine learning applications for image data. (C) Schematic representation of machine learning applications for language data. Gray parathesis separate respective downstream tasks.

Host–Pathogen Interactions Analysis from Genetic and Molecular Data

Perhaps one of the most direct applications of AI and ML for HPIs is to reveal patterns on the level of host and pathogen molecules and genes. The genetic or molecular information is typically represented on a single-character sequence level (Figure 2A). Small molecule information is often represented using a simplified molecular-input line-entry system (smiles).²⁷ In such a setting ML can assist in RNA and DNA accessibility analysis, transcription analysis (reviewed in²⁸), protein–protein interactions,^11–13,29 as well as, sequence-based host organism or receptor prediction.¹⁴ Noteworthy, in well-defined tasks simple ML algorithms like Random Forest (RF)³⁰ classification, Multilayer Perceptron (MLP) or kernel-based SVM³¹ perform remarkably well. For example, Karabulut et al show that on the task of adenovirus infection genus prediction kernel-based SVM reaches performance of 0.96 F1 score and 0.89 area under the receiver operating characteristics curve (AUC) with RF and MLP algorithms trailing remarkably close.²⁹ In such cases, more advanced algorithms like DL are not very likely to deliver a significant further improvement. However, in the settings outside of the very specific data set these algorithms may deliver a boost in generalization.

Identification of genetic variations in either host or pathogen genomes conferring higher pathogenicity may also be improved using DL.²⁸ Other examples of successful ML application for HPIs include base calling and SNP analysis³² and clinical metagenomics.³³ Algorithms of choice most commonly include recurrent neural networks (RNN), for example architecture known as long short-term memory (LSTM) RNN.²³ In some cases, more specialized CNNs are employed³⁴ or even a consecutive combination of CNN and LSTM.³⁵ Traditionally, RNN (and sometimes CNN) architectures have dominated the algorithmic landscape for sequence-based analysis, demonstrating state-of-the-art performance. However, recently introduced transformer architecture,²⁶ which is currently dominating natural language ML, is slowly entering the field.³⁶ Yet data-hungriness remains the biggest hurdle for the transformers to overcome in the HPI domain.

Beyond simple pattern recognition AI and ML algorithms trained on a large number of examples may be used to generate putative new molecules. This approach is showing promising results in the novel antiviral space. Beck and colleagues, for example, described prospective antivirals for the pandemic SARS-CoV2 using a DL model of drug-target interaction from a large number of commercially available antivirals. For this, they developed the Molecule Transformer - Drug Target Interaction architecture. Using this technique, authors identified human immunodeficiency virus drug as a potential candidate.³⁷

Host–Pathogen Interactions Analysis from Image Data

HPIs may be observed visually or using digital microscopy. On a subcellular level microscopic imaging may be employed to capture image-based data of individual virus particle interactions with host-cell proteins.^38,39 These data are typically obtained using fluorescence light microscopy at high magnification (e.g., 64x-100x), supperresolution microscopy^38,40,41 or electron microscopy.^42–44 Various ML techniques may be employed to analyze HPIs subcellular data, ranging from support vector machines⁴⁵ to DL.^46–50

Infection manifestation on a single-cell level is typically hallmarked by the onset of cytopathic effect (CPE).^51,52 Synchronized with virus entry, uncoating, and replication through a virus genetic program, virus-induced CPE involves dramatic changes of cell morphology,^51,52 which can be observed in cell culture using conventional light microscopy. These include, among other, cell rounding and swelling, focal patterns emergence, cytoplasmic vacuolization⁵³ pyknosis (cytoplasmic shrinkage),⁵¹ syncytia formation,⁵⁴ and may be linked with pathogen-related cell death,⁵¹ apoptosis⁵⁵ or motility.⁵⁶ The HPIs occurring on molecular level drive CPE. CPE can be observed in cell culture using conventional light microscopy techniques without specific labeling at a moderate magnification (e.g., 5×-20×). Its manifestation often differs substantially for various pathogens, cell types, and multiplicity of infection (MOI).⁵⁷

Downstream tasks (Figure 2B) for which ML is employed on such data typically include pathogen image segmentation⁴⁹ (often using a variety of the U-Net architecture⁵⁸), HPI events or virus classification from full or cropped field-of-view,^47,48,50 pathogen object detection⁵⁹ or infection detection.⁶⁰ Other examples include understanding structure and function relationships with the pathogens⁴⁰ or time-lapse analysis.^45,60

Being strongly related to the field of computer vision, HPI image analysis remains, thus far, strongly dominated by the CNN DL algorithms. Indeed, unsurprisingly for pathogen image classification tasks both shallower and deeper CNNs outperform conventional RF and MLP ML algorithms in metrics like F1 often by more than 30–40%.^47,60

Host–Pathogen Interactions Analysis from the Language Data in the Scientific Publications

ML for natural language processing (NLP) has recently seen an incredible boost in performance through the introduction of the transformer-based models.²⁶ Specifically, in their work Devlin et al reported that BERT transformer mode significantly outperformed bidirectional LSTM (state-of-the-art at that time) on the General Language Understanding Evaluation (GLUE)⁶¹ benchmark with 71 and 82 average GLUE performance, Transformer models leverage large text corpora, akin to BookCorpus⁶² or the English Wikipedia data set, and high expressive capacity to define the new state-of-the-art performance on a plethora of NLP tasks. These tasks include text classification, named entity recognition (NER), semantic text similarity (STS), text summary, question answering (QA), reading comprehension, knowledge discovery (KD) and mapping and other (reviewed in⁶³). Further boost in performance in the novel transformer architectures is achieved through the multi-headed attention mechanism.²⁴ Building upon it through transfer learning (i.e., repurposing a pretrained model), the general-purpose deep bidirectional transformers model was fine-tuned by Lee and colleagues to the domain of biomedical research texts.⁶⁴ This work, in turn, sparked a surge in biomedical NLP research.

With respect to HPI research, adoption of this novel methodology was limited by the availability of data sets large-enough to warrant a successful domain adaptation until a few years ago. However, amid the global SARS-CoV2 pandemic Wang and colleagues constructed the so-called COVID-19 open research data set (CORD-19).⁶⁵ This data set, in turn, inspired a plethora of HPI analysis approaches using the language data in scientific publications. A great number of NLP downstream tasks have been attempted on this and similar data sets in the past 12–18 month alone (Figure 2C).

Specifically, Koksal et al proposed a search engine approach to finding protein-compound pairs in COVID-19 literature formulated in the CORD-19 corpus.⁶⁶ Wang and co-authors have formulated a NER task data set that covers 75 detailed entity types.⁶⁷ These types include biomedical entities like genes, chemicals and diseases, as well as, entity types related explicitly to the SARS-CoV2 and COVID-19 research including coronaviruses, viral proteins, materials, evolution, immune responses and substrates. To capture subtle domain specific similarities in the CORD-19 data set Guo et al formulated an STS data set.⁶⁸ For the KD task on CORD-19 data set Tam and co-authors proposed a transformer-based target-query method they named Transformer Query-Target Knowledge Discovery (TEND).⁶⁹ Reddy and colleagues proposed a QA task domain adaptation for COVID-19 related questions⁷⁰ using a combination of CORD-19 and several COVID-19 QA data sets.^71–73 Their data suggest that pretrained models may successfully be fine-tuned for HPI domain, gaining 4–7% performance over the baseline. Noteworthy, many of these recent approaches remain to be peer-reviewed and the impact these techniques will have on the SARS-CoV2 and HPI research remains to be demonstrated.

Conclusion

Prior to the surge of AI-community interest in the biomedical domain, ML techniques have been only sporadically applied in analysis of interactions between pathogens and their hosts (reviewed in⁷⁴). However, in the wake of SARS-CoV2 pandemic ML and AI techniques application for HPI analysis sees an ever-growing interest. Transcending the status of a niche application of novel AI algorithms, pathogens and HPI research found itself in the spotlight of AI-researchers attention. This manifested in a variety of fundamentally new approaches to analyze molecular, image-based and more recently language-based HPI data mentioned in this review. Admittedly, the knowledge gap between AI and HPI research remains great. This is especially evident in the case of language-based data. Yet, given the sheer effort from both fields to bridge this gap, we may see fruition of these efforts in a not-so-distant future.

The lack of large HPI data sets remains one of the main hurdles for further penetration of AI and ML into the HPI field. However, recent years have seen a significant improvement in that matter. In case of image-based data, the advent of resources like Bio-Image Archive,⁷⁵ data-dedicated journals and social coding platforms fostered deposition of specialized image data sets, including those focused on HPI.^46,76 The influence of CORD-19⁶⁵ on the adoption of NLP techniques in HPI research was unequivocally self-evident. Perhaps, it is the access to open research data sets akin to CORD-19 together with image-based data repositories that will become the missing piece for AI and ML to become a viral case in HPI research.