MocFormer: A Two-Stage Pre-training-Driven Transformer for Drug-Target Interactions Prediction

Drug-target interactions (DTIs) is essential for advancing pharmaceuticals. Traditional drug-target interaction studies rely on labor-intensive laboratory techniques. Still, recent advancements in computing power have elevated the importance of deep learning methods, offering faster, more precise, and cost-effective screening and prediction. Nonetheless, general deep learning methods often yield low-confidence results due to the complex nature of drugs and proteins, bias, limited labeled data, and feature extraction challenges. To address these challenges, a novel two-stage pre-trained framework is proposed for DTIs prediction. In the first stage, pre-trained molecule and protein models develop a comprehensive feature representation, enhancing the framework's ability to handle drug and protein diversity. This also reduces bias, improving prediction accuracy. In the second stage, a transformer with bilinear pooling and a fully connected layer (FCN) enables predictions based on feature vectors. Comprehensive experiments were conducted using DrugBank dataset and Epigenetic-regulators dataset to evaluate the framework's effectiveness. The results demonstrate that the proposed framework outperforms the state-of-the-art methods regarding accuracy, area under the ROC curve (AUC), recall, and the area under the precision-recall curve (AUPRC). The code will be available after being accepted: https://github.com/rickwang28574/MocFormer