Binding Peptide Generation for MHC Class I Proteins with Deep Reinforcement Learning

This is the implementation of our PepPPO model for the paper below,

Ziqi Chen, Baoyi Zhang, Hongyu Guo, Prashant Emani, Trevor Clancy, Chongming Jiang, Mark Gerstein, Xia Ning, Chao Cheng, and Martin Renqiang Min. Binding Peptide Generation for MHC Class I Proteins with Deep Reinforcement Learning. Accepted to Bioinformatics. December 2022.

Our DRL framework is based on stable-baseline3.

Requirements

• python==3.6.13

• torch==1.9.1

• tensorflow==2.2.0

• gym==0.18.0

• Mhcflurry==2.0.1

Overview

code source code

baselines code for baselines including MCTS, VAE with backpropagation and VAE with bayesian optimization.

data data used for training

stable_baselines3 code from the package "stable_baseline3". Our implementation of RL model is based on this package.

Training

To train a PepPPO model, run

python ./code/peptide_env.py --path <model path> --gamma 0.9 --num_envs 20 --pretrain_iter 3000 --sample_rate 0.5

path: path used to save the model

gamma: discount factor

sample_rate : the percentage of initial peptides that are sampled from the dataset. For example, "sample_rate = 0.0" represents that all the initial peptides are randomly generated.

num_envs : number of environments running in parallel; determined by the cores of CPU.

Please check the details of all the parameters using the command python ./code/peptide_env.py -h

Testing

To test a trained PepPPO model with specific alleles, run

python ./code/test_RL.py --sample_rate 0.5 --alleles ./data/test_alleles.txt --out <test result file> --num_envs 5 --rollout 1000 --path ./model/ppo_peptide.zip

alleles : the path of the file with all the alleles to be tested.

out : the path of the output file

rollout : the number of optimized peptides for each allele.

path: model path

To optimize specific peptides for given alleles with the trained PepPPO model , run

python ./code/test_RL_with_peptide.py --sample_rate 0.5 --peptides <test peptide path> --alleles ./data/test_alleles.txt --out <test result file> --rollout 1000 --path ./model/ppo_peptide.zip

peptides : the path of the file with all the alleles to be tested.