This is the source code of COMP 767 group project of Tianyu Shi & Jiawei Wang.
Our main implementation of this model is in dgn_xxx (xxx represents for different scenario, e.g. ring network, figure eight network, or minicity network)
In each folder, you can find e.g. xxx_main-DGN.py this is the file to run;
In DGN.py, we define the main network structure and training process;
In xxx_Env.py we define the simulation environment.
Also, we implemented transportation method (Intelligent driver model), RL methods (DDPG, multi-agent version of PPO) in three different scenarios.
In explore folder, we saved some models that we tested, include different model structures, number of agents, etc.
Our model structure is :
Some parameters of models in the experiment settings:
| Models config | units of encoder layer | activation function | clip ratio | discount factor | Optimizer | softupdate parameter | learning rate(actor+critic) | max returns |
|---|---|---|---|---|---|---|---|---|
| DGN a | (128 , 128) | ReLU | 0.3 | 0.9 | adam | 0.01 | （1e-4 , 1e-4) | 2982.97 |
| DGN b | (512 , 128) |
ReLU | 0.3 | 0.9 | adam | 0.01 | （1e-4 , 1e-4) | 2956.09 |
| DGN c | (128 , 64,128) |
ReLU | 0.3 | 0.9 | adam | 0.01 | （1e-4 , 1e-4)（1e-4 , 1e-4) | 2900.04 |
| DGN d | (128 , 128) | ReLU | 0.15 |
0.9 | adam | 0.01 | （1e-4 , 1e-4) | 2972.85 |
| DGN e | (128 , 128) | eLU |
0.3 | 0.9 | adam | 0.01 | （1e-4 , 1e-4) | 2900.04 |
| DGN f | (128 , 128) | ReLU | 0.3 | 0.9 | adam | 0.01 | （2.5e-4 , 1e-4) |
2898.77 |
| DDPG | (128 , 128) | eLU | 0.3 | 0.9 | adam | 0.01 | （2.5e-4 , 1e-4) | 2660.89 |
| MAPPO | (128 , 128) | ReLU | 0.3 | 0.9 | adam | 0.01 | 1e-4 , 1e-4) | 2975.76 |
You can run the python files with these above configurations with 100 runs, the triaining perfomance averaged over 10 different seeds is shown in the following figure:
We use the environment exactly the same as the conda environment in Flow, you can install Flow by following this Instruction:
Tianyu Shi(tianyu.shi3@mail.mcgill.ca)