Bug in MultiAgentPolicyManager - and Fix

[FahmidMorshed]

• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, gymnasium as gym, torch, numpy, sys
  print(tianshou.__version__, gym.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)

I tried running MARL with multiple SAC as internal policies for each agent for one of my custom-made petting zoo environments. After rigorous debugging, I realized that the MultiAgentPolicyManager in version 0.5.1 needs a train(self, mode: bool = True) method that iteratively sets all the internal policies to mode. Otherwise, the internal policies are never called in eval mode. I am not sure if this is also true when used with other types of internal policies.

The following function solves the issue if added to tianshou/policy/multiagent/mapolicy.py :

def train(self, mode: bool = True):
      """Sets each internal policy in training mode."""
      for agent_id, policy in self.policies.items():
          policy.train(mode)
      return self

I can do a pull-request to the main branch, if needed.

[Trinkle23897]

Oops sorry, and yeah good catch! Feel free to submit a PR.

[FahmidMorshed]

Added a RP to fix the issue. Ty.