Memory error when sampling from collector

[Tortes]

• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, torch, sys
  print(tianshou.__version__, torch.__version__, sys.version, sys.platform)
  0.2.5 1.6.0 3.7.7
  [GCC 7.3.0] linux

The collector raised the memory error when I training under the self-defined environment with PPO algorithm. The full error message is as below:
MemoryError: Unable to allocate 8.05 GiB for an array with shape (20000, 54006) and data type float64
The self-defined environment uses List state observation and List discrete action. Is there any necessary to change the observation type to dict to fit tianshou and use a smaller buffer size?
I tried to change the sample size of on-policy but it will raise the dimension matching problem(of course). Any answer is welcome for the stupid problem.

[Trinkle23897]

Could you please provide a sample of your self-defined env data format? (obs, act, ...)

[Tortes]

I will show part of the reset() function:
The getState() function will collect and return the sensor data. Then the data will be packed and normalized as the observation (54006 dimension).

def reset(self):
        ...
        self.goal_distance = self.getNavGoalDistance()
        realsense_data, rel_dis, roll, pitch, yaw, rel_theta, diff_angle, done, arrive = self.getState()
        realsense_data = [i/12 for i in realsense_data]

        # Normalize the state
        state = realsense_data + [rel_dis / diagonal_dis, (roll+180)/360, (pitch+180)/360, yaw / 360, rel_theta / 360, diff_angle / 180]

        return np.asarray(state)

For the action, I will take step() function as example:
The action space is a discrete List with 2 dimension: (linear speed, angular speed), which is uniformly sampled.

    def step(self, action):
        linear_vel = self.action_space_discrete[action][0]
        ang_vel = self.action_space_discrete[action][1]
        # print(linear_vel, ang_vel)

        vel_cmd = Twist()
        vel_cmd.linear.x = linear_vel / 4
        vel_cmd.angular.z = ang_vel
        self.pub_cmd_vel.publish(vel_cmd)
        
        # Update sensor data
        # self.getSensor()

        # Update state observation
        realsense_data, rel_dis, roll, pitch, yaw, rel_theta, diff_angle, done, arrive = self.getState()

        # Normalize the state
        '''
        Realsense:  [0, 12] => [0,1]
        LiDAR:      [0, 30] => [0,1]
        roll, pitch:[-180, 180] => [0,1]
        '''
        # scan_data = [i/30 for i in scan_data]

        state = realsense_data + [rel_dis / diagonal_dis, (roll+180)/360, (pitch+180)/360, yaw / 360, rel_theta / 360, diff_angle / 180]
        reward = self.setReward(done, arrive)

        return np.asarray(state), reward, done, {}

Besides, I defined the action space with gym.spaces.Discrete(len(action_space)) and the observation space with gym.spaces.Box(low=0, high=1, shape=(54006,), dtype=np.float32)

[Trinkle23897]

Okay, so how many RAM does your machine have?
Did you use ignore_obs_next=True in the replay buffer? (This can save half of the memory)

[Tortes]

The machine have 16GiB RAM and 16GiB Swap area.
I set the replay buffer as the default.(Only set the buffer size as 20000. The ignore_obs_next is set to be False as default.)

[Trinkle23897]

So try opening ignore_obs_next first

[Tortes]

Really thanks, I will close the issue if there's no further problems.

[Trinkle23897]

You may get another issue: when it is testing the policy, the program will be killed due to the lack of memory. This is because the collector still creates some cache_buffer even if the main_buffer is None. It will be fixed soon.

I post a hotfix here:

[Tortes]

Solved after updating the RAM to 24Gib