Code

CUDA Memory Profile Analyzer#9860

Code

Closed

tonyjie wants to merge 13 commits intoNVIDIA:mainNVIDIA/NeMo:mainfrom tonyjie:jiajiel/mem_snapshot_pr1tonyjie/NeMo:jiajiel/mem_snapshot_pr1Copy head branch name to clipboard

+622-34

Commits

Commits on Sep 8, 2024

• Add peak_memory_analysis: with analysis_enabled in the config, we will analysis the memory at the global peak of the trace, and generate CSV files

  Show description for 47e61cc
  tonyjie

  committed
  47e61cc

  Copy full SHA for 47e61cc

• Add weight memory capturing and analysis. Now we separate weight and activation.

  Show description for bc1ba35
  tonyjie

  committed
  bc1ba35

  Copy full SHA for bc1ba35

• add OOM support. Verify it with LLaMA70b model on H100

  Show description for b46c226
  tonyjie

  committed
  b46c226

  Copy full SHA for b46c226

• add newline between frame to make it more readable

  Show description for cf14018
  tonyjie

  committed
  cf14018

  Copy full SHA for cf14018

• Fix a bug that occurs on multi-GPU setting. Can't call get_rank() during the setup

  Show description for 684ef24
  tonyjie

  committed
  684ef24

  Copy full SHA for 684ef24

• comment logging for the real_batch_idx

  Show description for 5935314
  tonyjie

  committed
  5935314

  Copy full SHA for 5935314

• add copyright; remove unused commented line

  Show description for 52b371d
  tonyjie

  committed
  52b371d

  Copy full SHA for 52b371d

• code cleaning based on PR review

  Show description for ba7f13e
  tonyjie

  committed
  ba7f13e

  Copy full SHA for ba7f13e

• Apply isort and black reformatting

  Show description for 65a33ea
  tonyjie

  committed
  65a33ea

  Copy full SHA for 65a33ea

• code cleaning

  Show description for e365893
  tonyjie

  committed
  e365893

  Copy full SHA for e365893

• Apply isort and black reformatting

  Show description for 8d33368
  tonyjie

  committed
  8d33368

  Copy full SHA for 8d33368

• remove unused code

  Show description for 2cb2497
  tonyjie

  committed
  2cb2497

  Copy full SHA for 2cb2497

• guard the pytorch memory snapshot function by providing a minimum_torch_version; fix other minor issues based on review

  tonyjie

  committed
  165e43e

  Copy full SHA for 165e43e