Skip to content

Commit

Permalink
[BE] remove old pytorch version warning on strided sharding since 2.5…
Browse files Browse the repository at this point in the history
… is official released

[ghstack-poisoned]
  • Loading branch information
XilunWu committed Oct 30, 2024
1 parent 53d0f69 commit ab922b6
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 36 deletions.
6 changes: 0 additions & 6 deletions torchtitan/parallelisms/parallelize_llama.py
Original file line number Diff line number Diff line change
Expand Up @@ -330,12 +330,6 @@ def apply_fsdp(
if cpu_offload:
fsdp_config["offload_policy"] = CPUOffloadPolicy()

# TODO: remove this check once PyTorch 2.5 is released. We can safely assume
# that users won't use a nightly build which is older than 20240809 by then.
if tp_enabled:
# check if strided sharding is enabled, which is necessary for 2D/3D DCP
check_strided_sharding_enabled()

for layer_id, transformer_block in model.layers.items():
if pp_enabled:
# For PP, do not reshard after forward to avoid per-microbatch
Expand Down
30 changes: 0 additions & 30 deletions torchtitan/parallelisms/utils.py

This file was deleted.

0 comments on commit ab922b6

Please sign in to comment.