merge upstream updates and adapt to the MPT fix in torch_spmd#5

Merged

tianyu-l merged 17 commits intomainfrom mpt

Jul 11, 2024

+467-188

Commits on Jun 26, 2024

by default disable heavy memory profiling
tianyu-l
committed

Commits on Jun 27, 2024

Add the option to turn on async-TP
yifuwang
committed

Commits on Jul 1, 2024

Modifying memory estimation options and minor changes
sanketpurandare
committed

Commits on Jul 8, 2024

add comment pointing to Sequence Parallel optimization example
tianyu-l
committed
switch float8 logic from Float8DynamicLinear to Float8Linear (pytorch#436 )
vkuzo
authored

Commits on Jul 11, 2024