[shardformer]support pipeline parallelism for mistral.#5642

Merged

ver217 merged 13 commits intohpcaitech:feature/update-transformersfrom flybird11111:feature/update-transformers

Apr 24, 2024

Commits on Apr 24, 2024

[shardformer] fix attn replacement (hpcaitech#5636 )

ver217
authored and
flybird11111
committed
[shardformer] update flashattention replacement (hpcaitech#5637 )

flybird11111
and
pre-commit-ci[bot]
committed
[Feature] Support LLaMA-3 CPT and ST (hpcaitech#5619 )

authored and
flybird11111
committed
[exampe] update llama example (hpcaitech#5626 )

ver217
authored and
flybird11111
committed
[example] llama3 (hpcaitech#5631 )

binmakeswell
authored and
flybird11111
committed
[test] fix llama test (hpcaitech#5638 )

ver217
authored and
flybird11111
committed
[gemini] fix buffer cast (hpcaitech#5639 )

ver217
authored and
flybird11111
committed
support pp for mistral
flybird11111
committed
Merge branch 'feature/update-transformers' into feature/update-transformers
flybird11111
authored
fix
flybird11111
committed
Merge branch 'feature/update-transformers' of github.com:flybird11111/ColossalAI into feature/update-transformers
flybird11111
committed
fix
flybird11111
committed
fix
flybird11111
committed