[HybridParallel]Add segment methods for pipelineparallel

[ForFishes]

PR types

New features

PR changes

Others

Description

[HybridParallel]Add segment methods for pipelineparallel

python code:

class GPT(PipelineLayer):
        self.descs = [.....]
        super().__init__(
            layers=self.descs,
            seg_method=[0, 3, 6],
            })

When the seg_method is a list, it uses left-closed-right-open, indicating that the segmentation is performed according to the method specified by the user. If the total number of layerdesc is 10, and the number of pp segmentation stages is 3, then there is

1. seg_method=[0, 3, 6]
   Indicates that the segmentation is grouped into
   stage0: 0, 1, 2
   stage1: 3, 4, 5
   stage2: 6, 7, 8, 9

2. seg_method=[0, 3, 4, 10]
   Indicates that the segmentation is grouped into
   stage0: 0, 1, 2
   stage1: 3
   stage2: 4, 5, 6, 7, 8, 9

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-bot]

❌ The PR is not created using PR's template. You can refer to this Demo.
Please use PR's template, it helps save our maintainers' time so that more developers get helped.

[gongweibao]

LGTM