Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Accelerated train using deepspeed and use an enlarged CommonVoice dataset #1

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Accelerated train using deepspeed and use an enlarged CommonVoice dataset #1

wants to merge 6 commits into from

Commits on Jul 14, 2023

  1. Add script for prepare raw CV dataset

    @bl4dylion4ik
    bl4dylion4ik committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    36a3eaa View commit details
    Browse the repository at this point in the history

  2. Add test function

    @bl4dylion4ik
    bl4dylion4ik committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    3974704 View commit details
    Browse the repository at this point in the history

  3. Add train script by using deepspeed

    @bl4dylion4ik
    bl4dylion4ik committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    5b07930 View commit details
    Browse the repository at this point in the history

  4. Init deepspeed config

    @bl4dylion4ik
    bl4dylion4ik committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    2848c5b View commit details
    Browse the repository at this point in the history

  5. Add option '--from_disk', that load dataset to train from disk

    @bl4dylion4ik
    bl4dylion4ik committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    2300b8e View commit details
    Browse the repository at this point in the history

  6. add deepspeed and accelerate library

    @bl4dylion4ik
    bl4dylion4ik committed Jul 14, 2023
    Configuration menu
    Copy the full SHA
    38ab6a2 View commit details
    Browse the repository at this point in the history