Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Shared t5 code #286

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

WIP: Shared t5 code #286

wants to merge 25 commits into from

Commits on Jun 27, 2022

  1. WIP

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    cc6dd1d View commit details
    Browse the repository at this point in the history

  2. Fix shared T5

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    fbc1d04 View commit details
    Browse the repository at this point in the history

  3. WIP

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    91a20aa View commit details
    Browse the repository at this point in the history

  4. Shared t5 tests

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    e55b603 View commit details
    Browse the repository at this point in the history

  5. Woops

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    b5568fb View commit details
    Browse the repository at this point in the history

  6. Woops

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    50cf6d9 View commit details
    Browse the repository at this point in the history

  7. Hack my way into fix attn_mask

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    dcb2d61 View commit details
    Browse the repository at this point in the history

  8. DS has poor default

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    04cdc2e View commit details
    Browse the repository at this point in the history

  9. Maybe this is better

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    ffbe3bf View commit details
    Browse the repository at this point in the history

  10. Maybe this is better

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    90ee34c View commit details
    Browse the repository at this point in the history

  11. SEP is only defined for HFTokenizer

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    ed3dbea View commit details
    Browse the repository at this point in the history

  12. add --vocab-extra-ids 100

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    c93b563 View commit details
    Browse the repository at this point in the history

  13. Hopefully this fixes MLM

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    ef205dd View commit details
    Browse the repository at this point in the history

  14. Hopefully this fixes MLM

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    c71b379 View commit details
    Browse the repository at this point in the history

  15. Fix MLM datasets length

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    9da7588 View commit details
    Browse the repository at this point in the history

  16. I think GPT2 tokenizer might be missing <sep> token

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    07f0ce5 View commit details
    Browse the repository at this point in the history

  17. I think GPT2 tokenizer might be missing <sep> token

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    12626f6 View commit details
    Browse the repository at this point in the history

  18. Flatten to pass tuples of tensor instead

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    52c42b2 View commit details
    Browse the repository at this point in the history

  19. Lambda function can't unpack tuples

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    b25803b View commit details
    Browse the repository at this point in the history

  20. Woops

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    26d9dce View commit details
    Browse the repository at this point in the history

  21. More dirty fixes

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    fc0560a View commit details
    Browse the repository at this point in the history

  22. Woops

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    8578ab3 View commit details
    Browse the repository at this point in the history

  23. Removing erroneous trailing layers

    @thomasw21
    thomasw21 committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    216a3f5 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2022

  1. Add TODO for loading previous checkpoint

    @thomasw21
    thomasw21 committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    f557523 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2022

  1. WIP: conversion script:

    @thomasw21
    thomasw21 committed Jul 4, 2022
    Configuration menu
    Copy the full SHA
    e73aa58 View commit details
    Browse the repository at this point in the history