Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) #4942

Merged
merged 624 commits into from
Aug 6, 2024
Merged

[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) #4942

merged 624 commits into from
Aug 6, 2024
This pull request is big! We’re only showing the most recent 250 commits.

Commits on Jul 15, 2024

  1. Format

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    10ed714 View commit details
    Browse the repository at this point in the history

  2. modified LLM.generate() error message

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    78d3d3c View commit details
    Browse the repository at this point in the history

  3. wip engine is_encoder_decoder() setting

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    6c95380 View commit details
    Browse the repository at this point in the history

  4. formatting

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    304caed View commit details
    Browse the repository at this point in the history

  5. formatting?

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    7b0803b View commit details
    Browse the repository at this point in the history

  6. Sequence may be constructed with encoder/decoder LLMInput configurations

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5525511 View commit details
    Browse the repository at this point in the history

  7. wip but having wllm.commit_id error

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    dd4031c View commit details
    Browse the repository at this point in the history

  8. correctly constructing enc/dec sequences

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    8dccaa5 View commit details
    Browse the repository at this point in the history

  9. formatting

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    336a77d View commit details
    Browse the repository at this point in the history

  10. wip

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    46397c7 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    f85997b View commit details
    Browse the repository at this point in the history

  12. wip

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    251f899 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    9141347 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. wip

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    ddaf0ad View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    54ff142 View commit details
    Browse the repository at this point in the history

  3. conftest: encoder/decoder example prompts

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    92d9f48 View commit details
    Browse the repository at this point in the history

  4. Hfrunner greedy logprobs limit

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    c5846ac View commit details
    Browse the repository at this point in the history

  5. input preparation now includes encoder-oriented input setup:

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    374880f View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    796d7a3 View commit details
    Browse the repository at this point in the history

  7. VllmRunner encoder/decoder methods

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    42ac66b View commit details
    Browse the repository at this point in the history

  8. bart parallel vocab

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    850a97e View commit details
    Browse the repository at this point in the history

  9. zip enc/dec prompts; formatting

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    3c7e19d View commit details
    Browse the repository at this point in the history

  10. wip

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    e534ffc View commit details
    Browse the repository at this point in the history

  11. encoder/decoder input processing; formatting

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    97d81f0 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    87ed3b6 View commit details
    Browse the repository at this point in the history

  13. incorporated encoder sequence into request-add functionality

    @afeldman-nm
    afeldman-nm committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    713d095 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    aea8d34 View commit details
    Browse the repository at this point in the history

  2. fixed decoder-only bug

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    159c7bc View commit details
    Browse the repository at this point in the history

  3. bugfix

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    16c9aa2 View commit details
    Browse the repository at this point in the history

  4. wip

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    03aea18 View commit details
    Browse the repository at this point in the history

  5. wip

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ef80c85 View commit details
    Browse the repository at this point in the history

  6. fixed scheduler bug

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f8dd4a5 View commit details
    Browse the repository at this point in the history

  7. format

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c2ff615 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    31127fa View commit details
    Browse the repository at this point in the history

  9. bugfix

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1c6e06d View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    0cc14ab View commit details
    Browse the repository at this point in the history

  11. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    3656dc6 View commit details
    Browse the repository at this point in the history

  12. fixed sequence bug

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    aee5f16 View commit details
    Browse the repository at this point in the history

  13. added examples utils w/ context manager for backend override; applied… 

    … to enc/dec example to force XFormers
    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ef94623 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    50ad5ff View commit details
    Browse the repository at this point in the history

  15. formatting

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    b277180 View commit details
    Browse the repository at this point in the history

  16. added encoder/decoder example to examples test

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    cac6283 View commit details
    Browse the repository at this point in the history

  17. wip refactoring

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f54f276 View commit details
    Browse the repository at this point in the history

  18. refactor

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    597a07d View commit details
    Browse the repository at this point in the history

  19. RequestOutput & SequenceGroup now include encoder prompt in output, a… 

    …s does encoder/decoder example.
    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9f5a02c View commit details
    Browse the repository at this point in the history

  20. wip parallel bart but encountering GPU count issue

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    94c904f View commit details
    Browse the repository at this point in the history

  21. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9da8fb3 View commit details
    Browse the repository at this point in the history

  22. tweaks to enc/dec example

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1f8c52f View commit details
    Browse the repository at this point in the history

  23. formatting

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1808846 View commit details
    Browse the repository at this point in the history

  24. wip

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f15eacf View commit details
    Browse the repository at this point in the history

  25. modified HF behavior in BART test to be truly greedy

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    6c940f8 View commit details
    Browse the repository at this point in the history

  26. formatting

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    949ac02 View commit details
    Browse the repository at this point in the history

  27. wip parallelizing BART

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    88c058e View commit details
    Browse the repository at this point in the history

  28. wip activation parallelization

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    31e335f View commit details
    Browse the repository at this point in the history

  29. merged in upstream changes; left some formatting issues which I expec… 

    …t to be fixed upstream
    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c092ed4 View commit details
    Browse the repository at this point in the history

  30. Merge branch 'infra_enc_dec_model_runner' into infra_enc_dec_model_ru… 

    …nner_parallel_bart
    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    d7bd617 View commit details
    Browse the repository at this point in the history

  31. wip:

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    69f0379 View commit details
    Browse the repository at this point in the history

  32. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9fdd047 View commit details
    Browse the repository at this point in the history

  33. Merge branch 'infra_enc_dec_model_runner_reviews' into infra_enc_dec_… 

    …model_runner_parallel_bart
    @afeldman-nm
    afeldman-nm committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    584c01e View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2024

  1. wip merge

    @afeldman-nm
    afeldman-nm committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    41ccf0c View commit details
    Browse the repository at this point in the history

  2. additional merge

    @afeldman-nm
    afeldman-nm committed Jul 20, 2024
    Configuration menu
    Copy the full SHA
    ffa99b2 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    a22f56c View commit details
    Browse the repository at this point in the history

  2. CommonMetadataBuilder sets block_tables constructor arg of metadata

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c00e0a8 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    32967c1 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'infra_enc_dec_model_runner' into infra_enc_dec_model_ru… 

    …nner_parallel_bart
    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    a33b501 View commit details
    Browse the repository at this point in the history

  5. equalized some generation/sampling config settings between enc/dec HF… 

    …,vLLM, nonetheless still not perfect match
    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    a16cabb View commit details
    Browse the repository at this point in the history

  6. Merge branch 'infra_enc_dec_model_runner' into infra_enc_dec_model_ru… 

    …nner_parallel_bart
    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    abbb427 View commit details
    Browse the repository at this point in the history

  7. BART MLPs parallelized

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    00198a6 View commit details
    Browse the repository at this point in the history

  8. parallelized BART learned positional embedding

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    fb3227f View commit details
    Browse the repository at this point in the history

  9. all attention layer output linears are parallelized

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    e5bb9de View commit details
    Browse the repository at this point in the history

  10. encoder attention & decoder self-attention parallelized

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    74abe22 View commit details
    Browse the repository at this point in the history

  11. parallelized LM head

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    9bbed43 View commit details
    Browse the repository at this point in the history

  12. parallelized enc/dec cross-attention, using a slight hack

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    fdf71de View commit details
    Browse the repository at this point in the history

  13. fixed bug where underlying Attention was constructed using full head-… 

    …count
    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    3551b6b View commit details
    Browse the repository at this point in the history

  14. bart is parallelized, modulo an unfortunate hack for QKVParallelLinea… 

    …r in cross-attention
    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    b174c7a View commit details
    Browse the repository at this point in the history

  15. commented out BART TP=4

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c43a6ed View commit details
    Browse the repository at this point in the history

  16. upstream merge

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    b90b6b6 View commit details
    Browse the repository at this point in the history

  17. Merge branch 'infra_enc_dec_model_runner_reviews' into infra_enc_dec_… 

    …model_runner
    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    14831b0 View commit details
    Browse the repository at this point in the history

  18. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    427032a View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. fixed bug in how conftest was handling HF encoder/decoder outputs; di… 

    …sabled HF engram repeat checks
    @afeldman-nm
    afeldman-nm committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    c51a168 View commit details
    Browse the repository at this point in the history

  2. set up None/empty str tests which are not passing

    @afeldman-nm
    afeldman-nm committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    b01937f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    48a742d View commit details
    Browse the repository at this point in the history

  4. Merge branch 'infra_enc_dec_model_runner_correctness' into infra_enc_… 

    …dec_model_runner_reviews
    @afeldman-nm
    afeldman-nm committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    b283544 View commit details
    Browse the repository at this point in the history

  5. wip

    @afeldman-nm
    afeldman-nm committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    059273f View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    229847b View commit details
    Browse the repository at this point in the history

  7. deleted unnecessary dependency

    @afeldman-nm
    afeldman-nm committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    7e7bbd9 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    4a6e39e View commit details
    Browse the repository at this point in the history

  2. empty-string decoder input is now handled for encoder/decoder

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    aa01d71 View commit details
    Browse the repository at this point in the history

  3. enc/dec handles empty str and None decoder prompts correctly

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    0b29fd2 View commit details
    Browse the repository at this point in the history

  4. typing fix

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    dd784b5 View commit details
    Browse the repository at this point in the history

  5. fixed bugs in handling non-text formats for individual prompts

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    61d2ad2 View commit details
    Browse the repository at this point in the history

  6. example includes prompt zipper

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    f36ffb5 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    c493d40 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    be58d8a View commit details
    Browse the repository at this point in the history

  9. _free_seq_group() -> _free_seq_group_cross_attn_blocks()

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    02114bd View commit details
    Browse the repository at this point in the history

  10. refactoring

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    5a270ff View commit details
    Browse the repository at this point in the history

  11. formatting

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ed4a56b View commit details
    Browse the repository at this point in the history

  12. removed unnecessary argument reordering

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    4b5b2cf View commit details
    Browse the repository at this point in the history

  13. enc/dec example comments'

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    d82b273 View commit details
    Browse the repository at this point in the history

  14. responses to feedback

    @afeldman-nm
    afeldman-nm committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    0af58ec View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    bed9bcd View commit details
    Browse the repository at this point in the history

  2. fixed bug caused by upstream refactoring

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    47b4eb2 View commit details
    Browse the repository at this point in the history

  3. formatting

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    393515e View commit details
    Browse the repository at this point in the history

  4. upstream merge

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    fb5a2bc View commit details
    Browse the repository at this point in the history

  5. Removed lora from enc/dec model runner

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    c2cc010 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    175ea95 View commit details
    Browse the repository at this point in the history

  7. removed lora & vision & mm code from enc/dec modelrunner

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    3327e5b View commit details
    Browse the repository at this point in the history

  8. checked out examples/offline_inference.py from main

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    47c5548 View commit details
    Browse the repository at this point in the history

  9. updated RequestOutput docstring

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    1bb7ad9 View commit details
    Browse the repository at this point in the history

  10. updated RequestOutput docstring

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    035d90d View commit details
    Browse the repository at this point in the history

  11. Sequence docstring

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    64685ac View commit details
    Browse the repository at this point in the history

  12. removed flashinfer references from enc/dec modelrunner

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d1751db View commit details
    Browse the repository at this point in the history

  13. format

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f0abcc2 View commit details
    Browse the repository at this point in the history

  14. removed chunked prefill logic/docstring text from enc/dec modelrunner

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    4bb7fc4 View commit details
    Browse the repository at this point in the history

  15. removed prefix caching from enc/dec modelrunner

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    a936faa View commit details
    Browse the repository at this point in the history

  16. updated _prepare_encoder_model_input_tensors docstring

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    7cdc1ca View commit details
    Browse the repository at this point in the history

  17. refactoring and cleanup

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    dc953e1 View commit details
    Browse the repository at this point in the history

  18. formatting

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d132a7f View commit details
    Browse the repository at this point in the history

  19. wip

    @afeldman-nm
    afeldman-nm committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    3938389 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. trimmed down enc/dec model runner

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    a77ef37 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into infra_enc_dec_model_runner

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    b358767 View commit details
    Browse the repository at this point in the history

  3. conftest refactor

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    b5f102f View commit details
    Browse the repository at this point in the history

  4. encoder/decoder model runner test docstrings

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    63c76d9 View commit details
    Browse the repository at this point in the history

  5. enc/dec model runner tests - added explanatory comments; removed unne… 

    …cessary tests; added sampling index validation to decode-phase test
    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    0d35915 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    4a3fcec View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    456090c View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    080fec6 View commit details
    Browse the repository at this point in the history

  9. Added all encoder/decoder unsupported scenario checks except for inva… 

    …lid kernels
    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    ef5d711 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    f12c53f View commit details
    Browse the repository at this point in the history

  11. added logic to force backend selection without environment variable.

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    1a8b284 View commit details
    Browse the repository at this point in the history

  12. wip removing context managers that force kernel choice for enc/dec

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    5606345 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'infra_enc_dec_model_runner_checks' into infra_enc_dec_m… 

    …odel_runner_reviews
    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    8513260 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    e0a5880 View commit details
    Browse the repository at this point in the history

  15. missing import; formatting

    @afeldman-nm
    afeldman-nm committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    783a416 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 27, 2024
    Configuration menu
    Copy the full SHA
    8aac5a3 View commit details
    Browse the repository at this point in the history

  2. wip removing kernel override fixture

    @afeldman-nm
    afeldman-nm committed Jul 27, 2024
    Configuration menu
    Copy the full SHA
    5d9f15d View commit details
    Browse the repository at this point in the history

  3. fully removed/repaced backend forcing

    @afeldman-nm
    afeldman-nm committed Jul 27, 2024
    Configuration menu
    Copy the full SHA
    e86f63c View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 27, 2024
    Configuration menu
    Copy the full SHA
    fe8a0dd View commit details
    Browse the repository at this point in the history

  5. Merge branch 'infra_enc_dec_model_runner_reviews' into infra_enc_dec_… 

    …model_runner_checks
    @afeldman-nm
    afeldman-nm committed Jul 27, 2024
    Configuration menu
    Copy the full SHA
    389d195 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    f081438 View commit details
    Browse the repository at this point in the history

  2. added enc/dec not-impl err strings import

    @afeldman-nm
    afeldman-nm committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    eab9e6b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    f595517 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    7f53daf View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    b2ac87c View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    47954c7 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    665e0c2 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. upstream merge

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    e48fdb0 View commit details
    Browse the repository at this point in the history

  2. BART uses position argument

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    6dc3fba View commit details
    Browse the repository at this point in the history

  3. merge; fix typing issues

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    3656094 View commit details
    Browse the repository at this point in the history

  4. removed prompt adapter logic from enc/dec model runner

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    2558061 View commit details
    Browse the repository at this point in the history

  5. removed redundant code from enc/dec modelrunner; fixed test comments

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c744be2 View commit details
    Browse the repository at this point in the history

  6. removed pipeline parallelism references

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    f94fea7 View commit details
    Browse the repository at this point in the history

  7. wip removing decoder-only stuff that should be singleton

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c448cc6 View commit details
    Browse the repository at this point in the history

  8. removed examples/utils

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    87c34f4 View commit details
    Browse the repository at this point in the history

  9. assorted fixes; corrected BART model output dimensionality

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    98cd380 View commit details
    Browse the repository at this point in the history

  10. formatting

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    eb7b80e View commit details
    Browse the repository at this point in the history

  11. removed get_encoder()/get_decoder()

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    d7b6c60 View commit details
    Browse the repository at this point in the history

  12. unnecessary imports

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    5c9d187 View commit details
    Browse the repository at this point in the history

  13. removed unnecessary code & comments

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    4e0cdd8 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    9fd0a88 View commit details
    Browse the repository at this point in the history

  15. ran isort

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    eda6857 View commit details
    Browse the repository at this point in the history

  16. reorganized enc/dec model runner encoder inputs computation

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    390afa5 View commit details
    Browse the repository at this point in the history

  17. formatting/typing

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    4540cfe View commit details
    Browse the repository at this point in the history

  18. removed redundant code for enc/dec MR test

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    f4bf69e View commit details
    Browse the repository at this point in the history

  19. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    6ce2c9d View commit details
    Browse the repository at this point in the history

  20. wip

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    ec728ab View commit details
    Browse the repository at this point in the history

  21. Merge branch 'infra_enc_dec_model_runner' into infra_enc_dec_model_ru… 

    …nner_review
    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    1e9fa39 View commit details
    Browse the repository at this point in the history

  22. Removed unnecessary code

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    57397c1 View commit details
    Browse the repository at this point in the history

  23. formatting; isort

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    61ba9dd View commit details
    Browse the repository at this point in the history

  24. Merge branch 'infra_enc_dec_model_runner' into infra_enc_dec_model_ru… 

    …nner_review
    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    5407769 View commit details
    Browse the repository at this point in the history

  25. unnecessary import

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    02fffd2 View commit details
    Browse the repository at this point in the history

  26. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    7e1d3c3 View commit details
    Browse the repository at this point in the history

  27. added type: ignore to ActorDiedError import because it was interferin… 

    …g with my type checker & the type checker failure seemed not to be meaningful
    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    2ab8a73 View commit details
    Browse the repository at this point in the history

  28. restructured & reformatted llm engine enc/dec code

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    fe4638e View commit details
    Browse the repository at this point in the history

  29. improved explanatory comment.

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    0c6cc90 View commit details
    Browse the repository at this point in the history

  30. LLMEngine logic working

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    af976d7 View commit details
    Browse the repository at this point in the history

  31. reorganized enc/dec llm engine logic

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c8cbcb1 View commit details
    Browse the repository at this point in the history

  32. format; refactor

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c305d16 View commit details
    Browse the repository at this point in the history

  33. enc/dec decoder prompt preprocessing is now entirely list-oriented

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c36f8d1 View commit details
    Browse the repository at this point in the history

  34. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    ade461f View commit details
    Browse the repository at this point in the history

  35. added decoder prompt processing comment

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    39e39cc View commit details
    Browse the repository at this point in the history

  36. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    88bcae2 View commit details
    Browse the repository at this point in the history

  37. enc/dec MR xformers log message

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c09b4ed View commit details
    Browse the repository at this point in the history

  38. XFormers kernel forcing log message; enforce eager log message with w… 

    …arning
    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    3a510b2 View commit details
    Browse the repository at this point in the history

  39. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    505b4ec View commit details
    Browse the repository at this point in the history

  40. explanatory comment

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    f279112 View commit details
    Browse the repository at this point in the history

  41. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    25e3bbd View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    41d8262 View commit details
    Browse the repository at this point in the history

  2. fixed comments

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    593d439 View commit details
    Browse the repository at this point in the history

  3. fixed BART bug for None prompts

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    b9941cd View commit details
    Browse the repository at this point in the history

  4. moved BART-specific test config from conftest to test_bart

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    82a2ba6 View commit details
    Browse the repository at this point in the history

  5. upstream merge

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    f713731 View commit details
    Browse the repository at this point in the history

  6. Updated vLLM enforce_eager default behavior

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    23fe370 View commit details
    Browse the repository at this point in the history

  7. LLM constructor comment explains enforce_eager defaults

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    47acc7e View commit details
    Browse the repository at this point in the history

  8. refactored out enc/dec mr helper function

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    f8ec35a View commit details
    Browse the repository at this point in the history

  9. added enforce_eager=True to list of valid-scenario requirements for e… 

    …ncoder/decoder model runner; added error strings for enforce_eager and backend related scenarios
    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    8aa8665 View commit details
    Browse the repository at this point in the history

  10. small fix

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    cf12aaf View commit details
    Browse the repository at this point in the history

  11. enc/dec MR attempts to override backend but fails if already overridd… 

    …en, accounting for both global var and env var overrides.
    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    1d6de48 View commit details
    Browse the repository at this point in the history

  12. added error message for unsupported enc/dec + prompt adapter scenario

    @abf149
    abf149 committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    fd6c59b View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_review

    @abf149
    abf149 committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    72c7c22 View commit details
    Browse the repository at this point in the history

  2. added encoder/decoder distributed correctness test

    @abf149
    abf149 committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    ead6527 View commit details
    Browse the repository at this point in the history

  3. enc dec changes in distributed test pipeline

    @afeldman-nm
    afeldman-nm committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    dfaf9cf View commit details
    Browse the repository at this point in the history

  4. enc/dec distributed test works

    @afeldman-nm
    afeldman-nm committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    e16bbd8 View commit details
    Browse the repository at this point in the history

  5. multi-GPU enc/dec test comment

    @afeldman-nm
    afeldman-nm committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    7944b89 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2024

  1. upstream merge

    @afeldman-nm
    afeldman-nm committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    fda01b2 View commit details
    Browse the repository at this point in the history

  2. 4 gpu enc/dec test

    @afeldman-nm
    afeldman-nm committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    e3715de View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    f42d1f8 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2024

  1. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    6d7821d View commit details
    Browse the repository at this point in the history

  2. removed commented-out tp arg from enc/dec example

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    28bd8db View commit details
    Browse the repository at this point in the history

  3. 'specified' typo

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    dabfd76 View commit details
    Browse the repository at this point in the history

  4. get_env_variable_attn_backend rename

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    61fc254 View commit details
    Browse the repository at this point in the history

  5. removed meaningless decode comment

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    3425076 View commit details
    Browse the repository at this point in the history

  6. removed unnecessary comment fomr bart.py

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    f93ca97 View commit details
    Browse the repository at this point in the history

  7. correct usage of decoder start token id

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    67aa90f View commit details
    Browse the repository at this point in the history

  8. handle all union type scenarios

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    7fa2177 View commit details
    Browse the repository at this point in the history

  9. removed 4 GPU BART test

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    d1ca700 View commit details
    Browse the repository at this point in the history

  10. enc/dec mr tests used prepare_model_input()

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    d66892f View commit details
    Browse the repository at this point in the history

  11. streamlined enc/dec mr encoder model input prep

    @afeldman-nm
    afeldman-nm committed Aug 4, 2024
    Configuration menu
    Copy the full SHA
    605b03c View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. bugfix: shape of empty cross-attn block table

    @afeldman-nm
    afeldman-nm committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    539c812 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    d209037 View commit details
    Browse the repository at this point in the history

  3. format

    @afeldman-nm
    afeldman-nm committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    528f2d0 View commit details
    Browse the repository at this point in the history

  4. refactoring

    @afeldman-nm
    afeldman-nm committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    b3a8564 View commit details
    Browse the repository at this point in the history

  5. sampling metadata simpliciation in test

    @afeldman-nm
    afeldman-nm committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    a9be976 View commit details
    Browse the repository at this point in the history

  6. unnecessary import

    @afeldman-nm
    afeldman-nm committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    6d428a4 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    8eb5d11 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. upstream merge

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    6f81d9d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    d132be4 View commit details
    Browse the repository at this point in the history

  3. Update vllm/core/scheduler.py 

    Co-authored-by: Nick Hill <nickhill@us.ibm.com>
    @afeldman-nm @njhill
    afeldman-nm and njhill authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    0756240 View commit details
    Browse the repository at this point in the history

  4. Update vllm/utils.py 

    Co-authored-by: Nick Hill <nickhill@us.ibm.com>
    @afeldman-nm @njhill
    afeldman-nm and njhill authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    d1bb4c5 View commit details
    Browse the repository at this point in the history

  5. Update vllm/utils.py 

    Co-authored-by: Nick Hill <nickhill@us.ibm.com>
    @afeldman-nm @njhill
    afeldman-nm and njhill authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    ba147a0 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'infra_enc_dec_model_runner' into infra_enc_dec_model_ru… 

    …nner_reviews
    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    4049564 View commit details
    Browse the repository at this point in the history

  7. conftest fix

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    79847b5 View commit details
    Browse the repository at this point in the history

  8. LLM Engine fixes

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    0b5bc8c View commit details
    Browse the repository at this point in the history

  9. vllm inputs fixes

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    50b4a4f View commit details
    Browse the repository at this point in the history

  10. Additional LLMEngine fixes

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    d6d3e92 View commit details
    Browse the repository at this point in the history

  11. moved enc/dec + enforce eager default logic to model config

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    56ab71d View commit details
    Browse the repository at this point in the history

  12. arg_utils cleanup

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    8b58f3c View commit details
    Browse the repository at this point in the history

  13. iterating over dict items

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    f8ab672 View commit details
    Browse the repository at this point in the history

  14. formatting

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    4c0d41c View commit details
    Browse the repository at this point in the history

  15. cleanup

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    74a6a21 View commit details
    Browse the repository at this point in the history

  16. wip

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    71456e7 View commit details
    Browse the repository at this point in the history

  17. last trailing commas

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    9625468 View commit details
    Browse the repository at this point in the history

  18. format

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    2e31471 View commit details
    Browse the repository at this point in the history

  19. formatting

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    60288d7 View commit details
    Browse the repository at this point in the history

  20. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    1ccee00 View commit details
    Browse the repository at this point in the history

  21. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    0fe9963 View commit details
    Browse the repository at this point in the history

  22. offline_inference_vision_language.py replaces llava test in pipeline

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    27617e0 View commit details
    Browse the repository at this point in the history

  23. atol 1e-2 -> 2e-2 in test_flash_attn.py

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    8ce1aef View commit details
    Browse the repository at this point in the history

  24. Merge branch 'main' into infra_enc_dec_model_runner_reviews

    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    e6b4c16 View commit details
    Browse the repository at this point in the history

  25. Merge branch 'infra_enc_dec_model_runner_reviews' into infra_enc_dec_… 

    …model_runner
    @afeldman-nm
    afeldman-nm committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    8ecbbc5 View commit details
    Browse the repository at this point in the history