Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decouple custom ops in llama_transformer.py Part 2/N #3007

Closed
wants to merge 2 commits into from
Closed

Decouple custom ops in llama_transformer.py Part 2/N #3007

wants to merge 2 commits into from

Commits on Apr 12, 2024

  1. Decouple custom ops in llama_transformer.py Part 1/N (pytorch#3005) 

    Summary:
This is a no-op

Pull Request resolved: pytorch#3005

Test Plan:
CI

Run with

`python -m examples.models.llama2.export_llama -c stories110M.pt -p params.json -kv --use_sdpa_with_kv_cache -X`

and with

`python -m examples.models.llama2.export_llama -c stories110M.pt -p params.json -kv -X`

Make sure both work

Differential Revision: D56048177

Pulled By: mergennachin
    @mergennachin @facebook-github-bot
    mergennachin authored and facebook-github-bot committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    d0d3ec9 View commit details
    Browse the repository at this point in the history

  2. Decouple custom ops in llama_transformer.py Part 2/N (pytorch#3007) 

    Summary:
Pull Request resolved: pytorch#3007

Keep llama_transformer.py to look like stock implementation, so that it can be reused everywhere.

Do module swap

Differential Revision: D56048640
    @mergennachin @facebook-github-bot
    mergennachin authored and facebook-github-bot committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    fbd4d36 View commit details
    Browse the repository at this point in the history