Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Choosing input/total tokens automatically based on available VRAM? #2673

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Choosing input/total tokens automatically based on available VRAM? #2673

wants to merge 12 commits into from
+289 −140

Commits on Oct 21, 2024

  1. Choosing input/total tokens automatically based on available VRAM?

    @Narsil
    Narsil committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    a1aac78 View commit details
    Browse the repository at this point in the history

  2. Update doc.

    @Narsil
    Narsil committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    79469f5 View commit details
    Browse the repository at this point in the history

  3. Remove generated files.

    @Narsil
    Narsil committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    a31db04 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. Trying to fix non chunking targets.

    @Narsil
    Narsil committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    0a01dde View commit details
    Browse the repository at this point in the history

  2. Attempt #2

    @Narsil
    Narsil committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    5c3efbc View commit details
    Browse the repository at this point in the history

  3. fix.

    @Narsil
    Narsil committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    82a6cb8 View commit details
    Browse the repository at this point in the history

  4. QuantLinear is rocm compatible.

    @Narsil
    Narsil committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    849d882 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. Much simpler logic after the overhead.

    @Narsil
    Narsil committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    1053451 View commit details
    Browse the repository at this point in the history

  2. Updating logic + non flash.

    @Narsil
    Narsil committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    6994fa1 View commit details
    Browse the repository at this point in the history

  3. Revert doc text.

    @Narsil
    Narsil committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    cacaba6 View commit details
    Browse the repository at this point in the history

  4. Simple updates.

    @Narsil
    Narsil committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    199973c View commit details
    Browse the repository at this point in the history

  5. Fix integration mt0 (transformers update).

    @Narsil
    Narsil committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    e3db525 View commit details
    Browse the repository at this point in the history