New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set wired limit before starting generation #40

Open

neilmehta24 opened this issue Nov 22, 2024 · 0 comments

Assignees

Labels

enhancement fixed-in-next-release

Member

neilmehta24 commented Nov 22, 2024 •

edited

Loading

mlx_lm added a new feature which sets a wired limit https://github.com/ml-explore/mlx-examples/blob/bd6d910/llms/mlx_lm/utils.py#L298 . This feature should be added to mlx-engine so that we can increase our memory working size.

This will only have an effect on macOS 15, and is a no-op on lower versions.

The text was updated successfully, but these errors were encountered:

neilmehta24 added the enhancement label

neilmehta24 self-assigned this

neilmehta24 mentioned this issue

Upgrade MLX framework, and refactor create_generator to use stream_generate #44

Merged

neilmehta24 added the fixed-in-next-release label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment