Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove testing litgpt falcon 40b/7b #1504

Open
t-vi opened this issue Dec 2, 2024 · 5 comments
Open

Remove testing litgpt falcon 40b/7b #1504

t-vi opened this issue Dec 2, 2024 · 5 comments
Assignees
@t-vi
Labels
mixology Issues that the mixology team has surfaced

Comments

@t-vi
Copy link
Collaborator

t-vi commented Dec 2, 2024

Disabled due to nvfuser internal assert
NVIDIA/Fuser#3505
@tfogal
Copy link
Collaborator

tfogal commented Dec 9, 2024

Lightning+NVIDIA had an interactive chat on 12/9 and decided that we can stop testing falcon 40b due to low utilization.

@t-vi t-vi changed the title Reenable testing litgpt falcon 40b like Remove testing litgpt falcon 40b like Dec 9, 2024
@tfogal tfogal changed the title Remove testing litgpt falcon 40b like Remove testing litgpt falcon 40b/7b/180b Dec 13, 2024
@tfogal
Copy link
Collaborator

tfogal commented Dec 13, 2024 •
edited
Loading

Thanks, I started talking with mixology about this. I think we might want to drop more than this; retitling.

https://pytorch-lightning.slack.com/archives/C04N128ARJ6/p1734009632309119?thread_ts=1733960395.508069&cid=C04N128ARJ6

  • We can drop Falcon 7B.
  • If Falcon 180B is our largest model, we should keep it.
  • If Falcon 180B is not our largest model, we can drop it.

I am reassigning to Voytec since it'd be on him or Grzegorz to remove them.

@tfogal tfogal changed the title Remove testing litgpt falcon 40b/7b/180b Remove testing litgpt falcon 40b/7b Dec 13, 2024
@tfogal tfogal removed their assignment Dec 13, 2024
@tfogal tfogal added the mixology Issues that the mixology team has surfaced label Dec 13, 2024
@wprazuch
Copy link
Contributor

Hello,
we will keep falcon-180b as it's the largest model.

@tfogal tfogal assigned wprazuch and unassigned tfogal Dec 16, 2024
@wprazuch
Copy link
Contributor

Hello, removed falcon-40b from benchmarks - this issue can be closed now (unfortunately, seems like I have no permissions to do that myself).

@t-vi t-vi assigned t-vi and unassigned wprazuch Dec 23, 2024
@t-vi
Copy link
Collaborator Author

t-vi commented Dec 23, 2024 •
edited
Loading

Thank you, @wprazuch . I'll remove it from the tests and close the issue with that PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@t-vi t-vi

Labels
mixology Issues that the mixology team has surfaced
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tfogal @t-vi @wprazuch