Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the error message and automatically download a dataset when a part of a recipe is not found #208

Merged
merged 49 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
c60e26e
Automatically download the dataset of a recipe if it's not already in…
sanjaycal Jan 10, 2025
5f7ab22
Updated the error messages to better match what was going on, with th…
sanjaycal Jan 10, 2025
85313d8
Updated the error messages to better match what was going on, with th…
sanjaycal Jan 10, 2025
ce8c49c
Merge branch 'sanjay-fix-recipes-that-need-downloading' of https://gi…
sanjaycal Jan 13, 2025
1de0216
change default api port to 8338
aliasaria Jan 13, 2025
8a20bb5
remove transparency from the main app window -- can't get it to work …
aliasaria Jan 13, 2025
e65131b
Make importing models work, still need to add the downloading popup
sanjaycal Jan 13, 2025
b2752af
Remove old comments from DownloadProgressBox
dadmobile Jan 13, 2025
af502c8
Rename parameter in DownloadProgressBox to not be model specific.
dadmobile Jan 13, 2025
d4fa2af
Make the currently downloading model show up on the Train page with D…
sanjaycal Jan 13, 2025
22470f1
fixes #61
aliasaria Jan 13, 2025
93b2306
Hide GPU memory if you are on an apple machine with MLX
aliasaria Jan 13, 2025
ca556f8
fade mlx logo a bit to reduce harshness
aliasaria Jan 13, 2025
ba5c5d2
expand python libraries section of computer tab
aliasaria Jan 13, 2025
d085c3b
split computer page into tabs
aliasaria Jan 13, 2025
c074f53
Fixed the download progress bar to import clamp, which is needed for …
sanjaycal Jan 13, 2025
a0ecaa1
Fixed the training download view so that it actually updates
sanjaycal Jan 13, 2025
6a1af06
show nvidia logo if you have it
aliasaria Jan 13, 2025
a5c9438
Fix variable name that wasn't updated on DownloadProgressBar.
dadmobile Jan 13, 2025
b9519c3
new logo
aliasaria Jan 14, 2025
3b0521b
hide the progress box if the info is loading
sanjaycal Jan 14, 2025
fe5c067
Fixed the training status text when a failure occurs from success to …
sanjaycal Jan 14, 2025
de708ea
Merge pull request #210 from transformerlab/fix-training-status-text
sanjaycal Jan 14, 2025
6395f67
hide the progress box if the info is loading
sanjaycal Jan 14, 2025
ce1162a
Merge branch 'sanjay-fix-recipes-that-need-downloading' of https://gi…
sanjaycal Jan 14, 2025
38dd9f6
checking in a temporary log viewer -- this is hardcoded to only work …
aliasaria Jan 15, 2025
b0c978d
improve look of xterm at bottom of screen
aliasaria Jan 15, 2025
c26fe79
remove hardcoded URL in log viewer
aliasaria Jan 15, 2025
dfff775
improve global log panel on resize
aliasaria Jan 15, 2025
2e876c3
resize log window using chevron
aliasaria Jan 15, 2025
43c442c
tweaks to size
aliasaria Jan 15, 2025
fa04640
fit terminal to window better
aliasaria Jan 15, 2025
daf30d5
hide terminal completely on collapse
aliasaria Jan 15, 2025
8f3c6ef
allow any logendpoint
aliasaria Jan 15, 2025
b2367a7
Streamthe output of trainers
aliasaria Jan 15, 2025
daf3f71
fix color
aliasaria Jan 16, 2025
6eab2cf
fix
aliasaria Jan 16, 2025
663c209
allow the output terminal to display an initial message
aliasaria Jan 16, 2025
bf1b90b
remove unused imports
aliasaria Jan 16, 2025
89a6aa4
add a counter to the login screen to show elapsed seconds
aliasaria Jan 16, 2025
331b915
display an alertbox that shows up if the install is taking too long
aliasaria Jan 16, 2025
1029bbd
Updated the error messages to better match what was going on, with th…
sanjaycal Jan 10, 2025
98d05b1
Make importing models work, still need to add the downloading popup
sanjaycal Jan 13, 2025
7c1ab09
Make the currently downloading model show up on the Train page with D…
sanjaycal Jan 13, 2025
9a0916c
Fixed the download progress bar to import clamp, which is needed for …
sanjaycal Jan 13, 2025
3872aff
Fixed the training download view so that it actually updates
sanjaycal Jan 13, 2025
8239e99
hide the progress box if the info is loading
sanjaycal Jan 14, 2025
c8400b0
Merge branch 'sanjay-fix-recipes-that-need-downloading' of https://gi…
sanjaycal Jan 16, 2025
ee0e255
Switch from alert to confirm to allow the user to not download
sanjaycal Jan 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 23 additions & 4 deletions src/renderer/components/Experiment/Train/ImportRecipeModal.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -110,11 +110,30 @@ export default function ImportRecipeModal({ open, setOpen, mutate }) {
alert("Warning: This recipe does not have an associated dataset")
} else {
let msg = "";
if (!response.model.downloaded) {
msg += "Download model " + response.model.path
}
if (!response.dataset.downloaded) {
msg += "Download dataset " + response.dataset.path
msg += "\n Download dataset " + response.dataset.path + ", this will be done automatically";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can automatically start these downloads without the user knowing what they are getting into? (even if we say this is just temporary)
My suggestion is:

  1. check if anything has to be downloaded. If so they show a confirm instead of an alert.
  2. If they click OK on the confirm THEN do the download.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be done now.

fetch(chatAPI.Endpoints.Dataset.Download(response.dataset.path))
.then((response) => {
if (!response.ok) {
console.log(response);
throw new Error(`HTTP Status: ${response.status}`);
}
return response.json();
})
.catch((error) => {
alert('Download failed:\n' + error);
});
}
if (!response.model.downloaded) {
msg += "\n Download model " + response.model.path + ", this will be done automatically";
chatAPI.downloadModelFromHuggingFace(response.model.path)
.then((response) => {
if (response.status == "error") {
console.log(response);
throw new Error(`${response.message}`);
}
return response;
})
}
if (msg) {
const alert_msg = "Warning: To use this recipe you will need to: " + msg
Expand Down
12 changes: 12 additions & 0 deletions src/renderer/components/Experiment/Train/TrainLoRA.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ import ImportRecipeModal from './ImportRecipeModal';

import dayjs from 'dayjs';
import relativeTime from 'dayjs/plugin/relativeTime';
import CurrentDownloadBox from 'renderer/components/currentDownloadBox';
import DownloadProgressBox from 'renderer/components/Shared/DownloadProgressBox';
dayjs.extend(relativeTime);
var duration = require('dayjs/plugin/duration');
dayjs.extend(duration);
Expand Down Expand Up @@ -118,6 +120,15 @@ export default function TrainLoRA({ experimentInfo }) {
refreshInterval: 2000,
});

const {
data: downloadJobs,
error: downloadJobsError,
isLoading: downloadJobsIsLoading,
mutate: downloadJobsMutate,
} = useSWR(chatAPI.Endpoints.Jobs.GetJobsOfType('DOWNLOAD_MODEL', 'RUNNING'), fetcher, {
refreshInterval: 2000,
});

//Fetch available training plugins
const {
data: pluginsData,
Expand Down Expand Up @@ -173,6 +184,7 @@ export default function TrainLoRA({ experimentInfo }) {
overflow: 'hidden',
}}
>
<DownloadProgressBox jobId={downloadJobs[0]?.id} assetName={downloadJobs[0]?.job_data.model}/>
{/* <Typography level="h1">Train</Typography> */}
<Stack direction="row" justifyContent="space-between" gap={2}>
<Typography level="title-md" startDecorator={<GraduationCapIcon />}>
Expand Down
1 change: 1 addition & 0 deletions src/renderer/components/Shared/DownloadProgressBox.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import {
} from '@mui/joy';

import {
clamp,
formatBytes,
} from '../../lib/utils';
import * as chatAPI from '../../lib/transformerlab-api-sdk';
Expand Down