Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change LLM templates with files #2

Open
Pelochus opened this issue Apr 29, 2024 · 3 comments
Open

Change LLM templates with files #2

Pelochus opened this issue Apr 29, 2024 · 3 comments

Comments

@Pelochus
Copy link
Owner

Essentially changing the LLM template dynamically at program execution. Add a parameter that reads a file and changes the corresponding variables. Especially useful if using various different LLMs. This needs some research first, as I know pretty much nothing about the templates.

@Pelochus
Copy link
Owner Author

Pelochus commented May 1, 2024

Initial point of reference:

https://gist.github.com/av1d/ce58ef738902ec0365fe828720be31e5

@Pelochus
Copy link
Owner Author

Important link to this issue:

https://www.reddit.com/r/RockchipNPU/comments/1cpngku/rknnllm_v101_lets_talk_about_converting_and/

I could also add an option to not use files, use parameters or something. Perhaps reconverting everything with a modified tokenizer.json could be also a good option, for having the same template for every converted model

@80Builder80
Copy link

We anticipated better with an update. Instead we get useless.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants