Skip to content

Train text generation model with JavaScript.

Notifications You must be signed in to change notification settings

frost-beta/train-model-with-js

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Train text generation model with JavaScript

This repo hosts some educational scripts for doing basic training on a decoder-only transformer, using node-mlx with Node.js.

Files:

  • model.js - defines the model.
  • input.txt - text file used for training the model.
  • train.js - script for traning.
  • generate.js - script for generating text using the trained model.

Platform

Only Macs with Apple Silicon are supported.

How to use

Download dependencies and run the training script, which generates tokenizer.json and weights.safetensors:

npm install
node train.js

Then use the generate script to actually generate some text from the weights:

node generate.js

What's next

After understanding the basics of model training, you can check the train-llama3-js repo on how to train a Llama3 model with large datasets.

License

Public domain.

About

Train text generation model with JavaScript.

Topics

Resources

Stars

Watchers

Forks