Skip to content

kb-labb/hface_transformer

Repository files navigation

hface_transformer

Baseline model assuming that everything works here :)

Also good to see how much time it actually takes to train with HF

ToDo

  • BERT wordpiece tokenizer with proper pre-tokenization
    • trained on latin oscar+wiki
  • train-script based on run_mlm.py
  • maybe add some deepspeed
  • sbatch etc.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published