Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

how to train RoBERTa from scratch #1381

Closed
008karan opened this issue Sep 30, 2019 · 4 comments
Closed

how to train RoBERTa from scratch #1381

008karan opened this issue Sep 30, 2019 · 4 comments
Labels
wontfix

Comments

@008karan
Copy link

❓ Questions & Help

I want to train RoBERTa model from scratch on different language. Is there any implementation available here to do this?
@rlouf rlouf added the usage label Oct 7, 2019
@FrankWork
Copy link

https://github.com/pytorch/fairseq/blob/master/examples/roberta/README.pretraining.md

@stale
Copy link

stale bot commented Dec 7, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Dec 7, 2019
@stale stale bot closed this as completed Dec 14, 2019
@ksopyla ksopyla mentioned this issue Feb 11, 2020
Closed
@julien-c
Copy link
Member

You can now leave --model_name_or_path to None in run_language_modeling.py to train a model from scratch.

See also https://huggingface.co/blog/how-to-train

@miluki01
Copy link

When I put new --config_name and --tokenizer_name. It shows me that
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Anyone can help me?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@julien-c @rlouf @miluki01 @FrankWork @008karan