How to pretrain my own data with this pytorch code?

[Gpwner]

I wonder how to pretrain with my own data.

[MuruganR96]

Sir @Gpwner i think you have to refer to do pretrain for google/bert repo( https://github.com/google-research/bert#pre-training-with-bert ) and then convert tensorflow model as pytorch.

[thomwolf]

A pre-training script is now included in master thanks to @tholor's PR #124

[Gpwner]

A pre-training script is now included in master thanks to @tholor's PR #124
Thanks ！！
Does it support Multiple GPU？Because the official script does not support Multiple GPU

[thomwolf]

It does (you can read more about it here in the readme)

[Gpwner]

It does

great job，thanks ～

[thomwolf]

All thanks should go to @tholor :-)