TypeError: _tokenize() got an unexpected keyword argument 'add_prefix_space'

[r-mohammadi]

Hi
In the preprocess, when we use albert model in 5_dump_features, we get this error:
TypeError: _tokenize() got an unexpected keyword argument 'add_prefix_space'
but it works fine for roberta model

[oldkong]

Hi
In the preprocess, when we use albert model in 5_dump_features, we get this error:
TypeError: _tokenize() got an unexpected keyword argument 'add_prefix_space'
but it works fine for roberta model

I think your error comes from line 353: sub_tokens = tokenizer.tokenize(token, add_prefix_space=True)

you can try to overwrtie it by: sub_tokens = tokenizer.tokenize(token)