Unexpected behavior in CharTokenizer when setting split_with_space=True and non_lang_syms is not None

[mlxu995]

An unexpected empty character appears in the output when using CharTokenizer with split_with_space=True to process text containing non-language symbols.
Input text: "你 好 问 问 <NIHAO_WENWEN>" (characters separated by spaces, <NIHAO_WENWEN> is a non-language symbol)
Expected output: ['你', '好', '问', '问', '<NIHAO_WENWEN>']
Actual output: ['你', '好', '问', '问', '', '<NIHAO_WENWEN>'] (contains an unexpected empty character)

I would like to change parts = [w for w in parts if len(w.strip()) > 0] to parts = [w.strip() for w in parts if len(w.strip()) > 0] in char_tokenizer.py line 42

This modification will affect tokens with leading or trailing spaces, and I am not sure if it has some side effects.

[mlxu995]

@Mddct 周哥给看下呗

[Mddct]

提个pr修一下 顺便测测不split的时候 表现是否正常

[mlxu995]

提个pr修一下 顺便测测不split的时候 表现是否正常

符合预期，然后不split时同样无法支持英文