Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save/restore the ANTLR RustLexer specific internals. #5624

Merged
merged 2 commits into from
Mar 8, 2023

Conversation

lkishalmi
Copy link
Contributor

@vieiro You might this one. I have not tested this trough, it just compiles.
The NB lexer saves/restores the lexer input at some wierd positions, and if the ANTLR lexer state is not fully saved/restored it can cause wide range of issues.

@lkishalmi lkishalmi added the Rust [ci] enable Rust tests label Mar 7, 2023
@lkishalmi lkishalmi requested a review from vieiro March 7, 2023 21:46
@lkishalmi
Copy link
Contributor Author

The second commit is kind of optional, though I thought it is not good to store the whole token as that has attached fields, like the input CharStream, etc. that is better not to pass around. It seems the RustLexerBase only relies the null-ness and the type of the token, so changed the code a bit...

Copy link
Contributor

@vieiro vieiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good, thanks, @lkishalmi !

@lkishalmi lkishalmi merged commit 12f20e8 into apache:master Mar 8, 2023
pepness pushed a commit to pepness/incubator-netbeans that referenced this pull request Mar 17, 2023
* Save/restore the ANTLR RustLexer specific internals.

* Use just the token type in RustLexerBase
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Rust [ci] enable Rust tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants