fix: Fix actual token lookup in completion's expand()

[ChayimFriedman2]

It should be left biased, not right biased, because when e.g. the use has typed h then requested completion, the h is what we want to find, not the next token (which might indeed be inside a macro call).

I'm not sure why I wrote right_biased() to begin with (I remember I had a reason and not just "both should work"), I might've copied the code in expand_and_analyze() (which is wrong, because there it lookups on the speculative file, where right biased will always find the correct token and left biased not).

This is still not perfect, because there might not be an identifier already typed then we might still end up in a macro call, but this is the best we can do.

Fixes #18888. The reason the final assert worked is not because it somehow "reset" things, but because there was no macro call after it.

[lnicola]

Could we add a test for this?

[Veykril]

We go have a pick_best_token helper as well for ranking the options by their kind if useful here.

[ChayimFriedman2]

We go have a pick_best_token helper as well for ranking the options by their kind if useful here.

I don't think this will help, I mean I can try both left and right and give left a higher score, but this will add overhead & I don't think it's worth it, it is also going to mess up with the rankings.

I will (hopefully) add a test today, but can't now, I have other things to do.

[Veykril]

Test can follow whenever then

[davidbarsky]

I’d be happy to add a test: where’s be a good spot?

[ChayimFriedman2]

I think it can live in the top-level ide-completion/src/tests.rs. A proc macro isn't important for this; a macro_rules that wraps the entire thing will work just as well. What's important is that there will be a macro call next to the identifier typed, and that the whole thing will be wrapped in a macro call, so that when we fall back (after we failed to expand) to the original position we can't analyze it.