Combine completion candidates with metadata regarding them

[bbatsov]

Emacs has a feature that allows you to annotate a completion candidate with addition information - e.g. the type of the candidate (function, macro, interface, etc). You can see the feature in action for Emacs Lisp completions. For this to effectively used, however, it'd be beneficial if one could retrieve a completion candidate together with some metadata regarding it (perhaps the type and the ns/package of the candidate, maybe others make sense as well), so avoid having to do one of two things:

• retrieve the metadata for each candidate individually (which is slow)
• retrieve all candidates and then do a second function call to retrieve the metadata for all of them

Obviously this is not a must-have feature, but it'd be nice to have something like this in tools like CIDER. I'm also not quite sure if packages like compliment should handle this or their clients, but I wanted to start a discussion on the subject.

[alexander-yakushev]

I know about this feature, ac-nrepl used that by calling the backend numerous times to retrieve candidates for each category. I didn't know how to preserve the functionality while reducing the number of calls to one, so I just gave up on that.

I'd be happy to bring it back if it is possible to do without hampering the performance. Perhaps the backend could return candidates in groups, according to their category. Although I'm not sure how then you can use it in ac (ac requires a new source per category which means a separate request per source — or doing some unreliable caching magic).

Also, what about the ns/package of the candidate? It is already given out in the popup doc that is requested per candidate. Is it important to have it in completion, for all candidates at once?

[bbatsov]

I'd be happy to bring it back if it is possible to do without hampering the performance. Perhaps the backend could return candidates in groups, according to their category. Although I'm not sure how then you can use it in ac (ac requires a new source per category which means a separate request per source — or doing some unreliable caching magic).

I think that only the nREPL calls are expensive and doing some type checks in Java should be fairly fast. Once a client receives the candidates and the metadata grouping them on the client side should be pretty fast as well. @gtrak suggested returning a list of maps for the candidates in a similar issue for ClojureScript completion. This seems like a good idea to me. Initially the map could have only two elements - the name and the type of the candidate.

Also, what about the ns/package of the candidate? It is already given out in the popup doc that is requested per candidate. Is it important to have it in completion, for all candidates at once?

I just mentioned this as some other metadata we might need (theoretically) beside the candidate's type.

[alexander-yakushev]

I think that only the nREPL calls are expensive and doing some type checks in Java should be fairly fast. Once a client receives the candidates and the metadata grouping them on the client side should be pretty fast as well. @gtrak suggested returning a list of maps for the candidates in a similar issue for ClojureScript completion. This seems like a good idea to me. Initially the map could have only two elements - the name and the type of the candidate.

Yes. What I'm saying is that the only way (at least that I'm aware about) in auto-complete to assign a letter for a candidate is to create a separate source for that letter. So that you have a source for vars, a source for namespaces, a source for functions etc. Now every source has its individual retrieval function that should return the list of completions of that type. How would you multiplex all those calls into single nREPL request? By saving the results of a single request into some local state, so that all the sources' retrieval functions will take candidates from that saved state? Seems pretty hackish to me.

[bbatsov]

Don't know about auto-complete, but if you're using completion-at-point every candidate is simply passed to an annotation function with adds the extra data for it (here's an example). This works transparently for company-mode and guess auto-complete-mode should start respecting completion-at-point instead of defining its own abstractions.

[alexander-yakushev]

auto-complete-mode should start respecting completion-at-point instead of defining its own abstractions.

I'm afraid you are addressing the wrong person with this claim:).

[bbatsov]

@alexander-yakushev So, shall we do this or no? People keep asking me about showing candidate type data for a while now and I have to know whether I'll have to do in the middleware or you'll do it in compliment.

[alexander-yakushev]

Yeah, perhaps this needs to be done after all. So, does it mean instead of "reduce-kv" to return {:name "reduce-kv" :type :var :namespace "clojure.core"}?

Also, in company mode do you plan just to attach the type tag to candidate name? How will it know not to insert that tag into buffer when completing?

[bbatsov]

Also, in company mode do you plan just to attach the type tag to candidate name? How will it know not to insert that tag into buffer when completing?

There's some standard Emacs support for this. See https://github.com/clojure-emacs/cider/blob/master/cider-interaction.el#L798

[alexander-yakushev]

OK, it will be easier just to "Hulk smash me write, Bozhidar comment". ETA is tomorrow.

[bbatsov]

:-) Guess so.

[bbatsov]

Ping :-)

[alexander-yakushev]

OK, I need some more time for this. I haven't yet figured the correct way to do this to preserve compatibility with the old method, and to keep the code nice. But I'll get to it soon.