IntelliJ becomes slow when using nest anonymous functions

[mkarnebeek]

Versions

macOS

10.12.2

Elixir

1.3.4

IntelliJ

IntelliJ IDEA 2016.3.2
Build #IU-163.10154.41, built on December 21, 2016
JRE: 1.8.0_112-release-408-b6 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o

intellij-elixir

4.7.0, zip file from Releases tab.

Reproduce steps

1. Create a new project
2. Create a new file, name it "test.ex" in the root of the project
3. Paste the code below
4. Move around the document using the cursor keys, using the "Enter", "Tab" and "Backspace" keys at various places. For example: Stand at line 5 and press "Enter" a few times

Code:

def a() do
  fn ->
    fn ->
      fn ->
        fn ->
          fn ->
            fn ->
              fn ->
              end
            end
          end
        end
      end
    end
  end
end

Problem

Responsiveness to the "Enter" key.

[KronicDeth]

It's a side-effect of having to do a negative look-ahead to stop the current stab operation when the next one starts and that this look-ahead looks like a normal expression due to no parentheses clause heads. There's no separator for stab operations unlike Erlang, which has ;, so the way Elixir does it is that the current clause stops when the next stab operation begins.

To do that in the plugin's grammar I have to do a negative look-ahead, !stabOperationPrefix, which is slow and costly.

private stabBodyExpression ::= !stabOperationPrefix expression

--

intellij-elixir/src/org/elixir_lang/Elixir.bnf

Line 2239 in 825344d

private stabBodyExpression ::= !stabOperationPrefix expression

private stabOperationPrefix ::= stabParenthesesSignature stabInfixOperator |
                                stabNoParenthesesSignature stabInfixOperator |
                                stabInfixOperation

--

intellij-elixir/src/org/elixir_lang/Elixir.bnf

Lines 2236 to 2238 in 825344d

	private stabOperationPrefix ::= stabParenthesesSignature stabInfixOperator |
	stabNoParenthesesSignature stabInfixOperator |
	stabInfixOperator

The native parser doesn't have to (and I don't think can because it uses yecc) do a negative look-ahead because it's format allows it to cheat: it allows the implementation to rearrange how the parse tree is returned

access_expr -> open_paren stab close_paren : build_stab(reverse('$2')).
access_expr -> open_paren stab ';' close_paren : build_stab(reverse('$2')).
access_expr -> open_paren ';' stab ';' close_paren : build_stab(reverse('$3')).
access_expr -> open_paren ';' stab close_paren : build_stab(reverse('$3')).
access_expr -> open_paren ';' close_paren : build_stab([]).

-- https://github.com/elixir-lang/elixir/blob/2a1f38fbab468a5f4330d56845fc1c21543a570b/lib/elixir/src/elixir_parser.yrl#L241-L245

build_stab([{'->', Meta, [Left, Right]} | T]) ->
  build_stab(Meta, T, Left, [Right], []);

build_stab(Else) ->
  build_block(Else).

build_stab(Old, [{'->', New, [Left, Right]} | T], Marker, Temp, Acc) ->
  H = {'->', Old, [Marker, build_block(reverse(Temp))]},
  build_stab(New, T, Left, [Right], [H | Acc]);

build_stab(Meta, [H | T], Marker, Temp, Acc) ->
  build_stab(Meta, T, Marker, [H | Temp], Acc);

build_stab(Meta, [], Marker, Temp, Acc) ->
  H = {'->', Meta, [Marker, build_block(reverse(Temp))]},
  reverse([H | Acc]).

https://github.com/elixir-lang/elixir/blob/2a1f38fbab468a5f4330d56845fc1c21543a570b/lib/elixir/src/elixir_parser.yrl#L754-L769

I've posted a question to the OpenAPI forums, so maybe someone at JetBrains can explain a better way to write the grammar.

[mkarnebeek]

Cool, tnx for taking a look!

[mkarnebeek]

Any update on this?

[KronicDeth]

JetBrains responded, but I didn't really find it that helpful. Too general. I've asked some more questions on one of the approaches today to see if I can get more concrete answers.

Just because I can find why it's slow does not mean I know how to fix it. This is parser design and optimization and I am barely good at grammar design (look at the first year of the git log). Feel free to read up on GrammarKit (the parser generator the plugin is using), Pratt and LL parsers to see if I missed some feature of GrammarKit that will make this faster.