Effects: partial CPS transform

[vouillon]

We identify functions that don't involve effects by analyzing the call graph and we keep then in direct style. This relies on a global control flow analysis to find which function might be called where.

The analysis is very effective on small / monomorphic programs.

• hamming is somewhat slower since it uses lazy values (we don't analyze mutable values).
• nucleic is faster since the global control flow analysis is used to avoid some slow function calls. This measurement was made before Apply functions: optimizations #1358 was merged. The gap is probably narrower now.
  

The analysis is less effective on large programs. Higher-order functions such as List.iter are turned into CPS and then all functions that calls directly or indirectly such a function needs to be turned into CPS as well. There is also some horizontal contamination, where a function needs to be turned into CPS since it is used in a context which expects a CPS function, and then this impacts all other places it is called. Still, ocamlc is now only about ~10% slower (it is about 60% slower with the released version of Js_of_ocaml).

CAMLboy is less than 25% slower (650 FPS instead of 800 FPS).

The size of the generated code is less than 20% larger, a few percents larger when compressed. For a large Web app, I have a 44% increase of generated code (6% when compressed).

Compiling ocamlc is about 25% slower.

[hhugo]

Effects: use global flow analysis to detect more exact calls doesn't affect any expect tests. I think we should add some.

[hhugo]

I would be nice to be able to propagate information across units during separate compilation.

(related to #550)

[hhugo]

Would global_flow.ml allow to address #594 ?

[vouillon]

Effects: use global flow analysis to detect more exact calls doesn't affect any expect tests. I think we should add some.

I have added a test

[vouillon]

Would global_flow.ml allow to address #594 ?

Yes, we can use it for that. My concern is that it might be a bit expansive.
Maybe we can have a variant where information does not flow through function arguments, which would still address #594 and should be faster.

[vouillon]

@pmwhite Do you want to give it a try?

[hhugo]

I didn't read in details but It look good from there.

The performance wiki page still need to be updated (text and graphs).

I would be nice to show timings improvement for non-benchmark progams.

[pmwhite]

@pmwhite Do you want to give it a try?

Indeed, I would. Likely some time next week.

[kayceesrk]

Thanks for this work. The improvements are great.

IIUC the benchmarks don't use effect handlers. I would also be interested in seeing the improvements in programs that use effect handlers. In the PLDI 21 paper on effect handlers, we studied the performance of effect handlers using two small benchmarks -- chameneos redux (Section 6.3.1) and generators (Section 6.3.2). The source code for the benchmarks is here: https://github.com/kayceesrk/code-snippets/tree/master/eff_bench. I would be interested in seeing the performance difference between --enable=effects on trunk and this PR.

[vouillon]

IIUC the benchmarks don't use effect handlers. I would also be interested in seeing the improvements in programs that use effect handlers. In the PLDI 21 paper on effect handlers, we studied the performance of effect handlers using two small benchmarks -- chameneos redux (Section 6.3.1) and generators (Section 6.3.2). The source code for the benchmarks is here: https://github.com/kayceesrk/code-snippets/tree/master/eff_bench. I would be interested in seeing the performance difference between --enable=effects on trunk and this PR.

This makes a significant difference as well. Here is a quick measurement:

	generator	chameneos (1000 iterations)
master	10.3s	2.3s
this PR	6.6s	1.5s

[kayceesrk]

Thanks for the results. It is good to see partial CPS doing well here.

The next question is harder to answer, because it may be ill informed, but let me ask that anyway. On these benchmarks, how close to perfectly precise / optimal performance is the current partial CPS? As in, if you had a chance to only CPS those functions which are absolutely needed in these benchmarks, what would the performance be?

[vouillon]

The next question is harder to answer, because it may be ill informed, but let me ask that anyway. On these benchmarks, how close to perfectly precise / optimal performance is the current partial CPS? As in, if you had a chance to only CPS those functions which are absolutely needed in these benchmarks, what would the performance be?

I think the code for generator is optimal.

chameneos is using Printf, which in unnecessarily CPS-transformed. By commenting out all printf, it goes down to 1.3s, but I'm not sure how much of the difference is due to just using Printf and how much comes from the CPS transformation. Otherwise, List.map is called both with function MVar.take that needs to be in CPS and with a function that could have been kept in direct style. But that does not make any performance difference.

  let chams = List.map ~f:(fun c -> ref c) colors in
...
  let ns = List.map ~f:MVar.take fs in

[kayceesrk]

Thanks @vouillon for your answer. It helped me put the numbers in perspective.

[hhugo]

Would global_flow.ml allow to address #594 ?

Yes, we can use it for that. My concern is that it might be a bit expansive. Maybe we can have a variant where information does not flow through function arguments, which would still address #594 and should be faster.

Do you expect the analysis to be more expansive when effects is off ?

[hhugo]

@pmwhite Do you want to give it a try?

Indeed, I would. Likely some time next week.

@pmwhite,any news ?

[pmwhite]

@pmwhite Do you want to give it a try?

Indeed, I would. Likely some time next week.

@pmwhite,any news ?

Just tried this patch out. I've run into an issue, I believe with the lexer, which is having trouble parsing column 57 of this line. I assume this has to do with the recent changes to the lexer/parser, and not with this particular PR, but it does block me from testing this PR itself.

[hhugo]

@pmwhite Do you want to give it a try?

Indeed, I would. Likely some time next week.

@pmwhite,any news ?

Just tried this patch out. I've run into an issue, I believe with the lexer, which is having trouble parsing column 57 of this line. I assume this has to do with the recent changes to the lexer/parser, and not with this particular PR, but it does block me from testing this PR itself.

@pmwhite Do you want to give it a try?

Indeed, I would. Likely some time next week.

@pmwhite,any news ?

Just tried this patch out. I've run into an issue, I believe with the lexer, which is having trouble parsing column 57 of this line. I assume this has to do with the recent changes to the lexer/parser, and not with this particular PR, but it does block me from testing this PR itself.

Should be fixed by #1395

[pmwhite]

I've now run into the following error:

> js_of_ocaml.exe -I . -o ./delimited_kernel.cma.js --enable with-js-error --source-map-inline --pretty ./delimited_kernel.cma
Error: Some variables escaped (#1)

This refers, I believe, to this library; hopefully that reproduces easily enough.

[hhugo]

I've now run into the following error:

> js_of_ocaml.exe -I . -o ./delimited_kernel.cma.js --enable with-js-error --source-map-inline --pretty ./delimited_kernel.cma
Error: Some variables escaped (#1)

This refers, I believe, to this library; hopefully that reproduces easily enough.

--debug shortvar should print the generated file on stderr with the problematic varaible of the form<var>

[hhugo]

@vouillon, should we merge ?

[vouillon]

@vouillon, should we merge ?

I still need to update the documentation...

[bnguyenvanyen]

I just ran these benchmarks against three versions of the compiler: "no effects", "partial CPS" which is this PR with effects enabled, and "full CPS", which is a recent-ish commit on master with effects enabled. Here are the results:

Hi, I'm surprised to see in this benchmark that partial CPS is in some cases much slower ? (The median is 0.65 faster than full CPS, but in some cases it's > 4 times slower, meaning 25/30 times slower than no CPS.
Is there an explanation ? Or am I reading the benchmark wrong ?

Here are the tests for which partial is slower:

Dynamic_cells: Focus up and down in 10 element map
Dynamic_cells: Page up and down in 101 element map
Dynamic_cells: Scroll 1-wide window from 0 to 9 and back in 1000 element map
Dynamic_cells: Apply 4 filters and clear with 101 element map using 10 window
Dynamic_cells: Apply 4 filters and clear with 10000 element map using 50 window
Dynamic_cells: Apply 4 filters and clear with 10000 element map using 100 window
Dynamic_cells: Invert ordering of 10 element map twice
Dynamic_cells: Invert ordering of 100 element map twice
Dynamic_cells: Invert ordering of 1000 element map twice
Dynamic_cells: Perform 10 sets of 1 items in a 10 element map with 10-wide window
Dynamic_cells: Perform 10 sets of 5 items in a 10 element map with 10-wide window
Dynamic_cells: Perform 10 sets of 1 items in a 11 element map with 10-wide window
Dynamic_cells: Perform 10 sets of 5 items in a 11 element map with 10-wide window
Dynamic_cells: Perform 10 sets of 1 items in a 100 element map with 10-wide window
Dynamic_cells: Perform 10 sets of 5 items in a 100 element map with 10-wide window
Dynamic_cells: Perform 10 sets of 1 items in a 1000 element map with 10-wide window
Dynamic_cells: Perform 10 sets of 5 items in a 1000 element map with 10-wide window

And a plot:

[hhugo]

The slower tests seems to be at the end of the table. If the order correspond the the order of execution, maybe something happened to the machine while running the tests ....

Another explanation could be that some control flow are exception based and the full cps version would throw less..

[vouillon]

The slower tests seems to be at the end of the table. If the order correspond the the order of execution, maybe something happened to the machine while running the tests ....

That was my hypothesis as well. We should try to reproduce this at some point. Unfortunately, compiling this benchmarks is a bit complicated when you are not at Jane Street...