Apply functions: optimizations

[vouillon]

Getting the arity of a function is slow with v8. On Chrome, function p is about ten times slower than function q:

function f (x) {}
f.l = 1;
var s = 0;
function p() {for (i = 0; i < 100000000; i++) { ((f.length)==1?s+=1:0)} }
function q(){for (i = 0; i < 100000000; i++) { ((f.l)==1?s+=1:0)}}
p(); q()

node takes 3.1s to execute this piece of code without this optimization and 1.3s with it (and 0,8s when manually removing the call to caml_call1).

let l = List.of_seq (Seq.init 10000 (fun i -> i))
let s = ref 0
let f x = s := !s + x
let iter f  = for i = 1 to 10000 do List.iter f l; s := 0; List.iter f l done
let () = iter f

I have known about this issue for a long time. I was considering wrapping functions inside an object: {arity:2,fun:function(...){..}}, with some optimization for when the function is statically known. But that would had been a large change with some impact on the generated code size and of lot of added complexity in the Js_of_ocaml compiler. This change seems to address the issue at a very low cost.

Related issue: #1246

[hhugo]

Is this draft on purpose ? what's the plan ?

[pmwhite]

f.length is slow with v8, so we cache the function arity in another property

Also, I'd be very interested in seeing some documentation of this aspect of v8. How do you know f.length is slow?

[vouillon]

Is this draft on purpose ? what's the plan ?

I would like to do more benchmarking.

[vouillon]

Also, I'd be very interested in seeing some documentation of this aspect of v8. How do you know f.length is slow?

I have not seen any documentation on this. I just noticed that somehow. This was also reported in #1246.

[TyOverby]

I pulled this change into our repo and saw a huge improvement for microbenchmarks (50% runtime reduction in some cases) but much smaller (2% - 3%) improvement in our benchmarks that attempt to replicate real-world applications and workloads. I suspect that this is because the fallback case in caml_call_gen returns a function that has a length of 0, which means that partially-applied functions don't benefit from length caching.

I suspect that if caml_callgen checked to see how many arguments are left unapplied and returned functions specialized to that size, then the the arity-cache proposed here would be hit far more often.

[TyOverby]

partially-applied functions don't benefit from length caching

This is because if f.l == 0 then the first argument to the || operator in f.l || f.l = f.length will evaluate to false, causing the right-hand-side of the operator to be evaluated unconditionally.

[hhugo]

not sure about the performance, but one could use a shorter sytnax

f.l >= 0 ? f.t : f.t=f.lenght

instead of

f.l === underfined ? f.l = f.lenght : f.l

[hhugo]

caml_js_function_arity should probably be updated as well to use (and set ?) the cache.

[hhugo]

function caml_call_gen(f, args) {
  if(f.fun)
    return caml_call_gen(f.fun, args);
  //FIXME, can happen with too many arguments
  if(typeof f !== "function") return f;
  var n = f.length | 0;
...

Shouldn't we also use the cache for computing n ?

[TyOverby]

I pulled this PR in for prototyping and added @hhugo's suggestion

-  var n = f.length | 0;
+  var n = (f.l >= 0 ? f.l : f.l = f.length) | 0;

And the benchmarks I mentioned

benchmarks that attempt to replicate real-world applications and workloads saw smaller (2% - 3%) improvements

saw improvements in the 10% to 30% range!

[vouillon]

I pulled this PR in for prototyping and added @hhugo's suggestion

-  var n = f.length | 0;
+  var n = (f.l >= 0 ? f.l : f.l = f.length) | 0;

And the benchmarks I mentioned

benchmarks that attempt to replicate real-world applications and workloads saw smaller (2% - 3%) improvements

saw improvements in the 10% to 30% range!

@TyOverby Thanks a lot for your tests! I'm surprised this makes that much of a difference since this is on a slow path. Maybe that's the optimization for partially applied functions (33c3af8) that makes such a difference?

[TyOverby]

I'm surprised this makes that much of a difference since this is on a slow path. Maybe that's the optimization for partially applied functions (33c3af8) that makes such a difference?

I totally agree that it's almost all due to returning functions with specialized arity when partially applied. I just wanted to mention that I cherry-picked a change that wasn't already in the PR.

[TyOverby]

I traced a few apps and found this distribution for the caml_call{n} functions

n	occurances
1	109
2	300
3	30
4	4
5	<1

Based on this, I think I'm happy with only building specialized partially-applied functions for the the cases where only 1 or 2 args remain.

[hhugo]

@vouillon, should we move this PR forward ?

[hhugo]

I've rebased the PR, fix tests and rewrote history