Forwarding functions

[eernstg]

I think we need a notion of a forwarding function which will faithfully invoke another function, as a semantic primitive in Dart. That is, we should specify once and for all what this is, and then we can use it in several different situations.

• One such situation is when a redirecting factory constructor is torn off, see Constructor tear-off of forwarding factory constructors is under-defined. #3427.
• Another use case is noSuchMethod forwarders, see Don't invent a default value when no well-defined value is available #3331.

In each of these cases, a forwarding function with the semantics described here would solve some gnarly issues concerning default values. Moreover, explicitly declared forwarding functions can be useful in their own right as a modeling device: In some cases it is simply natural and convenient for an object to provide a service by forwarding the method invocation to some other object, and in those cases it will also be useful to know that there is no problem with default values.

Here are some ideas that we could use to get started.

Such forwarding functions can't be written in plain Dart without whole-program information, because a function could be a forwarder to an instance method, and the forwarding invocation would then have to be written using global information about which overriding concrete declarations the statically known method has, and which default values they are using for each optional parameter.

The type of a forwarding function is specified as a function type.

In particular, it does not specify any default values for any optional parameters: Invocations will use the default values specified by the forwardee, not the forwarder.

Statically, a forwarding function f that forwards to a function g must have a type which is a supertype of the type of g.

We could think about this as a kind of an interface/implementation relationship where the forwarder is similar to an interface element (e.g., an abstract instance method declaration in a class), and the forwardee is similar to the concrete method which is actually invoked at run time.

Note that the type of f can be a proper supertype of the type of g. For example, f can have stronger constraints on the parameters (e.g., if f has type Object Function([int]) and g is bool g([num n = 3.14])) => e). The fact that g can have default values that f couldn't have if written as a regular function declaration (because they would be a type error) is another reason why we need a primitive mechanism to express forwarding functions.

The forwarding function should satisfy a rather simple requirement at run time:

If f is a forwarding function that forwards to a library function, a static method, or a local function g then the effect of executing f<typeArgs>(valueArgs) is the same as the effect of executing g<typeArgs>(valueArgs).

If f is a forwarding function that forwards to an instance method m with a receiver o, the effect of executing f<typeArgs>(valueArgs) is the same as the effect of executing o.m<typeArgs>(valueArgs).

It follows that the effect of performing a generic function instantiation (f<typeArgs>) yields a function object that behaves the same as the generic instantiation of the forwardee function (g<typeArgs> or o.g<typeArgs>, respectively).

We could consider allowing a given forwarding function object to rebind the forwardee to a different function. We could also consider allowing a forwarding function to forward to a function of type dynamic or Function. However, let's start with the simple and well-understood case where the forwardee has a statically known signature (for the library/static/local case), or it is a correct override of a statically known instance member signature.

We could consider forwarding to a getter. This seems to be unnecessary, because there is nothing wrong with a hand-written () => o.myGetter, that's not a task that requires semantics that we can't express.

We could consider equality of forwarding functions: Forwarders to the same library/static function could be equal if it is the same function and they have the same function type. Forwarders to local functions would only be equal if identical (two distinct forwarders could forward to local functions with different run-time environments). Finally, forwarders to instance methods could be equal if they have the same (identical) receiver object as well as the same member name, the same statically known declaration of that member name, and the same function type.

As long as we only wish to use this concept to clarify the semantics of torn-off redirecting factory constructors and noSuchMethod forwarders, there is no need to have syntax for it.

However, it seems very likely that we'd want to use this mechanism in a broader set of cases. So here's a possible syntax, just to have something concrete to talk about:

Object f1([int]) ==> g1; // Or `==> A.g2`.

bool g1([num n = 3.14]) => n > 0;

class A {
  static Object f2([int]) ==> g2; // Or `==> g1;`.
  static bool g2([num n = 3.14]) => n > 0;

  void foo() {
    void g3([num n = 3.14]) => print(n > 0);
    void Function(int) f3 = someCondition ? (int) ==> g1 : (int) ==> g3;
    [1, 2, 3].forEach(f3);
  }
}

class B {
  bool g4([num n = 3.14]) => n > 0;
}

class C implements B {
  final String s;
  C(this.s);

  bool g4([num? n = 5.25]) => n > 10;
  Object superG4(int) ==> super.g4;
  superG4 ==> super.g4; // Use the function type of `super.g4`.

  // Forwarding to a different object.
  mySubstring ==> s.substring;
  substring ==> s; // Just specify receiver to reuse the member name.
}

void main() {
  B b = C();
  [0, 10, 100].forEach((int) ==> b.g4);
}

In some cases we might want to change the signature of a function slightly when we create a forwarding function. We could use syntactic marker (I'll use an ellipsis, just to have something concrete) to specify that the forwarding function has the same named parameter declarations as the forwardee, except for the ones that we've mentioned.

Let's say we start with this one:

extension on BuildContext {
  TextStyle textStyleWith({
    bool? inherit,
    required Color color,
    Color? backgroundColor,
    double? fontSize,
    FontWeight? fontWeight,
    FontStyle? fontStyle,
    double? letterSpacing,
    double? wordSpacing,
    TextBaseline? textBaseline,
    double? height,
    TextLeadingDistribution? leadingDistribution,
    Locale? locale,
    Paint? foreground,
    Paint? background,
    List<Shadow>? shadows,
    List<FontFeature>? fontFeatures,
    List<FontVariation>? fontVariations,
    TextDecoration? decoration,
    Color? decorationColor,
    TextDecorationStyle? decorationStyle,
    double? decorationThickness,
    String? debugLabel,
    String? fontFamily,
    List<String>? fontFamilyFallback,
    String? package,
    TextOverflow? overflow,
  }) =>
      CupertinoTheme.of(this).textTheme.textStyle.copyWith(
            inherit: inherit,
            color: color, // Declared as `Color? color`.
            backgroundColor: backgroundColor,
            fontSize: fontSize,
            fontWeight: fontWeight,
            fontStyle: fontStyle,
            letterSpacing: letterSpacing,
            wordSpacing: wordSpacing,
            textBaseline: textBaseline,
            height: height,
            leadingDistribution: leadingDistribution,
            locale: locale,
            foreground: foreground,
            background: background,
            shadows: shadows,
            fontFeatures: fontFeatures,
            fontVariations: fontVariations,
            decoration: decoration,
            decorationColor: decorationColor,
            decorationStyle: decorationStyle,
            decorationThickness: decorationThickness,
            debugLabel: debugLabel,
            fontFamily: fontFamily,
            fontFamilyFallback: fontFamilyFallback,
            package: package,
            overflow: overflow,
          );
}

We could then reduce the verbosity by using a forwarding function declaration and specify that most of the parameter list is the same in the forwardee and the forwarder. As before, we get the return type from the forwardee when it is not specified:

extension on BuildContext {
  textStyleWith({required Color color, ...}) ==> CupertinoTheme.of(this).textTheme.textStyle.copyWith;
}

A partial specification of positional parameters would be possible as well: An ellipsis in the forwarding function parameter list after one or more positional parameters would stand for a copy of the positional parameters of the forwardee with a position that is higher:

num add(num x, num y) => x + y;
forwardToAdd(int, ...) ==> add;

It would be a compile-time error if the forwardee and the forwarder disagree on whether each of those positional parameters is optional.

[Wdestroier]

I would like to suggest the

bool compare(String a, String b) = super.compare;

syntax, which is shown twice in #3427, instead of

bool compare(String a, String b) ==> super.compare;

Kotlin's single-expression functions (equivalent to Dart's arrow functions) use an equal sign:

fun compare(a: String, b: String): Boolean = a.length < b.length

I have always found => more intuitive for such, while = seems more suitable for redirection or delegation to another function.

[eernstg]

@Wdestroier, I did consider using =, to continue the trend introduced by redirecting factory constructors. However, I tend to think that a more specific token like ==> is useful for the overall readability of the code, and in order to avoid parsing ambiguities.

In any case, by all means let's look at it, perhaps it grows on us:

Object f1([int]) = g1; // Or `= A.g2`. We could do this.

bool g1([num n = 3.14]) => n > 0;

class A {
  static Object f2([int]) = g2; // Or `= g1;`. We could do this, too.
  static bool g2([num n = 3.14]) => n > 0; 

  void foo() {
    void g3([num n = 3.14]) => print(n > 0);
    void Function(int) f3 = someCondition ? (int) = g1 : (int) = g3; // Confusing to read?
    [1, 2, 3].forEach(f3);
  }
}

class B {
  bool g4([num n = 3.14]) => n > 0;
}

class C implements B {
  final String s;
  C(this.s);

  bool g4([num? n = 5.25]) => n > 10;
  Object superG4(int) = super.g4; // We can do this, too.
  superG4 = super.g4; // Confusing to read?

  // Forwarding to a different object.
  mySubstring = s.substring; // Confusing?
  substring = s; // Confusing again?
}

void main() {
  B b = C();
  [0, 10, 100].forEach((int) = b.g4); // Possible, but perhaps confusing?
}

[Wdestroier]

I tend to think that a more specific token like ==> is useful for the overall readability of the code, and in order to avoid parsing ambiguities.

Thank you, I agree.

[rubenferreira97]

I really like this feature; however, the following feels off to me (maybe I am not used to it):

void Function(int) f3 = someCondition ? (int) = g1 : (int) = g3;

[0, 10, 100].forEach((int) = b.g4);

If possible I would write them as:

void Function(int) f3 = someCondition ? g1 : g3;
// or using ==>
void Function(int) f3 ==> someCondition ? g1 : g3;

// and

[0, 10, 100].forEach(b.g4);

@eernstg Are there cases where we explicitily need to write a forwarding function in the form (paramTypes) = forwardedFunction/ (paramTypes) ==> forwardedFunction? Couldn't we just write forwardedFunction (as long as it's assignable)?

[eernstg]

Couldn't we just write forwardedFunction (as long as it's assignable)?

Sure, that would be sufficient, and we would always have the assignability as long as the forwarding function is required to have a type which is a supertype of the forwardee.

The main use case would probably be that you want to enforce that supertype: If _g is a function which is declared in your library L, and you want to make it accessible to developers outside L, but you want to make sure they don't pass a specific optional argument then you can provide a function that forwards to _g and doesn't have that optional argument. Or, conversely, you could force callers to provide a specific argument by turning that parameter into a non-optional one:

const _secret = MyType();

void _g1({MyType myType = _secret}) {...}
void _g2(int arg1, [int arg2 = 42]) {...}

void foo(void Function(Function) callback) {
   ...
  // They can call `_g1`, but they can't use the default value.
  callback({required MyType myType}) ==> _g1); 
  ...
  // They can call `_g2`, but we're guaranteed that these calls
  // will use the default value `42` as the second argument.
  callback((int arg1) ==> _g2);
  ...
}

Being lazy, I just used the type Function for the function objects that are forwarding to _g1 and _g2, but the point is also that even dynamic calls can't break those rules: With _g1 they can't use the _secret default value, and with _g2 they cant pass the argument at all.

So the forwarding function literal might not be a very common construct, but it still seems potentially useful to me.

[FMorschel]

Somewhat related to #4159.

[tatumizer]

Forwarding is just a degenerate case of a more general problem of creating wrapper functions. The difference is that the wrapper can add or remove some parameters to/from the list of original parameters, and can call the target explicitly in its body.

Here's a raw idea for discussion. Let g be a function. We can refer to the list of its parameters as g.* in any context where this makes sense. E.g. we can write

typedef G=(g.*); // record type
void f(g.*) => g(...); // 3 dots mean: take parameters from the context
void f1(g.*) {
  print('HELLO');
  g(...); // same meaning as above
  print('BYE');
}

To override parameters in the parameter list, we must use a special syntax, e.g.

void f({int x! =42}, g.*) => g(...);

Here, x overrides the parameter x coming from g, with a different default value.
Adding more parameters is straightforward: just add them:

void f(int zzz, g.*) {
   // code using zzz
   g(...);
}

To override parameters while forwarding, use syntax

g(y!:15, ...); // pass a different value of y

One thing I haven't figured out yet is how to remove parameters from g.*. Maybe something like

void f(g.* hide x) {
  g(x!:0, ...);
}

[eernstg]

Here's a raw idea for discussion

Interesting! I thought about similar things, and it would indeed be tempting to enhance the mechanism such that it is possible to modify actual argument list in the "near-forwarding" invocation.

The catch is that we might need to have a huge amount of mechanism in order to allow for enough flexibility to handle the cases that arise in practice. Simple mechanisms might look good in examples, but there's going to be a steep cliff to jump off it it only works in the cases that we've considered when this mechanism was designed, and it may or may not work for real cases.

[TekExplorer]

I feel like straight forwarding is best done through the existing forwarding syntax present in factories;

Especially since it should be relatively trivial to implement given the existing factory code.

That's one thing.

Otherwise is getting the argument list itself in record form, and pass it straight in, where it automatically spreads. Ie, single-argument functions that contain a record are equivalent to an argument list with those fields

Ie: void func(({int? a, String? c}) args); called with func(a: 3, c: 'hi');

Then, we just use record type aliases to propagate them.

we would then get:

typedef Args = (String key, {bool? flag});
void func(Args args) => Somewhere.realFunc(args);
// Shorthand. Could be useful for other things
void func(Args args) = Somewhere.realFunc;


func('something', flag: true);

Then, if we want to exclude or add parameters, we would need static record type operations, which can be extremely useful in other places.

[tatumizer]

More examples:

typedef A=({int a, String b});
typedef B=(...A.*); 
typedef B=(...A.(* hide b));
typedef B=(int c, ...A.(* hide b));
typedef B=(...A.(* show b));
func(...A.*) {/*body*/}
func(...A.(* hide a)) {/*body*/}
// etc.,
class C {
  final int a;
  String b;
  C(...this.*);
  C.n1(...this.(* show a, b));
  C.n2(...C.new.(* show a, b)): this(...);  
}
class D extends C {
  double d;
  // etc.
  D(...this.*, ...super.new.*): super(...);
  D.id(...this.*, ...C.new.*): super(...);
}

[eernstg]

@TekExplorer wrote, about plain forwarding:

it should be relatively trivial to implement given the existing factory code.

I don't think it will be a huge implementation effort, but those two things are actually quite different. Redirecting factory constructors are resolved during compilation, so that's basically an inlining mechanism. A forwarding function invocation needs a bit of magic in order to preserve the semantics of default values of parameters:

abstract class A {
  int foo([int i]);
}

class B1 extends A {
  int foo([int i = 1]) => i;
}

class B2 extends A {
  int foo([int i = 2]) => i;
}

class C {
  A a;
  C(this.a);
  foo ==> a;
}

void main() {
  var c1 = C(B1()), c2 = C(B2());
  print('Should be 1: ${c1.foo()}, should be 2: ${c2.foo()}');
}

The approach to forwarding that one might take at first would be to say that foo ==> a; is simply an abbreviated notation for the declaration int foo([int i = k]) => a.foo(i);, where k is the default value.

However, there is no way to choose a value for k which will work: According to the invocation c1.foo(), the value must be 1, and according to the invocation c2.foo(), the value must be 2. Next, C.a is mutable so we can't even use the same default value for a given instance of C at different times, because it may need to be 1 at one occasion and 2 at some other occasion.

So we can't write a true forwarding function in Dart when the forwardee has one or more parameter default values, we need to have some extra mechanism. This mechanism could be support for detecting whether or not a given optional parameter has actually been passed (independently of the default value, that is, we can't just check whether it's null).

We had such a mechanism many years ago, and it was removed for a whole bunch of reasons, so it isn't likely to be re-added.

However, let's just consider how it could be used. We'll use the syntax passed(v) to denote a boolean expression which will be true if the formal parameter v was passed, and false otherwise. The class C might then be desugared as follows:

class C {
  A a;
  C(this.a);
  int foo([int i = k]) => passed(i) ? a.foo(i) : a.foo();
}

However, this won't scale up very well if we're dealing with a function that accepts, say, 10 optional named parameters, because it will have to spell out all 1024 possible invocations of the forwardee. We aren't going to have to write that huge expression with 1024 different invocations manually, it will be generated by the compiler, but it may still cause the program size to blow up.

So we'd actually want a different kind of "magic". One possibility is that the runtime introduces a new "universal object" which can be passed everywhere (it's like an instance of Never). Such an object could be passed in order to indicate that no actual argument was passed for that parameter, and we could then use the default value (in the body of the forwardee, not the forwarder), because that's the location where the actual default value is known (statically, even).

This approach would only have a linear cost (so 10 optional named parameters would be handled in 10 steps, not 1024 steps).

In any case, this illustrates why forwarding in Dart isn't trivial. ;-)

Otherwise is getting the argument list itself in record form

That's an interesting idea, and the language team has already had discussions about possible features whereby actual argument lists to functions could be considered equivalent to record values.

However, it's a non-starter to require that all function calls will receive their actual arguments as a record, and all the existing call sites will implicitly create such a record and pass it. That is simply prohibitively expensive in terms of run-time performance.

The reason why this perspective is relevant is that we can't predict which functions/methods will be the target of a forwarding invocation, which could mean that all functions and methods must work in such a way that a forwarding invocation is possible. We can't allow significant slowdowns of such a basic thing as function/method invocations in order to support a new feature, no matter how nice that feature might be.

Another reason why it may be expensive to pass an actual argument list as a record value is that record values don't have optional parameters (positional or named). This means that a function that accepts 10 optional named parameters will have to be prepared to accept 1024 different types of record, representing each of the possible combinations of passed/non-passed parameters. It might have to perform 1024 type tests on the actual argument record before it finds the one that has the right combination of present/absent arguments, and it would then need to assign the default values to the remaining parameters. That doesn't scale well, either.

So this argument-list-is-record idea is worth exploring further, but it wouldn't go anywhere if it degrades the run-time performance of all programs.

[eernstg]

@tatumizer wrote:

More examples:

Interesting! It looks like you're exploring a mechanism that would allow us to abstract over signatures or at least over formal parameter lists (so we wouldn't have to duplicate a list of 123 named parameters with all their associated information, we could just refer to that list as a declared entity).

This could clearly be more expressive. The mechanism I've proposed here allows for the formal parameter list to be (1) written explicitly (no abstraction, full expressive power available, it just has to be a parameter-wise subtype such that the forwarding call is guaranteed to be type correct). It also allows the formal parameter list to be (2) omitted entirely, in which case we use the formal parameter list from the forwardee (this may include private types from other libraries, so it's not just a textual copy of the formal parameter list). Similarly for the type parameter list.

It's worth keeping in mind that a forwarding (or a more general wrapping) mechanism could be made more flexible and powerful by means of this kind of abstraction. As usual, we'd need to find a good balance between the amount of machinery we're introducing and the amount of benefit that developers will gain from this machinery in practice.

[lrhn]

We had such a mechanism many years ago, and it was removed for a whole bunch of reasons, so it isn't likely to be re-added

The main reason was that it didn't prevent the exponential blowup for multiple optional named parameters with default values.
Instead it made it worse, because now the called function could distinguish no argument from an explicitly passed default value.

The feature itself is not the problem, the problem is not being able to forward a lack of argument using only a linear amount of code.

Argument elements would solve that.
If you can write

void foo({int v1, int v2, ... int vn}) =>
  bar(v1: if (passed(v1)) v1, …
         vn: if (passed(vn)) vn);

such that no v4 argument is passed to bar of none was passed to foo, then that problem goes away.

I'd be fine with having both operations. It's having only one that causes a problem.

[tatumizer]

@eernstg wrote:

As usual, we'd need to find a good balance between the amount of machinery we're introducing and the amount of benefit that developers will gain from this machinery in practice.

The new syntax is localized to the places where we have to either declare or to pass a list of parameters, so hopefully you won't need too much machinery (not sure).

On the plus side, the device turns the list of parameters into a kind of "mathematical object" with a small number of operations defined on it. There's something satisfying in having such a concept in a language - the users will appreciate it.

[TekExplorer]

@TekExplorer wrote, about plain forwarding:

it should be relatively trivial to implement given the existing factory code.

I don't think it will be a huge implementation effort, but those two things are actually quite different. Redirecting factory constructors are resolved during compilation, so that's basically an inlining mechanism. A forwarding function invocation needs a bit of magic in order to preserve the semantics of default values of parameters:

abstract class A {
  int foo([int i]);
}

class B1 extends A {
  int foo([int i = 1]) => i;
}

class B2 extends A {
  int foo([int i = 2]) => i;
}

class C {
  A a;
  C(this.a);
  foo ==> a;
}

void main() {
  var c1 = C(B1()), c2 = C(B2());
  print('Should be 1: ${c1.foo()}, should be 2: ${c2.foo()}');
}

You seem to forget that redurecting factories can't have default values because the real function defines them

That syntax is also... Kinda awful. At that point just do late final foo = a.foo;

It doesn't even look like a method, and the arrow syntax is being misused here I think.

Of course, we can't do that with methods, which is why we need proper redirection at all, otherwise I would just stick to the record type as Args issue, plus the record type operations issue for flexibility.

[TekExplorer]

I also don't see any reason why argument records would be any more expensive than what we have now, since it's not like we can do anything (directly) with records anyway. Plus, what does the argument list look like in memory in the first place? Could it just... Be a record?

Especially since records were made to look like argument lists.

We're just missing default and optional values, which I'm sure we can come up with a solution for.
Perhaps an Arguments type that's a subtype of Record that lets you define optional values? Could only be used as a parameter, otherwise it's treated as the record type ignoring those extra details? Just a thought. (Could be more akin to a typedef with extra details relevant only to the argument list)

[TekExplorer]

I really don't like the idea of a passed() function - it implies that parameters hold additional hidden data, when right now they're just variables defined in an unusual place, and just... Feels bad. What happens if you pass a normal variable into it? Would it detect uninitialized late values? It feels like reflection to me.

I would much rather it be some kind of coercion - as a straw man: foo({Option<Bar?> bar = None()})

Of course, with that literal example you would need to wrap the value in a Some, but perhaps with implicit constructors or some kind of coercion, it can sort of go away.

It honestly kind of sucks that we couldn't have used that for null from the get go, and get associated useful language features to make it great the way we do now with null, but I understand that that would have been way more complicated of a transition.

I do like the conditional passing as an independent feature for the sake of preserving default values when desired.

[tatumizer]

I'd rather have a pseudo-method default defined only for parameters, with two variants: p.default(value) and p.default() for no default. the latter would only be available in collection literals and argument lists (it cannot be a part of any expression).

it honestly kind of sucks that we couldn't have used that for null from the get go

Programming without the value null would be utterly impossible, or at least very inconvenient: it would have forced users to invent ad-hoc alternatives in the cases where they really need nulls - e.g. to indicate that the data is irrelevant or not applicable or missing or something. You may have an Address struct for an address, and a Person may have streetAddress and businessAddress with the possible value of null indicating the absence of such. There are many uses for nulls other than as indicators of a missing parameter.

NOTE: with the proposed syntax of redirection like f(...g.*)=>g(...), the parameters get passed behind the scenes without being observed by the caller.

[lrhn]

Plus, what does the argument list look like in memory in the first place?

Most likely it's a sequence of slots on the machine stack. Maybe including a count of arguments and a mapping from names to positions (but optimized calls can perhaps resolve some names statically. Dynamic invocations are the ones that really need to have all the structure provided at runtime.

Would it detect uninitialized late values?

That is one option that I'm pretty sure I've suggested somewhere.

Allow parameters to be late. A late optional parameter doesn't need a default value, it'll be uninitialized instead.
Then there is an operator to check if a late variable is initialized or not, which also promotes it to definitely assigned on the true branch. It works for both parameters and other late variables, which do have extra state to remember whether they're assigned or not.
(And then parameter elements again, to also be able to not pass an argument programmatically.)

There are many ways to solve this issue. I like the ones that do more than just solve one issue.

[TekExplorer]

it honestly kind of sucks that we couldn't have used that for null from the get go

Programming without the value null would be utterly impossible, or at least very inconvenient: it would have forced users to invent ad-hoc alternatives in the cases where they really need nulls - e.g. to indicate that the data is irrelevant or not applicable or missing or something. You may have an Address struct for an address, and a Person may have streetAddress and businessAddress with the possible value of null indicating the absence of such. There are many uses for nulls other than as indicators of a missing parameter.

Incorrect. It would have been very possible to exist without null - languages like Rust do just fine without it.

The Option monad is pretty neat. Of course, since dart does not have such a monad built in with special treatment (and something tells me thats not likely to change) we would then need a more general language feature that just so happens to suite our purposes.

The reason such a monad would be useful, is because unlike nullable types, Option<> is just a generic, so it can be nested as desired, unlike nullables which cannot.

It's easy to do Option<Option<T>>, but you can't do T?? (Ie for a copy with for a nullable field)

[tatumizer]

@lrhn: the syntax of bulk-declaration of parameters makes it easy to incorporate a late qualifier:

class B extends A {
   B(late A.new.*): super(...);
}

This means: copy all parameters of A.new constructor into the parameter list of B; while copying, transform every optional parameter of A.new into late optional parameter; forward everything as is: if parameter is omitted while calling B, it will still be omitted while forwarding to the super constructor. Since it's not observed by B, its default value in A is of no interest to B.

The same is true when you want to forward explicitly:

f([int x = -1]) {/*body*/}
g(late f.*) => f(x); // x is forwarded as is
g(late f.*) => f(...); // another form of the above, x is forwarded as is

In the rare cases where you really want to know whether the parameter was passed or not (remember, forwarding is already covered, so you need a different reason for wanting this), you can certainly devise a method like passed(x) or initialized(x), but then, as you pointed out, it should work for all late variables (not only for parameters). The syntax like x.default(()=>-1) covers all bases IMO (you actually don't need a no-arg x.default() for the purposes of forwarding - I was mistaken about that: for forwarding, the language should amend the rules of accessing late parameter by allowing it to be passed as is, be it initialized or not, as an optional parameter)

I've also realized that the syntax I proposed for bulk declaration was too noisy: there's no need to write ... there because * tells the whole story:

// rather then writing this
g(...f.*) => f(...);
// write this:
g(late f.*) => f(...);

[TekExplorer]

Hm... Using late to determine if something has been passed in is an interesting possibility.

I still don't like that it feels like reflection, but it's better than a passed() function, since at least it's now represented in the variable itself, and isn't so hyperspecific.

[tatumizer]

See the discussion of late parameters in #3680

[eernstg]

@TekExplorer wrote:

You seem to forget that redurecting factories can't have default values because the real function defines them

Here is the example again:

abstract class A {
  int foo([int i]);
}

class B1 extends A {
  int foo([int i = 1]) => i;
}

class B2 extends A {
  int foo([int i = 2]) => i;
}

class C {
  A a;
  C(this.a);
  foo ==> a;
}

void main() {
  var c1 = C(B1()), c2 = C(B2());
  print('Should be 1: ${c1.foo()}, should be 2: ${c2.foo()}');
}

It's correct that for a redirecting factory there is no support for specifying default values; the reason for this is that redirecting factories were designed to be eliminated at compile time.

The point I was making was that redirecting factories are very different from forwarding methods, and this is just one of the reasons why that is true.

(As I mentioned in the initial posting, a tear-off of a redirecting factory constructor needs forwarding semantics because we can't just use a tear-off of the redirectee, but that's exactly one of the motivations for proposing this mechanism, because we don't have a solution for that, yet.)

Another reason why forwarding methods require more than redirecting factories is that all the regular object-oriented mechanisms must work with forwarding methods. For example, late binding must work, which means that an instance of C must have an implementation of foo that follows all the normal rules for instance methods, such that it can be overridden by a completely normal instance method, and an invocation of foo on a receiver whose static type is C must still work even if the actual receiver has a different implementation of foo. In other words, there's no way we can eliminate forwarding methods at compile time.

In my example, the forwarding method was declared as foo ==> a;, that is, without any explicit default value declarations. This is an abbreviated form which means the same thing as foo ==> a.foo;, still without default value declarations.

The proposed mechanism allows for declarations like foo([int i = 17]) ==> a.foo;, as well as a lot of other forms that include default values. The point is that you might want to create a forwarding method that behaves slightly differently by having a different set of default values. Similarly, the forwarding method could have different parameter types (as long as they're strong enough to ensure that the forwarding call is type correct). However, the default form (the most concise form) will yield a forwarding method that only forwards the call and doesn't change anything.

That syntax is also... Kinda awful. At that point just do late final foo = a.foo;

Note that C.a is mutable. This means that late final foo = a.foo; will not have the same semantics as foo ==> a;. Also, we don't want to spend extra memory at run-time on storing a reference to a function object in the C instance, and storing the function object itself in the heap, if we could instead use a forwarding declaration to make these things shared among all instances of C. Finally, you can't override an existing method with signature int foo([int]) by a variable (late final or not), and you can't override the variable by another method with that signature. So there are lots of cases where the variable won't suffice.

Of course, we can't do that with methods, which is why we need proper redirection at all

I'm not quite sure what this means, perhaps you're making the same point?

[eernstg]

@TekExplorer wrote:

I also don't see any reason why argument records would be any more expensive than what we have now, since it's not like we can do anything (directly) with records anyway.

const count = 100_000_000;

int f1(int x, int y, int z) => x + y + z;

int f2((int x, int y, int z) r) => r.$1 + r.$2 + r.$3;

int sink = 0; // The compiler can't eliminate assignments to this variable.

void main() {
  Stopwatch stopwatch;
  double elapsedTime;
  
  stopwatch = Stopwatch()..start();
  for (int i = 0; i < count; ++i) {
    sink = f1(i, i, i);
  }
  elapsedTime = stopwatch.elapsedMilliseconds / 1000;
  print("${elapsedTime.toStringAsFixed(2)} seconds"); // '0.27 seconds'.
  
  stopwatch = Stopwatch()..start();
  for (int i = 0; i < count; ++i) {
    sink = f2((i, i, i));
  }
  elapsedTime = stopwatch.elapsedMilliseconds / 1000;
  print("${elapsedTime.toStringAsFixed(2)} seconds"); // '1.81 seconds'
}

(Caveat emptor, this is a micro-benchmark.)

As @lrhn mentioned, actual arguments of a function invocation (including instance method invocations) can very likely be stored on the run-time stack. This could be as references to existing heap objects or immediate representations like small integers (on some platforms), but it does not introduce a new object. Another option is to pass actual arguments in registers.

A record needs to be a proper object in the heap. It actually also needs to be an instance of a generic class, because it must be possible to assign (1, 2, 3) to a variable of type (num, Object, int).

It might be possible to reduce the overhead associated with actual-argument-lists-are-records using all kinds of optimizations, but it is a very tall order. It's basically a non-starter to change Dart such that every actual argument list is a record.

Another thing that we've discussed in the language team and which is doable is to support a mechanism whereby a record is subject to "spreading" into an actual argument list. For example, f1(1, 1, 1) would be the same thing as f1(1, ...r) where r is an expression of type (int, int) with the value (1, 1). This is fine, because it doesn't make all call sites in all programs more expensive, it just costs a bit more for that particular invocation.

[eernstg]

@TekExplorer wrote:

I do like the conditional passing as an independent feature

+1!

[eernstg]

@tatumizer wrote:

I'd rather have a pseudo-method default defined only for parameters

This might be similar to #2269, except that #2269 would need to be generalized in order to be able to handle the case where the default values of the forwardee aren't known statically.

[eernstg]

One request which comes up repeatedly in this discussion is "introduce language mechanisms to abstract over formal parameter declaration lists and actual argument lists", e.g., this comment. This would allow us to declare many formal parameters concisely, and passing them one in a forwarding call, with various degrees of modification (depending on how fancy those language mechanisms are).

I already had the following in the original posting, and I certainly acknowledge that this could be generalized:

One forwarding method: 56 lines of code in current Dart, reduced to 1 line

extension on BuildContext {
  TextStyle textStyleWith({
    bool? inherit,
    required Color color,
    Color? backgroundColor,
    double? fontSize,
    FontWeight? fontWeight,
    FontStyle? fontStyle,
    double? letterSpacing,
    double? wordSpacing,
    TextBaseline? textBaseline,
    double? height,
    TextLeadingDistribution? leadingDistribution,
    Locale? locale,
    Paint? foreground,
    Paint? background,
    List<Shadow>? shadows,
    List<FontFeature>? fontFeatures,
    List<FontVariation>? fontVariations,
    TextDecoration? decoration,
    Color? decorationColor,
    TextDecorationStyle? decorationStyle,
    double? decorationThickness,
    String? debugLabel,
    String? fontFamily,
    List<String>? fontFamilyFallback,
    String? package,
    TextOverflow? overflow,
  }) => CupertinoTheme.of(this).textTheme.textStyle.copyWith(
    inherit: inherit,
    color: color, // Declared as `Color? color`.
    backgroundColor: backgroundColor,
    fontSize: fontSize,
    fontWeight: fontWeight,
    fontStyle: fontStyle,
    letterSpacing: letterSpacing,
    wordSpacing: wordSpacing,
    textBaseline: textBaseline,
    height: height,
    leadingDistribution: leadingDistribution,
    locale: locale,
    foreground: foreground,
    background: background,
    shadows: shadows,
    fontFeatures: fontFeatures,
    fontVariations: fontVariations,
    decoration: decoration,
    decorationColor: decorationColor,
    decorationStyle: decorationStyle,
    decorationThickness: decorationThickness,
    debugLabel: debugLabel,
    fontFamily: fontFamily,
    fontFamilyFallback: fontFamilyFallback,
    package: package,
    overflow: overflow,
  );
}

// Becomes

extension on BuildContext {
  textStyleWith({required Color color, ...}) ==> CupertinoTheme.of(this).textTheme.textStyle.copyWith;
}

Note that the return type in the abbreviated form is still TextStyle, not dynamic. It is obtained implicitly by "inheriting" it from the forwardee, which is a built-in capability of ==>.

The only kind of abstraction I suggested was (1) that ... could stand for "all remaining positional parameters" or "all remaining named parameters", which means that we can specify formal parameters differently for a prefix of the positional parameters and for any subset of the named parameters, and then get all the rest for free.

That said, I'd still like to have support for the straightforward and faithful forwarding semantics using a concise notation like foo ==> a; or foo ==> a.foo;. This would also generalize to * ==> a; as a notation for "declare forwarding methods for every member of the statically known interface of a which isn't declared manually". Of course, this can also be generalized further in a lot of ways, but the point is that I'd like to have simple, faithful, and readable support for using the strategy pattern and similar techniques where methods are being forwarded (no tricks, just plain forwarding).

---

Next, there's a request to "introduce language mechanisms to abstract over passing or not passing an actual argument", e.g., this comment and this issue.

As I mentioned earlier, I certainly appreciate the value of this feature. I just don't think it's sufficiently powerful to fill the whole bill when it comes to method forwarding.

Not passing an actual argument will indeed yield a faithful reproduction of the semantics of the direct call with respect to that parameter and its default value. However, I think it needs to be combined with the previous kind of feature (allowing us to abstract over formal parameter declaration lists and actual argument lists). Otherwise we'll get an even more verbose result:

extension on BuildContext {
  TextStyle textStyleWith({
    bool? inherit,
    required Color color,
    Color? backgroundColor,
    double? fontSize,
    FontWeight? fontWeight,
    FontStyle? fontStyle,
    double? letterSpacing,
    double? wordSpacing,
    TextBaseline? textBaseline,
    double? height,
    TextLeadingDistribution? leadingDistribution,
    Locale? locale,
    Paint? foreground,
    Paint? background,
    List<Shadow>? shadows,
    List<FontFeature>? fontFeatures,
    List<FontVariation>? fontVariations,
    TextDecoration? decoration,
    Color? decorationColor,
    TextDecorationStyle? decorationStyle,
    double? decorationThickness,
    String? debugLabel,
    String? fontFamily,
    List<String>? fontFamilyFallback,
    String? package,
    TextOverflow? overflow,
  }) => CupertinoTheme.of(this).textTheme.textStyle.copyWith(
    inherit: if (??inherit) inherit,
    color: color,
    backgroundColor: if (??backgroundColor) backgroundColor,
    fontSize: if (??fontSize) fontSize,
    fontWeight: if (??fontWeight) fontWeight,
    fontStyle: if (??fontStyle) fontStyle,
    letterSpacing: if (??letterSpacing) letterSpacing,
    wordSpacing: if (??wordSpacing) wordSpacing,
    textBaseline: if (??textBaseline) textBaseline,
    height: if (??height) height,
    leadingDistribution: if (??leadingDistribution) leadingDistribution,
    locale: if (??locale) locale,
    foreground: if (??foreground) foreground,
    background: if (??background) background,
    shadows: if (??shadows) shadows,
    fontFeatures: if (??fontFeatures) fontFeatures,
    fontVariations: if (??fontVariations) fontVariations,
    decoration: if (??decoration) decoration,
    decorationColor: if (??decorationColor) decorationColor,
    decorationStyle: if (??decorationColor) decorationStyle,
    decorationThickness: if (??decorationThickness) decorationThickness,
    debugLabel: if (??debugLabel) debugLabel,
    fontFamily: if (??fontFamily) fontFamily,
    fontFamilyFallback: if (??fontFamilyFallback) fontFamilyFallback,
    package: if (??package) package,
    overflow: if (??overflow) overflow,
  );
}

// Versus

extension on BuildContext {
  textStyleWith({required Color color, ...}) ==> CupertinoTheme.of(this).textTheme.textStyle.copyWith;
}

By the way, did you notice the bug at decorationColor? ;-)

We could certainly aim for a proposal that has all of these elements (the ability to faithfully handle default values plus the ability to quantify over formal parameters / actual arguments), and it would be quite powerful.

I also think it would be nice to be able to test whether a formal parameter has been bound, and whether a late variable with no initializing expression has been initialized.

However, I still think that it may be useful to have the specialized forwarding syntax, even if it is considered to be syntactic sugar for code that uses ??param and such. For example, it is not obvious to me how those other mechanisms could :

class InterceptClearList<X> implements List<X> {
  final List<X> _list;
  void Function<Y>(List<Y> self)? callback;
  
  InterceptClearList(this._list);

  void clear() {
    callback?.call(_list); // Call if not null.
    _list.clear();
  }

  * ==> _list; // Forward all other members to `_list`.
}

void foo(List<int> xs) {
  xs.clear(); // Haha! This is where it happens.
}

void main() {
  var xs = [1, 2, 3];
  xs = InterceptClearList(xs)..callback = <Y>(ys) {
    print('Warning: Someone is clearing a List<$Y>!');
  };
  ...
  foo(xs); // 'Warning: ...'.  
  ...
}

[tatumizer]

@eernstg: I think "delegation" is a more established name for the concept.
Maybe instead of * ==> obj you can write more legibly
delegate method1, method2 to obj;
delegate List.(* hide clear) to _list;
delegate method1, method2 to _obj, _obj1; // send to both; see use cases in C# "delegate" feature

"Forwarding" may be considered a more general concept with fine-grained control over the parameter lists.

[eernstg]

I think I'll keep using the word 'forwarding' here.

Lieberman wrote this paper, ACM DL, in 1986 where 'delegation' was defined to be a mechanism that invokes a method on a different object than the receiver, but preserving the notion of self or this (such that subsequent invocations of methods would start their method search from the same object), and 'forwarding' was defined to be the simpler mechanism that invokes a method on a different object using that other object as the new self/this.

Lieberman's work has been very influential in the topic area of delegation, to some extent because he pointed out the need to distinguish between forwarding and delegation at a time where this wasn't generally acknowledged, so I think it's fair to say that this is the standard terminology on those matters. It's certainly the terminology that I'm using here.

Delegation is a more powerful mechanism than forwarding, but also a mechanism which is very tricky to handle in terms of safe, static typing. As an example, this paper is all about handling delegation in a type-safe manner, and it shows how difficult this is. In contrast, a language like Self uses delegation as one of the very few (but powerful) mechanisms in the language, and this enables all kinds of weird and wonderful things that have inheritance as a special case. Self is extremely dynamic.

Here is an example that shows the difference (assuming that we can delegate to an object by declaring delegate thatObject):

class A {
  void foo() => bar();
  void bar() => print('A.bar');
}

class Delegator {
  delegate A(); // Delegates `foo` to an `A`.
  void bar() => print('Delegator.bar');
}

class Forwarder {
  void bar() => print('Forwarder.bar');
  * ==> A(); // Forwards 'foo' to an `A`.
}

void main() {
  var d = Delegator();
  var f = Forwarder();
  d.bar(); // 'Delegator.bar'.
  d.foo(); // 'Delegator.bar'.
  f.bar(); // 'Forwarder.bar'.
  f.foo(); // 'A.foo'.
}

I'm proposing to add support for forwarding in this issue, not delegation.

Maybe instead of * ==> obj you can write more legibly
delegate method1, method2 to obj;

The abstraction which is provided by * can of course be generalized in a million ways, but if we keep it simple and specify that * means "all members of the forwardee which aren't declared here already" then we would have to use two declarations:

method1 ==> obj;
method2 ==> obj;

One generalization could be to support multiple methods, perhaps specified as a comma separated list of names:

method1, method2 ==> obj;

This would still be compatible with situations where we want to specify some elements of the signature:

MoreGeneralReturnType method1, method2(MoreSpecialParameterType p) ==> obj;

This could be useful if obj is a complex expression.

delegate List.(* hide clear) to _list;

Interesting! * could be generalized by means of a specified type, not just the type of the forwardee.

Iterable.(* hide length) ==> _list;

This would forward all other members if Iterable than length to _list (except that it would also omit the forwarding for any member whose name is already declared in this class/mixin/enum/... body).

delegate method1, method2 to _obj, _obj1; // send to both; see use cases in C# "delegate" feature

I should mention that a 'delegate' in C# is just a function type, it has no particular relevance to the notion of forwarding in this context.

However, it does have a very special property which is the ability to hold more than one function pointer, in a specific order. I don't really think we'd want to introduce anything of this nature into Dart. In particular, a multicast delegate in C# will ignore all results if it has return type void, and it will return the result from the last function invoked if the return type is non-void (and ignore all the other results). This seems very ad-hoc to me. So I'd prefer that Dart developers write a single function/method which has the desired combined semantics.

[TekExplorer]

I feel like we should try to stick to established patterns somehow.

Like, we have super. and this. In initializer lists. Could we find a good way to do something similar for forwarding?

Like:

class A {
  final B b;
  void action(it.n1, it.n2, {Specific it.n3, required it.n4}) = b.action;
  void action(.n1, .n2, {Specific .n3, required .n4}) = b.action;
  void action(n1, n2, {Specific n3, required n4}) = b.action;

  // Blind forwarding (implied type)
  void action(*args) = b.action;
  // _ for context type inference, if we think the above is dynamic
  void action(_ *args) = b.action;
  // More explicit. Could be a typedef
  void action((int n1, String n2, {NonSpecific n3, Foo n4}) *args) = b.action;
  // Other possibilities may be present.
}

Possibly find either a word to use it? or just keep it contextually implied with the ., or not specify anything weird in the first place.

The forwarding syntax is reused, and intuitive.

I think that if we really want to do more than that, a macro is probably better suited, because having those members all present on the class is probably better, and much less magic.

We also can't do mass forwarding as-is safely anyway due to the forwarded changing itself suddenly becoming a breaking change, even when it shouldn't be.

That's a lot of implicit stuff to be comfortable with.

[tatumizer]

@eernstg: other languages have no qualms reusing the verb "delegate" how they see fit, with no regard to the cited paper. E.g. in rust, "delegate" means (exactly? more or less?) the same as whatever you mean by "forward" https://crates.io/crates/delegate

If you don't like "delegate" and prefer to call it "forward" then you can write so:
forward * to obj;
My point is that not every feature requires the introduction of yet another symbol - these symbols are in short supply. :-)

[eernstg]

@TekExplorer wrote:

we have super. and this. in initializer lists. Could we find a good way to do something similar for forwarding?

I think it will be useful to be able to write a faithfully forwarding invocation in terms of lower-level mechanisms. I also think it will be useful to have support for abstraction (such that we don't get the verbose and error prone form which is obtained by building the invocation from first principles using a lower-level mechanism). Finally, I think it will be very convenient to be able to obtain the signature of the forwarding method implicitly from the forwardee (that is, we get the return type from there if it isn't specified explicitly, and we get the formal parameter declarations that weren't specified explicitly).

---

class A {
  final B b;
  void action(it.n1, it.n2, {Specific it.n3, required it.n4}) = b.action;
  void action(.n1, .n2, {Specific .n3, required .n4}) = b.action;
  void action(n1, n2, {Specific n3, required n4}) = b.action;
  ...
}

These notations are concerned with the third task: Writing an abbreviated signature and getting the missing parts from the static type of the forwardee.

For the declarations shown above, some parameters don't have a type (e.g., n1 and n4); I assume those types will be provided implicitly by lookups into the signature of the forwardee. So n3 will get the type Specific which would have to be a subtype of the parameter type in the forwardee (and probably a proper subtype because there's no reason to write it if it is the same type). Similarly for required which is true for A.action, n4, but presumably not for B.action, n4.

I think this kind of abstraction is already covered in a very similar way by my proposal: I'd write void action(..., {Specific n3, required n4}) ==> b;. The first ellipsis stands for all the positional parameters (we could write them explicitly, but we don't have to), Specific works the same, and so does required.

I don't see a need to use special syntax for the explicitly declared parts of the signature, it works just fine to declare the formal parameters using the standard syntax of formal parameters.

We could consider supporting reorganizations of the parameter list (e.g., changing the order of some positional parameters, changing some named parameters to positional or vice versa, and so on), in which case we would need to mention the names of positional parameters. However, I'd prefer to omit this kind of feature: If you want to change the shape/ordering of the formal parameter list then you must write it out in full. A major reason why this is a dangerous feature is that it makes the names of positional parameters in the forwardee significant (e.g., renaming one of them would now be a breaking change).

With respect to ==> vs =, see this comment.

---

class A {
  final B b;
  ...
  void action(*args) = b.action;
  ...
}

For the second task, I think we'd want something like ... as a way to specify all the remaining formal parameter declarations, positional or named. We could also use *args, but there's no way to use that name unless we come up with some additional machinery.

We also can't do mass forwarding as-is safely anyway due to the forwarded changing itself suddenly becoming a breaking change

I don't think this is a given. Mass forwarding is likely to be connected with a typing relation:

class A {
  ...
  int get aThing => 24;
  void anotherAThing(num n) {}
}

class B implements A {
  final A a;
  B(this.a);

  void bThing() {} // New, not in `A`'s interface.
  int get aThing => 42; // `B` will have its own implementation of `aThing`.
   
  * ==> a; // Forward all other members to `a`.
}

The class B is not only using the behavior of the A by forwarding to it, but it also has the ability to be used as an A by having implements A.

If there is a breaking change to the interface of A then B will continue to work as an A. Other parts of the program may need to change because A changed (that's true whether or not B exists), but the fact that B still implements A might be the most important part with respect to B.

It's a software engineering decision which way to go. If B should not be able to be used as an A and we're worried about breaking changes to A then we should write the signatures in full (and we probably shouldn't have implements A). We could do this in a post-hoc fashion, by the way:

class A {
  ...
  int get anAThing => 24;
  void anotherAThing(int i) {} // BREAKING CHANGE!
}

class B implements A { // `B` actually still implements `A`, so let's keep it.
  final A a;
  B(this.a);

  void bThing() {}
  int get anAThing => 42;
  void anotherAThing(num n) => anotherAThing(n.toInt()); // Compensate for the breaking change.

  * ==> a; // Forward all other members to `a`.
}

---

We didn't get to the first task (writing a faithfully forwarding invocation in terms of lower-level mechanisms). This is mainly handled by mechanisms like passed(p) or ??p (to determine whether or not the given parameter p was passed), but we could also have abstractions over actual argument lists (like b.action(42 + n2, 2 * n1, ...) passing different positional parameters at position 1 and 2, and passing all other parameters as actual arguments in the straightforward way).

[eernstg]

@tatumizer wrote:

other languages have no qualms reusing the verb "delegate" how they see fit

Perhaps the Rust folks haven't been thinking about this distinction? Anyway, we can play around with terminology as much as we want if we're willing to explain the meaning of every word as we write/speak it. If we want to communicate more effectively then it does make sense to try to stick to existing definitions.

you can write so:
forward * to obj;

Indeed, lots of things could be expressed using that kind of declaration.

However, I chose ==> because it's similar to =>, and it already provides a location for the name of the declaration (to the left of ==>) and a location for the body (to the right), and safe & simple parsing of things like a return type and/or a formal parameter list.

For instance, (String, ...) ==> obj.myMethod is a function literal yielding a function whose first positional parameter has type String and is mandatory (the one in obj.myMethod could, say, be optional and have type Object), and the remaining parameters are declared as in myMethod. I can't immediately see a natural way to express a similar thing using forward ....