Allow == null to promote to Null

[stereotype441]

In my investigation of #1618, I ran into several instances in not-yet-migrated code of conditional expressions like this: v == null ? v : f(v) (where v is a local variable and f(v) is some complex expression involving v. Prior to null safety, this generally works because if v and f(v) have unrelated types, the type of the conditional expression is Object, and Object can be implicitly downcast to any other type. But when this pattern is migrated to null safety, it almost always leads to a compile-time error, because null safe code only allows implicit downcasts from dynamic.

Users can of course work around this problem by changing to v == null ? null : f(v). But @lrhn points out that if we allowed a successful v == null test to promote the type of v to Null, they wouldn't need to.

Off the top of my head, I can't think of any reason why we shouldn't allow this promotion. (I suspect the only reason it didn't get implemented as part of flow analysis in the first place is because we couldn't think of a use case for it).

[lrhn]

There really is no necessary use-case.
If we know that v has type Null, we know that writing v is the same as writing null, so just write the darn null. It's easier to read what's happening that way.

Except possibly the case of promoting a type-variable typed variable to X&Null. I can't come up with a good use of it, but that variable would be an expression with a type different from just null.

[stereotype441]

@lrhn yeah, honestly I'm really torn about this one. On the one hand, != null promotes to non-nullable types, so I think it's quite reasonable for people to expect that == null should promote to the Null type. On the other hand, I think this v == null ? v : f(v) pattern is ugly and confusing, and I agree with you that users should just write the damn null.

On the other other hand, I'm not sure "just write the damn null" is legitimate grounds for objecting to a language feature. I can come up with objections to a lot of good language features using that pattern (type inference? Just write the damn type! Initializing formals? Just write the damn initializer! Enums? Just write the damn boilerplate class). On the other^3 hand, those other features actually save time and make the code easier to read; this one really doesn't. On the other^4 hand, maybe the fact that this pattern actually crops up in the wild in unmigrated code is a sign that it makes more sense to some people, and why should I stop them from writing the code in a way that makes sense to them?

Ok, I'm out of hands now. I may play with this just out of curiosity, to see how difficult it would be to implement (I suspect it's very easy), and to double check whether there are any unintended consequences. But it definitely seems like a low priority improvement at best.

[stereotype441]

I was just reminded that this issue has come up before, and it proved to have more complex consequences than we expected: #1505

[rakudrama]

On the web compilers,

v == null ? v : f(v)

is not the same as

v == null ? null : f(v)

since there are two JavaScript values (null, undefined) that are considered to be Dart null.

This matters for js-interop. We must allow the programmer to copy a null value from one external call to another without changing the 'bit pattern' of the null value.

Since we cannot avoid multiple Null values, perhaps the language specification should be agnostic to the number of different bit patterns that are considered to be null. A potential application that could help developers is for each null literal to be a distinct value, providing a degree of provenance for erroneous null values (see https://dl.acm.org/doi/10.1145/1297105.1297057). In this application, v == null ? v : f(v) is superior in preserving provenance since it copy-propagates the null value rather than replacing it.

[stereotype441]

@rakudrama Oh, that's very interesting. I love the idea of multiple null values that can be distinguished by debugging tools (but not by the program itself) for the purpose of tracking the provenance of nulls. We should totally do this in the Dart VM.
Though for this case I would argue that allowing the source code to say v == null ? v : f(v) is not strictly necessary, because it would be better for the compiler to just recognize the pattern v == null ? null : f(v), and translate it to v == null ? v : f(v) during code generation.

As for the idea of multiple null values that can be distinguished by the program itself, well, I wish it weren't a thing, but you're right, it totally is in Javascript, and for that case, v == null ? v : f(v) and v == null ? null : f(v) do really have different meanings. Sigh. I guess that bumps up the priority of looking for a way to promote to Null 😃

[lrhn]

Though for this case I would argue that allowing the source code to say v == null ? v : f(v) is not strictly necessary, because it would be better for the compiler to just recognize the pattern v == null ? null : f(v), and translate it to v == null ? v : f(v) during code generation.

I'd bet that there will be situations where that too would be wrong, and you would want to definitely send the Dart null value, not the null-or-undefined value of v to the native JS code.
What I would encourage is to have a JS.undefined constant that you can use instead, when you absolutely need undefined instead of null. If you know v is actually undefined instead of null after v == null, then you should write v == null ? JS.undefined : f(v) if that's what you want, and not rely on Dart making a difference between the different null implementation values.

[rakudrama]

I want to ensure that the language specification is not unfriendly towards the reality of multiple null values by cooking in the assumption that there is a single value null. It makes JavaScript-interop more difficult, and makes ideas like the cited paper more difficult. I would claim that parts of the language specification currently are unfriendly.

Imagine that js-interop is used, in effect, to store a JavaScript value in a Dart Map under some well-behaved Dart key. It would be incorrect if any code involved was to 'convert' the undefined into null or vice-versa. At no point in this scenario does the programmer know anything about the 'abstract' JavaScript value, whether it is null, undefined or some other value, so providing JS.undefined it not helpful here.

One way that an unwanted conversion might happen is if the promotion-to-Null of v == null is followed by some assumption in the language specification or tools (CFE etc) that Null values can be replaced with null.
Conversely, there should not be a rewrite of null -> to v just because v is the controlling value promoted to Null.
Another source of unwanted conversions is the current specification of ?..
This specification could be amended to have better provenance tracking.
(Interestingly, JavaScript also has a ?. operator that results in undefined for a null-ish receiver, which hampers optimizing Dart ?. to JavaScript ?.. This optimization is not currently on our radar since we support browsers a little older that the introduction of ?.)

[leafpetersen]

I'm a little lost as to where the discussion is going here. Nothing in this proposal touches on semantics, and whether compilers do or do not replace values of type Null with a fresh Null value. It's true that programmers may do so, but they may do so whether the type system does or does not acknowledge the fact that a variable is always Null.

In general, I'm suspicious of claims that we should not reflect things that are blatantly obvious to the programmer into the type system: if a user writes x = null, it seems useful to have the type system know that x is in fact now null, for precisely the reasons we are describing (e.g. that if (x == null) is a trivially true condition).

[stereotype441]

I did some investigation of what the consequences would be of allowing == null to promote to Null. Please see my analysis here: #1505 (comment)

The TL/DR is, I don't see a big benefit to allowing == null to promote to Null, and there are some common coding patterns that would be broken by it. On the whole I don't think the change would be worthwhile.