From 8bf642cf9d0306e91543b8ef5fd86d80e6b754a4 Mon Sep 17 00:00:00 2001 From: Josh L Date: Fri, 2 Jul 2021 16:44:48 -0700 Subject: [PATCH 01/21] Good start --- docs/design/generics/appendix-coherence.md | 149 +++++++++++++++++++++ docs/design/generics/goals.md | 17 ++- docs/design/generics/terminology.md | 24 +++- 3 files changed, 179 insertions(+), 11 deletions(-) create mode 100644 docs/design/generics/appendix-coherence.md diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md new file mode 100644 index 0000000000000..100bff8c84a27 --- /dev/null +++ b/docs/design/generics/appendix-coherence.md @@ -0,0 +1,149 @@ +# Carbon: alternatives to coherence + + + +This document explains the rationale for choosing to make +[implementation coherence](terminology.md#coherence) +[a goal for Carbon](goals.md#coherence), and the alternatives considered. + +## Approach taken: coherence + +The main thing to understand is that coherence is a desirable property, but to +get that property we need an orphan rule, and that rule has a cost. It in +particular limits how much control users of a type have over how that type +implements interfaces. There are two main use cases to consider: + +- Selecting between multiple implementations of a `Comparable` interface for a + `Song` type to support "by title", "by artist", and "by album" orderings. +- Implementing an interface for a type when there is no relationship between + the libraries defining the interface and the type. + +Since Carbon is bundling interface implementations into types, for the +convenience and expressiveness that provides, we satisfy those use cases by +giving the user control over the type of a value. This means having facilities +for defining new [compatible types](terminology#compatible-types) with different +interface implementations, and casting between those types as needed. + +## The "Hashtable Problem" + +The "Hashtable" problem is that the specific hash function used to compute the +hash of keys in a hashtable must be the same when adding an entry, when looking +it up, and other operations like resizing. So a hashtable type is dependent on +both the key type, and the key type's implementation of the `Hashable` +interface. If the key type can have more than one implementation of `Hashable`, +there needs to be some mechanism for choosing a single one to be used +consistently by the hashtable type, or the invariants of the type will be +violated. + +Without the orphan rule to enforce coherence, we might have a situation like +this: + +- Package `Container` defines a `HashSet` type. + + ``` + package Container; + struct HashSet(Hashable$ Key) { ... } + ``` + +- A `Song` type is defined in package `SongLib`. +- Package `SongHashArtistAndTitle` defines an implementation of `Hashable` for + `SongLib.Song`. + + ``` + package SongHashArtistAndTitle; + import SongLib; + impl SongLib.Song as Hashable { + method (me: Self) Hash() -> UInt64 { ... } + } + ``` + +- Package `SongUtil` uses the `Hashable` implementation from + `SongHashArtistAndTitle` to define a function `IsInHashSet`. + + ``` + package SongUtil; + import SongLib; + import SongHashArtistAndTitle; + import Containers; + + fn IsInHashSet( + s: SongLib.Song, + h: Containers.HashSet(SongLib.Song)*) -> Bool { + return h->Contains(s); + } + ``` + +- Package `SongHashAppleMusicURL` defines a different implementation of + `Hashable` for `SongLib.Song` than package `SongHashArtistAndTitle`. + + ``` + package SongHashAppleMusicURL; + import SongLib; + impl SongLib.Song as Hashable { + method (me: Self) Hash() -> UInt64 { ... } + } + ``` + +- Finally, package `Trouble` imports `SongHashAppleMusicURL`, creates a hash + set, and then calls the `IsInHashSet` function from package `SongUtil`. + + ``` + package Trouble; + import SongLib; + import SongHashAppleMusicURL; + import Containers; + import SongUtil; + + fn SomethingWeirdHappens() { + var unchained_melody: SongLib.Song = ...; + var song_set: auto = Containers.HashSet(SongLib.Song).Create(); + song_set.Add(unchained_melody); + // Either this is a compile error or does something unexpected. + if (SongUtil.IsInHashSet(unchained_melody, &song_set)) { + Print("This is expected, but doesn't happen."); + } else { + Print("This is what happens even though it is unexpected."); + } + } + ``` + +The issue is that in package `Trouble`, the `song_set` is created in a context +where `SongLib.Song` has a `Hashable` implementation from +`SongHashAppleMusicURL`, and stores `unchained_melody` under that hash value. +When we go to look up the same song in `SongUtil.IsInHashSet`, it uses the hash +function from `SongHashArtistAndTitle` which returns a different hash value for +`unchained_melody`, and so reports the song is missing. + +FIXME: https://gist.github.com/nikomatsakis/1421744 + +## Problems with incohernce + +FIXME + +- "Import what you use" is hard to measure: libraries `Y.T1` and `Z.T2` are + important/used even though `Y` and `Z` are not mentioned outside the + `import` statement. +- The call `F(a)` has different interpretations depending on what libraries + are imported: + - If neither is imported, it is an error. + - If both are imported, it is ambiguous. + - If only one is imported, you get totally different code executed + depending on which it is. + +## Rejected alternative: dynamic + +FIXME + +## Rejected alternative: manual conflict resolution + +FIXME: +[Addressing "the hashtable problem" with type classes](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) + +## Rejected alternative: scoped conformance + +FIXME: +[scoped conformances](https://forums.swift.org/t/scoped-conformances/37159). diff --git a/docs/design/generics/goals.md b/docs/design/generics/goals.md index e993f37d70edf..81de273e1a1af 100644 --- a/docs/design/generics/goals.md +++ b/docs/design/generics/goals.md @@ -438,11 +438,10 @@ will necessarily be less incremental. ### Coherence -We want the generics system to have the _coherence_ property. This means that -there is a single answer to the question "what is the implementation of this -interface for this type, if any?" independent of context, such as the libraries -imported into a given file. Since a generic function only depends on interface -implementations, they will always behave consistently on a given type, +We want the generics system to have the +[_coherence_ property](terminology#coherence), so that the implementation of an +interface for a type is well defined. Since a generic function only depends on +interface implementations, they will always behave consistently on a given type, independent of context. For more on this, see [this description of what coherence is and why Rust enforces it](https://github.com/Ixrec/rust-orphan-rules#what-is-coherence). @@ -459,8 +458,8 @@ It also has a number of benefits for users: Carbon template on that type. The main downside of coherence is that there are some capabilities we would like -for interfaces which are in tension with the coherence property. For example, we -would like to address +for interfaces which are in tension with having an orphan rule limiting where +implementations may be defined. For example, we would like to address [the expression problem](https://eli.thegreenplace.net/2016/the-expression-problem-and-its-solutions#another-clojure-solution-using-protocols). We can get some of the way there by allowing the implementation of an interface for a type to be defined with either the interface or the type. But some use @@ -486,8 +485,8 @@ approaches that could work: interface implementations. This is the approach used by Rust ([1](https://doc.rust-lang.org/book/ch19-03-advanced-traits.html#using-the-newtype-pattern-to-implement-external-traits-on-external-types), [2](https://github.com/Ixrec/rust-orphan-rules#user-content-why-are-the-orphan-rules-controversial)). -- Carbon could support - [scoped conformances](https://forums.swift.org/t/scoped-conformances/37159). + +Alternatives to coherence are discussed in [an appendix](appendix-coherence.md). ### No novel name lookup diff --git a/docs/design/generics/terminology.md b/docs/design/generics/terminology.md index 8921d307d5b60..7368da6ea7eff 100644 --- a/docs/design/generics/terminology.md +++ b/docs/design/generics/terminology.md @@ -27,6 +27,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception - [Impls: Implementations of interfaces](#impls-implementations-of-interfaces) - [Compatible types](#compatible-types) - [Subtyping and casting](#subtyping-and-casting) +- [Coherence](#coherence) - [Adapting a type](#adapting-a-type) - [Type erasure](#type-erasure) - [Facet type](#facet-type) @@ -358,6 +359,24 @@ be explicit where an implicit cast would otherwise occur. For now, we are saying type `y`. Note that outside of generics, the term "casting" includes any explicit type change, including those that change the data representation. +## Coherence + +A generics system has the _implementation coherence_ property, or simply +_coherence_, if there is a single answer to the question "what is the +implementation of this interface for this type, if any?" independent of context, +such as the libraries imported into a given file. + +This is typically enforced by making sure the definition of the implementation +must be imported if you import both the interface and the type. This may be done +by requiring the implementation to be in the same library as the interface or +type. This is called an _orphan rule_, meaning we don't allow an implementation +that is not with either of its parents (parent type or parent interface). + +Note that in addition to an orphan rule that implementations are visible when +queried, coherence also requires a rule for resolving what happens if there are +multiple non-orphan implementations. This could be just producing an error in +that situation, or picking one using some specialization rule. + ## Adapting a type A type can be adapted by creating a new type that is @@ -372,11 +391,12 @@ between those two types without any dynamic checks or danger of [object slicing](https://en.wikipedia.org/wiki/Object_slicing). This is called "newtype" in Rust, and is used for capturing additional -information in types to improve type safety of move some checking to compile +information in types to improve type safety by moving some checking to compile time ([1](https://doc.rust-lang.org/rust-by-example/generics/new_types.html), [2](https://doc.rust-lang.org/book/ch19-04-advanced-types.html#using-the-newtype-pattern-for-type-safety-and-abstraction), [3](https://www.worthe-it.co.za/blog/2020-10-31-newtype-pattern-in-rust.html)) -and as a workaround for Rust's orphan rules for coherence. +and as a workaround for +[Rust's orphan rules for coherence](https://github.com/Ixrec/rust-orphan-rules#why-are-the-orphan-rules-controversial). ## Type erasure From 3f50459b5edb2690e31d18cfddc1e0d201314de3 Mon Sep 17 00:00:00 2001 From: Josh L Date: Sun, 4 Jul 2021 08:51:22 -0700 Subject: [PATCH 02/21] Name overlap rule --- docs/design/generics/terminology.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/design/generics/terminology.md b/docs/design/generics/terminology.md index 7368da6ea7eff..60f32a1a42468 100644 --- a/docs/design/generics/terminology.md +++ b/docs/design/generics/terminology.md @@ -374,8 +374,10 @@ that is not with either of its parents (parent type or parent interface). Note that in addition to an orphan rule that implementations are visible when queried, coherence also requires a rule for resolving what happens if there are -multiple non-orphan implementations. This could be just producing an error in -that situation, or picking one using some specialization rule. +multiple non-orphan implementations. In Rust, this is called the +[overlap rule or overlap check](https://rust-lang.github.io/chalk/book/clauses/coherence.html#chalk-overlap-check). +This could be just producing an error in that situation, or picking one using +some specialization rule. ## Adapting a type From ed24dfa2ff14e91fbc0b0c810258f41043ccf852 Mon Sep 17 00:00:00 2001 From: Josh L Date: Fri, 9 Jul 2021 16:12:12 -0700 Subject: [PATCH 03/21] Checkpoint progress. --- docs/design/generics/appendix-coherence.md | 51 +++++++++++++++------- 1 file changed, 35 insertions(+), 16 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 100bff8c84a27..e3a29ad80de4f 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -10,6 +10,19 @@ This document explains the rationale for choosing to make [implementation coherence](terminology.md#coherence) [a goal for Carbon](goals.md#coherence), and the alternatives considered. + + +## Table of contents + +- [Approach taken: coherence](#approach-taken-coherence) +- [The "Hashtable Problem"](#the-hashtable-problem) +- [Incoherence means context sensitivity](#incoherence-means-context-sensitivity) +- [Rejected alternative: dynamic](#rejected-alternative-dynamic) +- [Rejected alternative: manual conflict resolution](#rejected-alternative-manual-conflict-resolution) +- [Rejected alternative: scoped conformance](#rejected-alternative-scoped-conformance) + + + ## Approach taken: coherence The main thing to understand is that coherence is a desirable property, but to @@ -17,7 +30,8 @@ get that property we need an orphan rule, and that rule has a cost. It in particular limits how much control users of a type have over how that type implements interfaces. There are two main use cases to consider: -- Selecting between multiple implementations of a `Comparable` interface for a +- Selecting between multiple implementations of an interface for a type. For + example selecting the implementation of the `Comparable` interface for a `Song` type to support "by title", "by artist", and "by album" orderings. - Implementing an interface for a type when there is no relationship between the libraries defining the interface and the type. @@ -118,30 +132,35 @@ When we go to look up the same song in `SongUtil.IsInHashSet`, it uses the hash function from `SongHashArtistAndTitle` which returns a different hash value for `unchained_melody`, and so reports the song is missing. -FIXME: https://gist.github.com/nikomatsakis/1421744 +**Background:** [This post](https://gist.github.com/nikomatsakis/1421744) +discusses the hashtable problem in the context of Haskell, and +[this 2011 Rust followup](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) +discusses how to detect problems at compile time. + +## Incoherence means context sensitivity -## Problems with incohernce +The undesirable result of incoherence is that the interpretation of source code +changes based on imports. In particular, imagine there is a function call that +depends on a type implementing an interface, and two different implementations +are defined in two different libraries. A call to that function will be treated +differently depending on which of those two libraries are imported: -FIXME +- If neither is imported, it is an error. +- If both are imported, it is ambiguous. +- If only one is imported, you get totally different code executed depending + on which it is. -- "Import what you use" is hard to measure: libraries `Y.T1` and `Z.T2` are - important/used even though `Y` and `Z` are not mentioned outside the - `import` statement. -- The call `F(a)` has different interpretations depending on what libraries - are imported: - - If neither is imported, it is an error. - - If both are imported, it is ambiguous. - - If only one is imported, you get totally different code executed - depending on which it is. +Furthermore, this means that the behavior of a file can depend on an import even +if nothing from that package is referenced explicitly. ## Rejected alternative: dynamic -FIXME +One possible approach would be to bind interface implementations to a value at +the point it was created. In the example above, ## Rejected alternative: manual conflict resolution -FIXME: -[Addressing "the hashtable problem" with type classes](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) +[The problems with this approach have been considered in the context of Rust](https://github.com/Ixrec/rust-orphan-rules#whats-wrong-with-incoherence). ## Rejected alternative: scoped conformance From 80f544847b97009b1a92d8482f5590e96da4b56e Mon Sep 17 00:00:00 2001 From: Josh L Date: Fri, 9 Jul 2021 16:32:41 -0700 Subject: [PATCH 04/21] Checkpoint progress. --- docs/design/generics/appendix-coherence.md | 50 +++++++++++++++++----- 1 file changed, 40 insertions(+), 10 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index e3a29ad80de4f..c0cfc247163cb 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -17,9 +17,8 @@ This document explains the rationale for choosing to make - [Approach taken: coherence](#approach-taken-coherence) - [The "Hashtable Problem"](#the-hashtable-problem) - [Incoherence means context sensitivity](#incoherence-means-context-sensitivity) -- [Rejected alternative: dynamic](#rejected-alternative-dynamic) +- [Rejected alternative: dynamic implementation binding](#rejected-alternative-dynamic-implementation-binding) - [Rejected alternative: manual conflict resolution](#rejected-alternative-manual-conflict-resolution) -- [Rejected alternative: scoped conformance](#rejected-alternative-scoped-conformance) @@ -153,16 +152,47 @@ differently depending on which of those two libraries are imported: Furthermore, this means that the behavior of a file can depend on an import even if nothing from that package is referenced explicitly. -## Rejected alternative: dynamic +## Rejected alternative: dynamic implementation binding One possible approach would be to bind interface implementations to a value at -the point it was created. In the example above, +the point it was created. In [the example above](#the-hashtable-problem), the +implementation of the `Hashable` interface for `Song` would be fixed for the +`song_set` `HashSet` object based on which implementation was in scope in the +body of the `SomethingWeirdHappens` function. + +This has some downsides: + +- It is harder to reason about. The behavior of `SongUtil.IsInHashSet` depends + on the dynamic behavior of the program. At the time of the call, we may have + no idea where the `HashSet` argument was created. +- It requires more data space at runtime because we need to store a pointer to + the witness table representing the implementation with the object, since it + varies instead of being known statically. +- It is slower to execute from dynamic dispatch and the inability to inline. + +As a result, this doesn't make sense as the default behavior for Carbon based on +its [goals](/project/goals.md). That being said, this could be a feature added +later as opt-in behavior to either allow users to reduce code size or support +use cases that require dynamic dispatch. ## Rejected alternative: manual conflict resolution -[The problems with this approach have been considered in the context of Rust](https://github.com/Ixrec/rust-orphan-rules#whats-wrong-with-incoherence). - -## Rejected alternative: scoped conformance - -FIXME: -[scoped conformances](https://forums.swift.org/t/scoped-conformances/37159). +Carbon could alternatively provide some kind of manual disambiguation syntax to +resolve problems where they arise. The problems with this approach have been +[considered in the context of Rust](https://github.com/Ixrec/rust-orphan-rules#whats-wrong-with-incoherence). + +A specific example of this approach is called +[scoped conformance](https://forums.swift.org/t/scoped-conformances/37159), +where the conflict resolution is based on limiting the visibility of +implementations to particular scopes. This hasn't been implemented, but it has +the drawbacks described above. Depending on the details of the implementation, +either: + +- there are incompatible values with types that have the same name, or +- it is difficult to reason about the program's behavior because it behaves + like + [dynamic implementation binding](#rejected-alternative-dynamic-implementation-binding) + (though perhaps with a monomorphization cost instead of a runtime cost). + +In general, +[Carbon is avoiding this sort of context sensitivity](https://docs.google.com/document/d/1dLpEmUbE2_JQZ-pRNgEDnw6VuH4FXfURSYej6nPV6m0/edit#). From f98822b8c7b1a3718e01771497d495aa0e6cdeb1 Mon Sep 17 00:00:00 2001 From: Josh L Date: Mon, 12 Jul 2021 10:11:58 -0700 Subject: [PATCH 05/21] Another problem with dynamic impl binding --- docs/design/generics/appendix-coherence.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index c0cfc247163cb..309f4e75c3063 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -165,6 +165,9 @@ This has some downsides: - It is harder to reason about. The behavior of `SongUtil.IsInHashSet` depends on the dynamic behavior of the program. At the time of the call, we may have no idea where the `HashSet` argument was created. +- An object may be created far from a call that has a particular interface + requirement, with no guarantee that the object was created with any + implementation of the interface at all. - It requires more data space at runtime because we need to store a pointer to the witness table representing the implementation with the object, since it varies instead of being known statically. From 83d0939d614a99f798beb661384c430b95ffad80 Mon Sep 17 00:00:00 2001 From: Josh L Date: Mon, 12 Jul 2021 15:32:02 -0700 Subject: [PATCH 06/21] Add reference --- docs/design/generics/appendix-coherence.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 309f4e75c3063..79f8f8052d8fe 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -160,6 +160,9 @@ implementation of the `Hashable` interface for `Song` would be fixed for the `song_set` `HashSet` object based on which implementation was in scope in the body of the `SomethingWeirdHappens` function. +This idea is discussed briefly in section 5.4 on separate compilation of +[this proposal for implementing "Indiana" C++0x concepts proposal](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.9526&rep=rep1&type=pdf). + This has some downsides: - It is harder to reason about. The behavior of `SongUtil.IsInHashSet` depends @@ -167,7 +170,8 @@ This has some downsides: no idea where the `HashSet` argument was created. - An object may be created far from a call that has a particular interface requirement, with no guarantee that the object was created with any - implementation of the interface at all. + implementation of the interface at all. This error would only be detected at + runtime, not at type checking time. - It requires more data space at runtime because we need to store a pointer to the witness table representing the implementation with the object, since it varies instead of being known statically. From 1271b6ee699616d856a2a5204dd6890d26050714 Mon Sep 17 00:00:00 2001 From: Josh L Date: Fri, 2 Jul 2021 16:44:48 -0700 Subject: [PATCH 07/21] Good start --- docs/design/generics/appendix-coherence.md | 149 +++++++++++++++++++++ docs/design/generics/goals.md | 17 ++- docs/design/generics/terminology.md | 24 +++- 3 files changed, 179 insertions(+), 11 deletions(-) create mode 100644 docs/design/generics/appendix-coherence.md diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md new file mode 100644 index 0000000000000..100bff8c84a27 --- /dev/null +++ b/docs/design/generics/appendix-coherence.md @@ -0,0 +1,149 @@ +# Carbon: alternatives to coherence + + + +This document explains the rationale for choosing to make +[implementation coherence](terminology.md#coherence) +[a goal for Carbon](goals.md#coherence), and the alternatives considered. + +## Approach taken: coherence + +The main thing to understand is that coherence is a desirable property, but to +get that property we need an orphan rule, and that rule has a cost. It in +particular limits how much control users of a type have over how that type +implements interfaces. There are two main use cases to consider: + +- Selecting between multiple implementations of a `Comparable` interface for a + `Song` type to support "by title", "by artist", and "by album" orderings. +- Implementing an interface for a type when there is no relationship between + the libraries defining the interface and the type. + +Since Carbon is bundling interface implementations into types, for the +convenience and expressiveness that provides, we satisfy those use cases by +giving the user control over the type of a value. This means having facilities +for defining new [compatible types](terminology#compatible-types) with different +interface implementations, and casting between those types as needed. + +## The "Hashtable Problem" + +The "Hashtable" problem is that the specific hash function used to compute the +hash of keys in a hashtable must be the same when adding an entry, when looking +it up, and other operations like resizing. So a hashtable type is dependent on +both the key type, and the key type's implementation of the `Hashable` +interface. If the key type can have more than one implementation of `Hashable`, +there needs to be some mechanism for choosing a single one to be used +consistently by the hashtable type, or the invariants of the type will be +violated. + +Without the orphan rule to enforce coherence, we might have a situation like +this: + +- Package `Container` defines a `HashSet` type. + + ``` + package Container; + struct HashSet(Hashable$ Key) { ... } + ``` + +- A `Song` type is defined in package `SongLib`. +- Package `SongHashArtistAndTitle` defines an implementation of `Hashable` for + `SongLib.Song`. + + ``` + package SongHashArtistAndTitle; + import SongLib; + impl SongLib.Song as Hashable { + method (me: Self) Hash() -> UInt64 { ... } + } + ``` + +- Package `SongUtil` uses the `Hashable` implementation from + `SongHashArtistAndTitle` to define a function `IsInHashSet`. + + ``` + package SongUtil; + import SongLib; + import SongHashArtistAndTitle; + import Containers; + + fn IsInHashSet( + s: SongLib.Song, + h: Containers.HashSet(SongLib.Song)*) -> Bool { + return h->Contains(s); + } + ``` + +- Package `SongHashAppleMusicURL` defines a different implementation of + `Hashable` for `SongLib.Song` than package `SongHashArtistAndTitle`. + + ``` + package SongHashAppleMusicURL; + import SongLib; + impl SongLib.Song as Hashable { + method (me: Self) Hash() -> UInt64 { ... } + } + ``` + +- Finally, package `Trouble` imports `SongHashAppleMusicURL`, creates a hash + set, and then calls the `IsInHashSet` function from package `SongUtil`. + + ``` + package Trouble; + import SongLib; + import SongHashAppleMusicURL; + import Containers; + import SongUtil; + + fn SomethingWeirdHappens() { + var unchained_melody: SongLib.Song = ...; + var song_set: auto = Containers.HashSet(SongLib.Song).Create(); + song_set.Add(unchained_melody); + // Either this is a compile error or does something unexpected. + if (SongUtil.IsInHashSet(unchained_melody, &song_set)) { + Print("This is expected, but doesn't happen."); + } else { + Print("This is what happens even though it is unexpected."); + } + } + ``` + +The issue is that in package `Trouble`, the `song_set` is created in a context +where `SongLib.Song` has a `Hashable` implementation from +`SongHashAppleMusicURL`, and stores `unchained_melody` under that hash value. +When we go to look up the same song in `SongUtil.IsInHashSet`, it uses the hash +function from `SongHashArtistAndTitle` which returns a different hash value for +`unchained_melody`, and so reports the song is missing. + +FIXME: https://gist.github.com/nikomatsakis/1421744 + +## Problems with incohernce + +FIXME + +- "Import what you use" is hard to measure: libraries `Y.T1` and `Z.T2` are + important/used even though `Y` and `Z` are not mentioned outside the + `import` statement. +- The call `F(a)` has different interpretations depending on what libraries + are imported: + - If neither is imported, it is an error. + - If both are imported, it is ambiguous. + - If only one is imported, you get totally different code executed + depending on which it is. + +## Rejected alternative: dynamic + +FIXME + +## Rejected alternative: manual conflict resolution + +FIXME: +[Addressing "the hashtable problem" with type classes](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) + +## Rejected alternative: scoped conformance + +FIXME: +[scoped conformances](https://forums.swift.org/t/scoped-conformances/37159). diff --git a/docs/design/generics/goals.md b/docs/design/generics/goals.md index fc5f3c3c3f21d..2c5684f00dc17 100644 --- a/docs/design/generics/goals.md +++ b/docs/design/generics/goals.md @@ -438,11 +438,10 @@ will necessarily be less incremental. ### Coherence -We want the generics system to have the _coherence_ property. This means that -there is a single answer to the question "what is the implementation of this -interface for this type, if any?" independent of context, such as the libraries -imported into a given file. Since a generic function only depends on interface -implementations, they will always behave consistently on a given type, +We want the generics system to have the +[_coherence_ property](terminology#coherence), so that the implementation of an +interface for a type is well defined. Since a generic function only depends on +interface implementations, they will always behave consistently on a given type, independent of context. For more on this, see [this description of what coherence is and why Rust enforces it](https://github.com/Ixrec/rust-orphan-rules#what-is-coherence). @@ -459,8 +458,8 @@ It also has a number of benefits for users: Carbon template on that type. The main downside of coherence is that there are some capabilities we would like -for interfaces which are in tension with the coherence property. For example, we -would like to address +for interfaces which are in tension with having an orphan rule limiting where +implementations may be defined. For example, we would like to address [the expression problem](https://eli.thegreenplace.net/2016/the-expression-problem-and-its-solutions#another-clojure-solution-using-protocols). We can get some of the way there by allowing the implementation of an interface for a type to be defined with either the interface or the type. But some use @@ -486,8 +485,8 @@ approaches that could work: interface implementations. This is the approach used by Rust ([1](https://doc.rust-lang.org/book/ch19-03-advanced-traits.html#using-the-newtype-pattern-to-implement-external-traits-on-external-types), [2](https://github.com/Ixrec/rust-orphan-rules#user-content-why-are-the-orphan-rules-controversial)). -- Carbon could support - [scoped conformances](https://forums.swift.org/t/scoped-conformances/37159). + +Alternatives to coherence are discussed in [an appendix](appendix-coherence.md). ### No novel name lookup diff --git a/docs/design/generics/terminology.md b/docs/design/generics/terminology.md index c4dc04fcdb9cb..806bfd934fcf9 100644 --- a/docs/design/generics/terminology.md +++ b/docs/design/generics/terminology.md @@ -27,6 +27,7 @@ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception - [Impls: Implementations of interfaces](#impls-implementations-of-interfaces) - [Compatible types](#compatible-types) - [Subtyping and casting](#subtyping-and-casting) +- [Coherence](#coherence) - [Adapting a type](#adapting-a-type) - [Type erasure](#type-erasure) - [Facet type](#facet-type) @@ -359,6 +360,24 @@ be explicit where an implicit cast would otherwise occur. For now, we are saying type `y`. Note that outside of generics, the term "casting" includes any explicit type change, including those that change the data representation. +## Coherence + +A generics system has the _implementation coherence_ property, or simply +_coherence_, if there is a single answer to the question "what is the +implementation of this interface for this type, if any?" independent of context, +such as the libraries imported into a given file. + +This is typically enforced by making sure the definition of the implementation +must be imported if you import both the interface and the type. This may be done +by requiring the implementation to be in the same library as the interface or +type. This is called an _orphan rule_, meaning we don't allow an implementation +that is not with either of its parents (parent type or parent interface). + +Note that in addition to an orphan rule that implementations are visible when +queried, coherence also requires a rule for resolving what happens if there are +multiple non-orphan implementations. This could be just producing an error in +that situation, or picking one using some specialization rule. + ## Adapting a type A type can be adapted by creating a new type that is @@ -373,11 +392,12 @@ between those two types without any dynamic checks or danger of [object slicing](https://en.wikipedia.org/wiki/Object_slicing). This is called "newtype" in Rust, and is used for capturing additional -information in types to improve type safety of move some checking to compile +information in types to improve type safety by moving some checking to compile time ([1](https://doc.rust-lang.org/rust-by-example/generics/new_types.html), [2](https://doc.rust-lang.org/book/ch19-04-advanced-types.html#using-the-newtype-pattern-for-type-safety-and-abstraction), [3](https://www.worthe-it.co.za/blog/2020-10-31-newtype-pattern-in-rust.html)) -and as a workaround for Rust's orphan rules for coherence. +and as a workaround for +[Rust's orphan rules for coherence](https://github.com/Ixrec/rust-orphan-rules#why-are-the-orphan-rules-controversial). ## Type erasure From 4046b24a5caaa2ea75b74ca59c295cb7ee551495 Mon Sep 17 00:00:00 2001 From: Josh L Date: Sun, 4 Jul 2021 08:51:22 -0700 Subject: [PATCH 08/21] Name overlap rule --- docs/design/generics/terminology.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/design/generics/terminology.md b/docs/design/generics/terminology.md index 806bfd934fcf9..3faee864a3266 100644 --- a/docs/design/generics/terminology.md +++ b/docs/design/generics/terminology.md @@ -375,8 +375,10 @@ that is not with either of its parents (parent type or parent interface). Note that in addition to an orphan rule that implementations are visible when queried, coherence also requires a rule for resolving what happens if there are -multiple non-orphan implementations. This could be just producing an error in -that situation, or picking one using some specialization rule. +multiple non-orphan implementations. In Rust, this is called the +[overlap rule or overlap check](https://rust-lang.github.io/chalk/book/clauses/coherence.html#chalk-overlap-check). +This could be just producing an error in that situation, or picking one using +some specialization rule. ## Adapting a type From dc9e09b57297fa3977debd70c4eb62f97e677c2f Mon Sep 17 00:00:00 2001 From: Josh L Date: Fri, 9 Jul 2021 16:12:12 -0700 Subject: [PATCH 09/21] Checkpoint progress. --- docs/design/generics/appendix-coherence.md | 51 +++++++++++++++------- 1 file changed, 35 insertions(+), 16 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 100bff8c84a27..e3a29ad80de4f 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -10,6 +10,19 @@ This document explains the rationale for choosing to make [implementation coherence](terminology.md#coherence) [a goal for Carbon](goals.md#coherence), and the alternatives considered. + + +## Table of contents + +- [Approach taken: coherence](#approach-taken-coherence) +- [The "Hashtable Problem"](#the-hashtable-problem) +- [Incoherence means context sensitivity](#incoherence-means-context-sensitivity) +- [Rejected alternative: dynamic](#rejected-alternative-dynamic) +- [Rejected alternative: manual conflict resolution](#rejected-alternative-manual-conflict-resolution) +- [Rejected alternative: scoped conformance](#rejected-alternative-scoped-conformance) + + + ## Approach taken: coherence The main thing to understand is that coherence is a desirable property, but to @@ -17,7 +30,8 @@ get that property we need an orphan rule, and that rule has a cost. It in particular limits how much control users of a type have over how that type implements interfaces. There are two main use cases to consider: -- Selecting between multiple implementations of a `Comparable` interface for a +- Selecting between multiple implementations of an interface for a type. For + example selecting the implementation of the `Comparable` interface for a `Song` type to support "by title", "by artist", and "by album" orderings. - Implementing an interface for a type when there is no relationship between the libraries defining the interface and the type. @@ -118,30 +132,35 @@ When we go to look up the same song in `SongUtil.IsInHashSet`, it uses the hash function from `SongHashArtistAndTitle` which returns a different hash value for `unchained_melody`, and so reports the song is missing. -FIXME: https://gist.github.com/nikomatsakis/1421744 +**Background:** [This post](https://gist.github.com/nikomatsakis/1421744) +discusses the hashtable problem in the context of Haskell, and +[this 2011 Rust followup](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) +discusses how to detect problems at compile time. + +## Incoherence means context sensitivity -## Problems with incohernce +The undesirable result of incoherence is that the interpretation of source code +changes based on imports. In particular, imagine there is a function call that +depends on a type implementing an interface, and two different implementations +are defined in two different libraries. A call to that function will be treated +differently depending on which of those two libraries are imported: -FIXME +- If neither is imported, it is an error. +- If both are imported, it is ambiguous. +- If only one is imported, you get totally different code executed depending + on which it is. -- "Import what you use" is hard to measure: libraries `Y.T1` and `Z.T2` are - important/used even though `Y` and `Z` are not mentioned outside the - `import` statement. -- The call `F(a)` has different interpretations depending on what libraries - are imported: - - If neither is imported, it is an error. - - If both are imported, it is ambiguous. - - If only one is imported, you get totally different code executed - depending on which it is. +Furthermore, this means that the behavior of a file can depend on an import even +if nothing from that package is referenced explicitly. ## Rejected alternative: dynamic -FIXME +One possible approach would be to bind interface implementations to a value at +the point it was created. In the example above, ## Rejected alternative: manual conflict resolution -FIXME: -[Addressing "the hashtable problem" with type classes](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) +[The problems with this approach have been considered in the context of Rust](https://github.com/Ixrec/rust-orphan-rules#whats-wrong-with-incoherence). ## Rejected alternative: scoped conformance From dc4666a5f07427929389b5d950d85765f76bd957 Mon Sep 17 00:00:00 2001 From: Josh L Date: Fri, 9 Jul 2021 16:32:41 -0700 Subject: [PATCH 10/21] Checkpoint progress. --- docs/design/generics/appendix-coherence.md | 50 +++++++++++++++++----- 1 file changed, 40 insertions(+), 10 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index e3a29ad80de4f..c0cfc247163cb 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -17,9 +17,8 @@ This document explains the rationale for choosing to make - [Approach taken: coherence](#approach-taken-coherence) - [The "Hashtable Problem"](#the-hashtable-problem) - [Incoherence means context sensitivity](#incoherence-means-context-sensitivity) -- [Rejected alternative: dynamic](#rejected-alternative-dynamic) +- [Rejected alternative: dynamic implementation binding](#rejected-alternative-dynamic-implementation-binding) - [Rejected alternative: manual conflict resolution](#rejected-alternative-manual-conflict-resolution) -- [Rejected alternative: scoped conformance](#rejected-alternative-scoped-conformance) @@ -153,16 +152,47 @@ differently depending on which of those two libraries are imported: Furthermore, this means that the behavior of a file can depend on an import even if nothing from that package is referenced explicitly. -## Rejected alternative: dynamic +## Rejected alternative: dynamic implementation binding One possible approach would be to bind interface implementations to a value at -the point it was created. In the example above, +the point it was created. In [the example above](#the-hashtable-problem), the +implementation of the `Hashable` interface for `Song` would be fixed for the +`song_set` `HashSet` object based on which implementation was in scope in the +body of the `SomethingWeirdHappens` function. + +This has some downsides: + +- It is harder to reason about. The behavior of `SongUtil.IsInHashSet` depends + on the dynamic behavior of the program. At the time of the call, we may have + no idea where the `HashSet` argument was created. +- It requires more data space at runtime because we need to store a pointer to + the witness table representing the implementation with the object, since it + varies instead of being known statically. +- It is slower to execute from dynamic dispatch and the inability to inline. + +As a result, this doesn't make sense as the default behavior for Carbon based on +its [goals](/project/goals.md). That being said, this could be a feature added +later as opt-in behavior to either allow users to reduce code size or support +use cases that require dynamic dispatch. ## Rejected alternative: manual conflict resolution -[The problems with this approach have been considered in the context of Rust](https://github.com/Ixrec/rust-orphan-rules#whats-wrong-with-incoherence). - -## Rejected alternative: scoped conformance - -FIXME: -[scoped conformances](https://forums.swift.org/t/scoped-conformances/37159). +Carbon could alternatively provide some kind of manual disambiguation syntax to +resolve problems where they arise. The problems with this approach have been +[considered in the context of Rust](https://github.com/Ixrec/rust-orphan-rules#whats-wrong-with-incoherence). + +A specific example of this approach is called +[scoped conformance](https://forums.swift.org/t/scoped-conformances/37159), +where the conflict resolution is based on limiting the visibility of +implementations to particular scopes. This hasn't been implemented, but it has +the drawbacks described above. Depending on the details of the implementation, +either: + +- there are incompatible values with types that have the same name, or +- it is difficult to reason about the program's behavior because it behaves + like + [dynamic implementation binding](#rejected-alternative-dynamic-implementation-binding) + (though perhaps with a monomorphization cost instead of a runtime cost). + +In general, +[Carbon is avoiding this sort of context sensitivity](https://docs.google.com/document/d/1dLpEmUbE2_JQZ-pRNgEDnw6VuH4FXfURSYej6nPV6m0/edit#). From 26f6a70fb17eabce0303dfbc0b0cc70359b8f692 Mon Sep 17 00:00:00 2001 From: Josh L Date: Mon, 12 Jul 2021 10:11:58 -0700 Subject: [PATCH 11/21] Another problem with dynamic impl binding --- docs/design/generics/appendix-coherence.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index c0cfc247163cb..309f4e75c3063 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -165,6 +165,9 @@ This has some downsides: - It is harder to reason about. The behavior of `SongUtil.IsInHashSet` depends on the dynamic behavior of the program. At the time of the call, we may have no idea where the `HashSet` argument was created. +- An object may be created far from a call that has a particular interface + requirement, with no guarantee that the object was created with any + implementation of the interface at all. - It requires more data space at runtime because we need to store a pointer to the witness table representing the implementation with the object, since it varies instead of being known statically. From 65063571b59cad11b94851ffe43fc4be8c64c00b Mon Sep 17 00:00:00 2001 From: Josh L Date: Mon, 12 Jul 2021 15:32:02 -0700 Subject: [PATCH 12/21] Add reference --- docs/design/generics/appendix-coherence.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 309f4e75c3063..79f8f8052d8fe 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -160,6 +160,9 @@ implementation of the `Hashable` interface for `Song` would be fixed for the `song_set` `HashSet` object based on which implementation was in scope in the body of the `SomethingWeirdHappens` function. +This idea is discussed briefly in section 5.4 on separate compilation of +[this proposal for implementing "Indiana" C++0x concepts proposal](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.9526&rep=rep1&type=pdf). + This has some downsides: - It is harder to reason about. The behavior of `SongUtil.IsInHashSet` depends @@ -167,7 +170,8 @@ This has some downsides: no idea where the `HashSet` argument was created. - An object may be created far from a call that has a particular interface requirement, with no guarantee that the object was created with any - implementation of the interface at all. + implementation of the interface at all. This error would only be detected at + runtime, not at type checking time. - It requires more data space at runtime because we need to store a pointer to the witness table representing the implementation with the object, since it varies instead of being known statically. From 3cd03ed558b61a1fdb5185a4d76c71355ad0d5b2 Mon Sep 17 00:00:00 2001 From: Josh L Date: Mon, 12 Jul 2021 15:37:18 -0700 Subject: [PATCH 13/21] Checkpoint progress. --- docs/design/generics/appendix-coherence.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 79f8f8052d8fe..4e3940caadd33 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -201,5 +201,5 @@ either: [dynamic implementation binding](#rejected-alternative-dynamic-implementation-binding) (though perhaps with a monomorphization cost instead of a runtime cost). -In general, -[Carbon is avoiding this sort of context sensitivity](https://docs.google.com/document/d/1dLpEmUbE2_JQZ-pRNgEDnw6VuH4FXfURSYej6nPV6m0/edit#). +In general, Carbon is +[avoiding this sort of context sensitivity](https://docs.google.com/document/d/1dLpEmUbE2_JQZ-pRNgEDnw6VuH4FXfURSYej6nPV6m0/edit#). From 5c78cdbbf0487ed46924e7148d9fb30ce3085aac Mon Sep 17 00:00:00 2001 From: Josh L Date: Thu, 15 Jul 2021 11:23:24 -0700 Subject: [PATCH 14/21] TODO to fix link to Google doc --- docs/design/generics/appendix-coherence.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 4e3940caadd33..fc60971096db7 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -203,3 +203,6 @@ either: In general, Carbon is [avoiding this sort of context sensitivity](https://docs.google.com/document/d/1dLpEmUbE2_JQZ-pRNgEDnw6VuH4FXfURSYej6nPV6m0/edit#). +TODO: update to [this link](/docs/project/principles/low_context_sensitivity.md) +once [proposal #646](https://github.com/carbon-language/carbon-lang/pull/646) is +submitted. From 9c5df8da6fc6cf738839ce0d60ec243f11600007 Mon Sep 17 00:00:00 2001 From: josh11b Date: Mon, 19 Jul 2021 13:27:36 -0700 Subject: [PATCH 15/21] Apply suggestions from code review Co-authored-by: Chandler Carruth --- docs/design/generics/appendix-coherence.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index fc60971096db7..c308037d5fb57 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -43,7 +43,7 @@ interface implementations, and casting between those types as needed. ## The "Hashtable Problem" -The "Hashtable" problem is that the specific hash function used to compute the +The "Hashtable problem" is that the specific hash function used to compute the hash of keys in a hashtable must be the same when adding an entry, when looking it up, and other operations like resizing. So a hashtable type is dependent on both the key type, and the key type's implementation of the `Hashable` @@ -59,7 +59,7 @@ this: ``` package Container; - struct HashSet(Hashable$ Key) { ... } + struct HashSet(Key:! Hashable) { ... } ``` - A `Song` type is defined in package `SongLib`. From 5182e05492c89332c9b055b3eab823befd25f2b0 Mon Sep 17 00:00:00 2001 From: Josh L Date: Tue, 27 Jul 2021 11:18:18 -0700 Subject: [PATCH 16/21] Make all incoherence choices one rejected alternative --- docs/design/generics/appendix-coherence.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index c308037d5fb57..8bb82510f8634 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -16,9 +16,10 @@ This document explains the rationale for choosing to make - [Approach taken: coherence](#approach-taken-coherence) - [The "Hashtable Problem"](#the-hashtable-problem) -- [Incoherence means context sensitivity](#incoherence-means-context-sensitivity) -- [Rejected alternative: dynamic implementation binding](#rejected-alternative-dynamic-implementation-binding) -- [Rejected alternative: manual conflict resolution](#rejected-alternative-manual-conflict-resolution) +- [Rejected alternative: incoherence](#rejected-alternative-incoherence) + - [Incoherence means context sensitivity](#incoherence-means-context-sensitivity) + - [Rejected variation: dynamic implementation binding](#rejected-variation-dynamic-implementation-binding) + - [Rejected variation: manual conflict resolution](#rejected-variation-manual-conflict-resolution) @@ -136,7 +137,9 @@ discusses the hashtable problem in the context of Haskell, and [this 2011 Rust followup](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) discusses how to detect problems at compile time. -## Incoherence means context sensitivity +## Rejected alternative: incoherence + +### Incoherence means context sensitivity The undesirable result of incoherence is that the interpretation of source code changes based on imports. In particular, imagine there is a function call that @@ -152,7 +155,7 @@ differently depending on which of those two libraries are imported: Furthermore, this means that the behavior of a file can depend on an import even if nothing from that package is referenced explicitly. -## Rejected alternative: dynamic implementation binding +### Rejected variation: dynamic implementation binding One possible approach would be to bind interface implementations to a value at the point it was created. In [the example above](#the-hashtable-problem), the @@ -182,7 +185,7 @@ its [goals](/project/goals.md). That being said, this could be a feature added later as opt-in behavior to either allow users to reduce code size or support use cases that require dynamic dispatch. -## Rejected alternative: manual conflict resolution +### Rejected variation: manual conflict resolution Carbon could alternatively provide some kind of manual disambiguation syntax to resolve problems where they arise. The problems with this approach have been From f16c6f878fb481302107087b82e8d4b067fd0943 Mon Sep 17 00:00:00 2001 From: Josh L Date: Tue, 3 Aug 2021 13:19:18 -0700 Subject: [PATCH 17/21] Updates re: context sensistivity --- docs/design/generics/appendix-coherence.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 8bb82510f8634..fdee2a2e0f8f4 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -153,7 +153,10 @@ differently depending on which of those two libraries are imported: on which it is. Furthermore, this means that the behavior of a file can depend on an import even -if nothing from that package is referenced explicitly. +if nothing from that package is referenced explicitly. In general, Carbon is +[avoiding this sort of context sensitivity](/docs/project/principles/low_context_sensitivity.md). +This context sensitivity would make moving code between files when refactoring +more difficult and less safe. ### Rejected variation: dynamic implementation binding @@ -203,9 +206,3 @@ either: like [dynamic implementation binding](#rejected-alternative-dynamic-implementation-binding) (though perhaps with a monomorphization cost instead of a runtime cost). - -In general, Carbon is -[avoiding this sort of context sensitivity](https://docs.google.com/document/d/1dLpEmUbE2_JQZ-pRNgEDnw6VuH4FXfURSYej6nPV6m0/edit#). -TODO: update to [this link](/docs/project/principles/low_context_sensitivity.md) -once [proposal #646](https://github.com/carbon-language/carbon-lang/pull/646) is -submitted. From 683b4cdc4a4a35563df1819e779bbb95ea0a3b40 Mon Sep 17 00:00:00 2001 From: Josh L Date: Thu, 12 Aug 2021 06:02:05 -0700 Subject: [PATCH 18/21] Add Swift alternative --- docs/design/generics/appendix-coherence.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index fdee2a2e0f8f4..1bced0777e7f8 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -16,6 +16,7 @@ This document explains the rationale for choosing to make - [Approach taken: coherence](#approach-taken-coherence) - [The "Hashtable Problem"](#the-hashtable-problem) +- [Rejected alternative: no orphan rule](#rejected-alternative-no-orphan-rule) - [Rejected alternative: incoherence](#rejected-alternative-incoherence) - [Incoherence means context sensitivity](#incoherence-means-context-sensitivity) - [Rejected variation: dynamic implementation binding](#rejected-variation-dynamic-implementation-binding) @@ -137,6 +138,22 @@ discusses the hashtable problem in the context of Haskell, and [this 2011 Rust followup](https://mail.mozilla.org/pipermail/rust-dev/2011-December/001036.html) discusses how to detect problems at compile time. +## Rejected alternative: no orphan rule + +In Swift an implementation of an interface, or a "protocol" as it is called in +Swift, can be provided in any module. As long as any module provides an +implementation, that implementation is +[used globally throughout the program](https://stackoverflow.com/questions/48762971/swift-protocol-conformance-by-extension-between-frameworks). + +In Swift, since some protocol implementations can come from the runtime +environment provided by the operating system, multiple implementations for a +protocol can arise as a runtime warning. When this happens, Swift picks one +implementation arbitrarily. + +In Carbon, we could make this a build time error. However, there would be +nothing preventing two independent libraries from providing conflicting +implementations. Furthermore, the error would only be diagnosed at link time. + ## Rejected alternative: incoherence ### Incoherence means context sensitivity From 19d7e875d7f0b91ed0550f2db81809d901698845 Mon Sep 17 00:00:00 2001 From: josh11b Date: Wed, 1 Dec 2021 12:26:03 -0800 Subject: [PATCH 19/21] Apply suggestions from code review Co-authored-by: Richard Smith --- docs/design/generics/appendix-coherence.md | 6 +++--- docs/design/generics/goals.md | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 1bced0777e7f8..6a7cec510db64 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -72,7 +72,7 @@ this: package SongHashArtistAndTitle; import SongLib; impl SongLib.Song as Hashable { - method (me: Self) Hash() -> UInt64 { ... } + fn Hash[me: Self]() -> u64 { ... } } ``` @@ -87,7 +87,7 @@ this: fn IsInHashSet( s: SongLib.Song, - h: Containers.HashSet(SongLib.Song)*) -> Bool { + h: Containers.HashSet(SongLib.Song)*) -> bool { return h->Contains(s); } ``` @@ -99,7 +99,7 @@ this: package SongHashAppleMusicURL; import SongLib; impl SongLib.Song as Hashable { - method (me: Self) Hash() -> UInt64 { ... } + fn Hash[me: Self]() -> u64 { ... } } ``` diff --git a/docs/design/generics/goals.md b/docs/design/generics/goals.md index 2c5684f00dc17..cf4a1a5bfef35 100644 --- a/docs/design/generics/goals.md +++ b/docs/design/generics/goals.md @@ -439,7 +439,7 @@ will necessarily be less incremental. ### Coherence We want the generics system to have the -[_coherence_ property](terminology#coherence), so that the implementation of an +[_coherence_ property](terminology.md#coherence), so that the implementation of an interface for a type is well defined. Since a generic function only depends on interface implementations, they will always behave consistently on a given type, independent of context. For more on this, see @@ -458,7 +458,7 @@ It also has a number of benefits for users: Carbon template on that type. The main downside of coherence is that there are some capabilities we would like -for interfaces which are in tension with having an orphan rule limiting where +for interfaces that are in tension with having an orphan rule limiting where implementations may be defined. For example, we would like to address [the expression problem](https://eli.thegreenplace.net/2016/the-expression-problem-and-its-solutions#another-clojure-solution-using-protocols). We can get some of the way there by allowing the implementation of an interface From d47dd80d038401f0fcade8165fe291b39097e5cb Mon Sep 17 00:00:00 2001 From: Josh L Date: Mon, 6 Dec 2021 14:07:49 -0800 Subject: [PATCH 20/21] Add link to Rust orphan rule concerns --- docs/design/generics/appendix-coherence.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 6a7cec510db64..33926cda4a7ab 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -29,13 +29,19 @@ This document explains the rationale for choosing to make The main thing to understand is that coherence is a desirable property, but to get that property we need an orphan rule, and that rule has a cost. It in particular limits how much control users of a type have over how that type -implements interfaces. There are two main use cases to consider: +implements interfaces. There are a few main problematic use cases to consider: - Selecting between multiple implementations of an interface for a type. For example selecting the implementation of the `Comparable` interface for a `Song` type to support "by title", "by artist", and "by album" orderings. - Implementing an interface for a type when there is no relationship between the libraries defining the interface and the type. +- When the implementation of an interface for a type uses an associated type + that can't be referenced from the file or files where the implementation is + allowed to be defined. + +These last two cases are highlighted as concerns in Rust in +[Rust RFC #1856: orphan rules are stricter than we would like](https://github.com/rust-lang/rfcs/issues/1856). Since Carbon is bundling interface implementations into types, for the convenience and expressiveness that provides, we satisfy those use cases by @@ -183,8 +189,10 @@ implementation of the `Hashable` interface for `Song` would be fixed for the `song_set` `HashSet` object based on which implementation was in scope in the body of the `SomethingWeirdHappens` function. -This idea is discussed briefly in section 5.4 on separate compilation of -[this proposal for implementing "Indiana" C++0x concepts proposal](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.9526&rep=rep1&type=pdf). +This idea is discussed briefly in section 5.4 on separate compilation of WG21 +proposal n1848 for implementing "Indiana" C++0x concepts +([1](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.9526&rep=rep1&type=pdf), +and [2](https://wg21.link/n1848)). This has some downsides: From ba53310503069a95f8edc1428cdf054dd3db6a96 Mon Sep 17 00:00:00 2001 From: Josh L Date: Mon, 6 Dec 2021 17:30:12 -0800 Subject: [PATCH 21/21] Add suggested caveat of dynamic dispatch --- docs/design/generics/appendix-coherence.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/design/generics/appendix-coherence.md b/docs/design/generics/appendix-coherence.md index 33926cda4a7ab..c809014e4990f 100644 --- a/docs/design/generics/appendix-coherence.md +++ b/docs/design/generics/appendix-coherence.md @@ -207,6 +207,10 @@ This has some downsides: the witness table representing the implementation with the object, since it varies instead of being known statically. - It is slower to execute from dynamic dispatch and the inability to inline. +- In some cases it may not be feasible to use dynamic dispatch. For example, + if an interface method returns an associated type, we might not know the + calling convention of the function without knowing some details about the + type. As a result, this doesn't make sense as the default behavior for Carbon based on its [goals](/project/goals.md). That being said, this could be a feature added