"hard" type alias or Inheritance from a concrete type

[SimonDanisch]

I've addressed this feature multiple times, but never really got feedback on it, so I decided to open an issue for it, as I keep coming back to it as a very elegant solution to many problems.
I hope you can answer at least, how feasible this is, what needs to be done for it (so that I can maybe implement it myself, if no one has the time), or explain me why this is a silly idea.

What I want:

# Introducing a new type, in a typealias like fashion.
# first syntax idea
hardtypealias RGB{T <: FloatingPoint} NTuple{3, T}
# second syntax idea
immutable Meter{T <: Real} <: T
# third syntax idea
concrete FixedMatrix{T, M, N} <: NTuple{NTuple{M, T}, N} # in constrast to abstract

Intended semantic: The first type (e.g. RGB) is a either subtype of the second (e.g NTuple), or has the type Union(RGB, NTuple).
I'm not sure what makes more sense. But it should solve the following problem:

There is an already existing type and one wants to add functions to it, without adding them to the original type.

This behavior is wanted in a myriad of cases and I've seen workarounds for this situation in many packages (including mine).

Examples

A few examples, which can be elegantly implemented with this feature, which assume, that NTuple implements common, simd accelerated, vector operations

Adding functions to an existing type:

# under the assumption, that NTuple implements common vector operations
immutable RGB{T} <: NTuple{3, T}

# under the assumption, that getfield can be extended
Base.getfield(c::RGB, ::Field{:r}) = c[1]
Base.getfield(c::RGB, ::Field{:g}) = c[2]
Base.getfield(c::RGB, ::Field{:b}) = c[3]
# which gives you the access to RGB which you'd expect from it.
# could also be implemented with a macro: @accessors RGB :r => 1, :g => 2, :b => 3 
# So this works:
a = RGB(0.1,0.2,0.3)
a+a # is implemented by NTuple
a.r # implemented by RGB
#Should not work:
(1,2,3).r

To further dwell on the point, that we can have a diversity of simd accelerated Vector types, with minimal re-implementation of common functionality, here is another example:

Adding Semantic to generic Types

immutable Point{T, N} <: NTuple{N, T}
immutable RGB{T, N} <: NTuple{N, T}

immutable Red{T <: Real} <: T
immutable Green{T <: Real} <: T
immutable Blue{T <: Real} <: T

red = Red(1f0) # -> this will behave exactly like a normal Float32 value 
# making it unnecessary, to re-implement all the operations on it

immutable X{T <: Real} <: T
immutable Y{T <: Real} <: T
immutable Z{T <: Real} <: T

Base.getfield(c::RGB, field::Type{Red}) = Red(c[1])
...
Base.getfield(p::Point, field::Type{X}) = X(p[1])
...
Base.getfield(p::Matrix{RGB}, field::Type{Red}) = ... # -> Matrix{Red}
Base.getfield(p::Matrix{Point}, field::Type{X}) = ... # -> Matrix{X}
...
image = [RGB(1f0, 0f0,0f0) for i=1:512, j=1:512]
points = [Point(1f0, 0f0,0f0) for i=1:512, j=1:512]

redchannel = image.Red
zplane = image.Z

Now the big magic, won by more type information

visualize(image) #  -> rgb image display
visualize(points) # -> point cloud 
visualize(redchannel) # I can actually infer what the user wants to see: a red intensity image
visualize(zplane) # Again, it's relatively clear what is expected here: a heightfield

This would be pretty much the paradise for any visualization library. Also, visual debugging is made a lot easier, as the debugger doesn't need any magic to infer how it needs to visualize some random variable.
But this matters for more than that, as it becomes easier to do the correct thing with any value.
I you want to see something else, for example the pixel of the image in RGB space, you can simply reinterpret the RGB to Point (which has O(1)), which can be useful to make out color clusters.

Let function bodies only do, what is implied by their function name

Compare these two implementations:

#First implementation of matrix multiplication
function (*)(a::Matrix, b::Matrix) 
  @assert size(a, 2) == size(b, 1) #This isn't part of the multiplication code
  #matrix multiplication code
end
# This is probably a matter of taste, but I like this implementation more:
FixedSizeArray{SZ, T, N} <: Array{T,N}
Base.call{T,N}(::Type{FixedSizeArray}, x::Array{T,N}) = FixedSizeArray{size(x), T,N}(x)
function (*)(a::Matrix, b::Matrix) 
 return (FixedSizeArray(a) * FixedSizeArray(b))
end
# Like this, in the function body is only, what is really part of the matrix multiplication:
function (*)(a::FixedMatrix{T, N, M}, b::FixedMatrix{T, M, P})
 # code for multiplication of matrices with well suited dimensions
end
function (*)(a::FixedMatrix, b::FixedMatrix)
# code for multiplication of matrices with arbitrary dimensions
# which usually throws an error
end

This all is mostly graphic related, which isn't a big surprise considering my background. But I'm pretty sure, that there are good use cases for other fields ;)
But I better not iterate more use cases here, as this would turn even more into a novel.

Hope we can realize this in some way!

Best,
Simon

[JeffBezanson]

See also #2248

[JeffBezanson]

I don't understand "Let function bodies only do, what is implied by their function name". The example seems to be restricting matrix multiply to arrays with appropriate sizes, using types instead of an == check? That's fine, we can totally do that. (Though one should not use an assert here, but a check that throws a DimensionMismatch.)

[JeffBezanson]

Here is my guess as to the single biggest difficulty. Let's say we have:

immutable X{T <: Real} <: T

...

function f(x::Float64, y::Float64)
  ...
  convert(Float64, g(x+1, y-1))
end

f(X{Float64}(1.0), X{Float64}(2.0))

So X is a Float64, so we can call f on it. But f was written assuming its arguments are Float64, and it tries to return something of the same type. Is it ok for f to return a Float64 (exactly, not an X, ... see it's a bit hard even to talk about)? Or should we assume that g will return an X, in which case the convert(Float64, ...) is a no-op? But in that case we've subverted f's intentions: it tried to get a Float64, but it got something else.

The reason I think this is the fundamental problem is that it holds no matter how we make f applicable to X --- i.e. whether X is a subtype of Float64, or we somehow arrange for f to also be defined for X, without X actually being a subtype.

[SimonDanisch]

Oh yes, you pointed this out another time, sorry to make you repeat yourself.
But I just can't stop thinking, that we absolutely must have this, even though, that this is a tough one....
Not having this is such a big source of code duplication.
I guess that there are only inconvenient solutions for this.
Is there a possibility of compiling a specialized version of f, which replaces the Float64 values with X{Float64}, offering another code path?
I guess this would involve some special treatment of these types in function calls....

[nalimilan]

Possibly related to the discussion we had about f(::Nullable, ...) at #9446 (comment) One could imagine defining call(f::Function, x::X{T}, ...) = convert(X{T}, f(convert(T, x), ...)), i.e. call the function defined for the parent type, and convert the result to the child type.

[SimonDanisch]

Ha! Yeah, I like the idea of overloading call(::Function, ...), especially for bench marking reasons.
But isn't it kind of complicated in this case, as you actually have to search for occurrences of X in any of the parameters? And then, to correctly do this, follow the argument down its path to figure out, if it has to be converted in the end?

[SimonDanisch]

I just realized how truly evil jeff's example is.
There has to be magic involved, to figure out his case correctly.

function f(x::Float64, y::Float64)
  ...
  convert(Float64, g(x+1, y-1))
end

Doesn't imply

function f(x::X{Float64}, y::Float64)
  ...
  convert(X{Float64}, g(x+1, y-1))
end

Cruel world...
Well, this could be a case for using more parametric types in your function:

function f{T <: Float64}(x::T, y::T)
  ...
  convert(T, g(x+1, y-1))
end

Looks a lot more managable

[SimonDanisch]

To come back to the other question:
"Let function bodies only do, what is implied by their function name"
(*)(a::Matrix, b::Matrix) is called (implicitly) multiply, and not check_matrix_dimensions_and_multply().
Which is a subtle case. There are others, where it's a lot clearer, that there is too much code in a single function, which doesn't belong there.
Julia is especially well suited for defining clean methods, which only solve a single, well defined problem. Using this pattern more often seems to me a natural continuation of using multiple dispatch to solve more control flow problems.

Also, the size of the array becomes a constant inside the body correct?
Which might be nice for optimizations.
On the other side, I just realize, that you would recompile the same code a lot of times, just with different sizes of the array inlined...If you work with a lot of differently sized matrices, this could be horrible... Or is this somehow optimized?

[tknopp]

All this is a lot about type hierarchies and the question weather functions have been defined for concrete types or not.

If one has an interface for a type (say FixedVector) it will likely only have less than 5 methods (setindex!, getindex, size, ...?) If all other methods are implemented ontop of the abstract type one will always gain everything when subtyping from the abstract type and implementing the interface.

So as pointed in #7568 we may simply implement the methods for the fixed size array and can later have "the real one" based on tuples. The only issue was construction but haven't we even solved that with the call syntax?

[tknopp]

And on letting the size of an array be a type parameter: This would not work for vectors currently (we have resize) but in ND for N>=2 there don't seem to be an issue.

[toivoh]

I'm trying to understand the examples, and to me, reusing an existing type
like this but adding behavior to it seems like subtyping.

I'm sure some of the types of issues that you bring up can be solved by
defining the functionality that you want to reuse at the appropriate level
and then creating new subtypes.

But seen in that light, a number of your examples include subtyping of
concrete types, which I believe is not going to happen, for good reasons.
This in a way also includes subtyping NTuple; all subtypes of it are tuple
types.

I think that solving many of the issues that you bring up has to start
through thinking about the problem in a different way.

[SimonDanisch]

I think you're right.
So we should probably start thinking about what the real problem is.
We probably can all agree, that it would be very beneficial to have the kind of features I mention.
Even if not, there are already a lot of Packages working around this problem, meaning there are Julia users that implicitly want something like this.
So where did we go wrong?
A lot of the problems I mention could be solved by having more functions defined on an abstract type, making it unnecessary to inherit from a concrete type:

# I must admit, I don't really know what I'm doing here, but if something relies on the exact number 
# of bits, this type hierarchy could achieve this, without using concrete bitstypes:
abstract FloatingPoint{Bits}
bitstype 32 Float32 <: FloatingPoint{32}
#e.g.
+{T <: FloatingPoint}(x::T, y::T) = box(T, add_float(unbox(T, x),unbox(T, y)))
# Now I can define my own floatingpoint, with all the relevant functionality working:
abstract Meter{T} <: FloatingPoint{T}
bitstype 32 Meter32 <: Meter{32}

This is just a quick idea, someone with more knowledge about Julia's internals should probably flesh out something better.

While similar concepts apply to NTuple it seems a little bit more complicated to me. But the reasons behind this could be simply, that I've been more involved with FixedSizeVectors. I hope that at some point, valid SPIR code can be emitted for FixedSizeArrays, so we can compile to the GPU seamlessly. This on its own further complicates issues.
Discussions about this are on #7568

[mauro3]

I once asked a similar question on the mailing list:
https://groups.google.com/d/msg/julia-dev/v8B1tI_NB5E/tk98D0iKopYJ

In Haskell these things are somewhat possible, constructed with newtype, whereas a type-alias (just like Julia has) is done with type (see). But as there is no subtyping in Haskell the newtype (I think) only copies the structure of the type but none of the behavior. But here, we're after the behavior...

[RauliRuohonen]

But as there is no subtyping in Haskell the newtype (I think) only copies the structure of the type but none of the behavior. But here, we're after the behavior...

@mauro3 Actually, you can use the deriving clause for that, like this:

$ ghci -XGeneralizedNewtypeDeriving
Prelude> newtype Red = Red Double deriving (Eq, Ord, Num, Fractional, Floating, Real, RealFrac, RealFloat, Show)
Prelude> Red 3 / Red 4
Red 0.75
Prelude>

For example, there's a type class Num T that has a function (+): T -> T -> T and there's an instance for T=Double. In principle GHC unwraps inputs and wraps outputs as needed to produce an instance for T=Red, which is straightforward as Red is just a wrapper for Double and you do it for every T in the signature. Julia could do this too if it had type class declarations, but without them it's ambiguous, as Jeff pointed out. If the output of f(x, y) is Float64, you either should or should not convert depending on whether the signature is T -> T -> T or T -> T -> Float64. The latter might be e.g. the probability that two reds are distinguished by a human, and you shouldn't convert it into Red. Type classes would be unlikely to work for Julia because its type signatures are far more complex than Haskell's. (e.g. (+)(::Date, ::Dates.Day) does not match T -> T -> T)

Instead of type classes, in Julia you could define macros like @deriving_Num, @deriving_Ord, and then define a @deriving (X, Y, ...) macro that calls each of @deriving_X, @deriving_Y; and use them like this:

@deriving (Ord, Num) immutable Red
    value :: Float64
end

and that would define methods for the desired functions with appropriate unwrappings and wrappings. To mimic Haskell's constraints like sort: Ord T => [T] -> [T], one could have Ord be a trait and @deriving_Ord would add Red to that trait (i.e. Cmp{Red, Red} in your Traits.jl).

In that vein, you could actually define @deriving to work with traits only. Traits are a lot like type classes, and you don't need the full signature for wrapping/unwrapping, you only need a boolean for each argument/return value telling you whether you should do it or not when it's of (un)wrappable type. This may well be more suitable for Traits.jl than for Julia core.

[vtjnash]

closing as this is the "fragile base class problem", as jeff pointed out above