[DISCUSSION] High-level scope and idea of Arrow Exact

[ILIYANGERMANOV]

The idea of Arrow Exact is to provide common value class wrappers required for defining precise, type-safe, and explicit data models. Along with those value classes, we'll also need to:

• Provide safe (Option or Raise or null) and unsafe ways for creating them.
• Each value class will guarantee certain invariants/properties.
• Overload relevant operators: e.g. +, -, *, /, %
• Extension functions for mapping the wrapped value
• Transformation between the Arrow Exact types (e.g. PositiveDouble -> NonNegativeDouble)
• Nice "toString()" implementation

Suggestions for Arrow Exact types:

• PositiveDouble
• NonNegativeDouble
• PositiveInt
• NonNegativeInt
• NotBlankString
• NotBlankTrimmedString
• Percent
• FiniteDouble

There are many more possible but we should decide which ones makes practical sense to have.

Here's an example implementation of some of them:
https://github.com/Ivy-Apps/ivy-wallet/tree/multiplatform/core/src/commonMain/kotlin/ivy/core/data/primitives

With their respective invariants (properties):
https://github.com/Ivy-Apps/ivy-wallet/tree/multiplatform/core/src/commonTest/kotlin/ivy/core/data/primitives

[ILIYANGERMANOV]

I'm also thinking about a Text API where the user of Arrow Exact can provide a custom validation + transformation API. Ofcourse, we can pre-define common Text types like Text<Email>, Text<NotBlankAndTrimmed>, etc

Here's a quick and ugly prototype of it:

interface Constraints {
  val name: String
  val requirement: String

  fun apply(text: String): Option<String>
}

@JvmInline
value class Text<C : Constraints> private constructor(
  private val value: String,
) {
  companion object {
    fun <C : Constraints> fromString(text: String, constraints: C): Option<Text<C>> =
      constraints.apply(text).map(::Text)

    fun <C : Constraints> unsafe(
      text: String,
      constraints: C,
      tag: String? = null
    ): Text<C> = fromString(text, constraints).fold(
      ifEmpty = {
        throw ArrowExactTextConstraintError(
          constraints = constraints,
          violatingValue = text,
          tag = tag,
        )
      },
      ifSome = ::identity
    )

    operator fun <C : Constraints> invoke(text: String, constraints: C): Text<C> =
      unsafe(text, constraints)
  }
}

class ArrowExactTextConstraintError(
  constraints: Constraints,
  violatingValue: String,
  tag: String?
) : ArrowExactConstraintViolationError(
  constraint = constraints.name,
  requirement = constraints.requirement,
  violatingValue = violatingValue,
  tag = tag,
)

object Email : Constraints {
  override val name: String
    get() = "Email"
  override val requirement: String
    get() = "has @ and is trimmed"

  override fun apply(text: String): Option<String> = if ('@' in text) Some(text.trim()) else None
}

object PersonName : Constraints {
  override val name: String
    get() = "PersonName"
  override val requirement: String
    get() = "not blank and trimmed"

  override fun apply(text: String): Option<String> {
    return text.trim().takeIf { it.isNotBlank() }?.some() ?: None
  }
}

data class User(
  val email: Text<Email>,
  val firstName: Text<PersonName>,
  val lastName: Text<PersonName>,
)

fun main() {
  val user = User(
    email = Text("iliyan.germanov971@gmail.com", Email),
    firstName = Text("Iliyan  ", PersonName),
    lastName = Text("Germanov", PersonName)
  )

  println(user)
}

[CLOVIS-AI]

Looks interesting. I wonder how well it generalizes to any type (all your examples are for String).

Constraint.name should probably have a default implementation of get() = ::class.simpleName.

Maybe the library should be split into two modules, one that provides the utilities to set everything up (e.g. your Constraint interface and the related methods), and another module that provides the constraints we decide to create, so the mechanism can be used in other libraries more easily.

[ILIYANGERMANOV]

Good point @CLOVIS-AI! I agree with you - tried generalizing the Constraint API but lost the value class inline optimization - maybe this can be worked around or we'll just duplicate classes in the library. Here's how a generalized implementation might look like.

Constrained API

interface Constraint<A> {
  val name: String
    get() = this::class.simpleName ?: ""
  val requirement: String

  fun apply(value: A): Option<A>
}

open class Constrained<A, Cons : Constraint<A>>(
  val value: A,
) {
  companion object {
    fun <A, Cons : Constraint<A>> fromValue(
      value: A, constraint: Cons
    ): Option<Constrained<A, Cons>> =
      constraint.apply(value).map(::Constrained)

    fun <A : Any, Cons : Constraint<A>> unsafe(
      value: A,
      constraint: Cons,
      tag: String? = null
    ): Constrained<A, Cons> = fromValue(value, constraint).fold(
      ifEmpty = {
        throw ArrowExactConstraintViolationError(
          constraint = constraint.name,
          requirement = constraint.requirement,
          violatingValue = value,
          tag = tag,
        )
      },
      ifSome = ::identity
    )

    operator fun <A : Any, Cons : Constraint<A>> invoke(
      value: A, constraint: Cons
    ): Constrained<A, Cons> = unsafe(value, constraint)
  }
}

// This is bad! Needs improvement but sharing just for the sake of discussion
fun <A : Any, Cons : Constraint<A>> Constrained<A, Cons>.fmap(
  constraint: Cons,
  f: (A) -> (A)
): Option<Constrained<A, Cons>> = Constrained.fromValue(f(value), constraint)

Demo

object PositiveIntCons : Constraint<Int> {
  override val requirement = "positive, > 0"

  override fun apply(value: Int): Option<Int> {
    return if (value > 0) Some(value) else None
  }
}
typealias PositiveInt = Constrained<Int, PositiveIntCons>

object NotBlankTrimmedStringCons : Constraint<String> {
  override val requirement = "not blank after trim"

  override fun apply(value: String): Option<String> {
    return value.trim().takeIf { it.isNotBlank() }?.some() ?: None
  }
}

class NotBlankTrimmedString(value: String) : Constrained<String, NotBlankTrimmedStringCons>(value)

object CitizenIdCons : Constraint<Int> {
  override val requirement = "divisible by 2"

  override fun apply(value: Int): Option<Int> {
    return if (value % 2 == 0) Some(value) else None
  }
}

class CitizenId(value: Int) : Constrained<Int, CitizenIdCons>(value)

data class User(
  val id: CitizenId,
  val name: NotBlankTrimmedString,
  val age: PositiveInt,
)

fun demo() {
  val user = User(
    id = CitizenId(4), //PositiveInt(4) won't work which is good
    name = NotBlankTrimmedString("Iliyan"),
    age = PositiveInt(26),
  )
}

[pacher]

I see as a goal a "type-safe"(?) arithmetic, where e.g. division by zero would raise a typed error instead of throwing an exception, and integer overflow would be an error instead of silently overflowing.

[ILIYANGERMANOV]

I see as a goal a "type-safe"(?) arithmetic, where e.g. division by zero would raise a typed error instead of throwing an exception, and integer overflow would be an error instead of silently overflowing.

Yes, that should be one of its goals but IMO it should be way more than just arithmetic.
(personal opinion) Here's how I see it:

Following this FP principle:

Data (Types) >> Calculations (pure functions) >> Actions (side effects)

The stricter we define our types => the less validation and cases our domain logic would have to handle.

Using only Arrow Core, the NonEmptyList isn't enough to do so. I hope that Arrow Exact can solve this for me:

• provide built-in commonly used types like PositiveInt, NonNegativeDouble, etc
• provide a way for me to define "constrained" domain types (see the example below)
• write as little boiler-plate possible for the above two

---

Example:

data class Player(
  val alias: NotBlankTrimmedString, // let's say we don't care about the max length
  val username: Username, // constrained to [a-z][0-9]_
  val winSteak: NonNegativeInt,
  val level: PlayerLevel, // constrained in [1,120]
  val winRate: Percent,
)

We can say that the Player above is well-defined. I imagine Arrow Exact to help me with the definition of all the types used and provide me with the boiler-plate needed.

💡 [just for brainstorming] Many times this Player may come from an API - it would be extremely cool if Arrow could offer an extension library for parsing types such as player with KotlinX serialization.

---

I tried defining my data model in Ivy Wallet following this principle... And ended up needing to manually create a lot of primitives like NonNegativeInt (transactions count), and PositiveDouble (transaction amount) and that's why I suggested this feature request in the #Arrow Slack. I believe that if we all combine our brainpower and expertise we can come-up with an awesome Kotlin library that will make our code: safer, simpler, and easier to write and maintain.

[kyay10]

We could have Constraint<A> alongside IntConstraint, FloatConstraint etc and then value class Constrained<A, Cons: Constraint<A>> and value class ConstrainedInt<Cons: IntConstraint> etc. and all Constrained types shall be defined as type-aliases (e.g. PositiveInt = ConstrainedInt<PositiveIntCons>)

We could also look at https://github.com/ustits/krefty for some prior art.

[kyay10]

Defining intersecting constraints will be tough, though. There are some (technically unsupported) compiler tricks to create intersection types from thin air, but I don't know if that's the most apt thing here.

[vandeven]

Is there a reason to use Option over kotlins nullability system? Using the nullability of Kotlin makes sure we do not need a dependency on Arrow-Core for now.
Also I had the question of what would this library offer over making these types yourself (since making a value class with "require" would give the same result as the create unsafe option), which would mean making yourself would be quite trivial without the need to depend on a library.

[ILIYANGERMANOV]

Thank you for joining the discussion @vandeven!

Is there a reason to use Option over kotlins nullability system? Using the nullability of Kotlin makes sure we do not need a dependency on Arrow-Core for now.

In reality, null vs Option depends on the person using it and their use-case. IMO, Option is a better choice because Option is a monad and provides nice extensions like the option {} effect, flatMap and all the goodies from the FP world. Here we're talking Arrow and adoption of FP so Option wouldn't be a bad choice and I see no hard if we introduce people to Arrow Core.

Ofc, we can also choose null or provide both. That's up for a discussion - I personally vote for Option - @nomisRev and the other Arrow maintainers will provide further feedback after KotlinConf.

Example:

// With Option we can use the entire Arrow Raise API and all the goodides
context(Raise<String>)
fun optionDemo() = option {
    val a = PositiveDouble.fromDouble(0.5).bind()
    val b = PositiveDouble.fromDouble(-5.0).bind()
    ensure(a > b)
}

Note: I am biased because I am full on FP. Yes can achieve the above things with ?: error("") and reuqire but I prefer the FP error handling that Arrow promotes.

Also I had the question of what would this library offer over making these types yourself (since making a value class with "require" would give the same result as the create unsafe option), which would mean making yourself would be quite trivial without the need to depend on a library.

Yes, you totally can. However, if you want really "exact" and precise data models you'll need a lot of those value class wrappers. I implemented a lot of those for my Ivy Wallet project:
https://github.com/Ivy-Apps/ivy-wallet/tree/multiplatform/core/src/commonMain/kotlin/ivy/core/data/primitives

But still they're not enough and I haven't implemented all the FP extensions that I need. Definitely, I'd need something like the Constrained API that I suggested above to precisely define my domain data model. Not thta I can write it myself for a day or two but it would be nice if I had in an Arrow library.

For some types it would be nice if we provide operators overloading (e.g. +, -, *, /), docs describing the edge cases (e.g. Double.MAX_VALUE + 1, Double.MAX_VALUE * 2, Double.POSITIVE_INFINITY). There are many common values classes that can also be built-in like Percent, NotBlankTrimmedString, Email, Domain, etc. We need to scope what's worth it and what not.

Also, if we provide a good API for "refining" / constraining new types it would be nice.

[pacher]

From the arrow docs:

Typically, when working in Kotlin, you should prefer working with nullable types over Option as it is more idiomatic. However, when writing generic code, we sometimes need Option to avoid the nested nullability issues, or when working with libraries that don't support null values such as Project Reactor or RxJava.

There is no nested nullability or other libraries here. Also, Option or null implies the absence of value, which is not the case. It should be at least Either with a typed error clearly stating what went wrong and which constraints got violated. Otherwise this information is lost and that is not good. If I understand arrow correctly these functions should return the desired non-nullable type without wrappers and have a Raise context receiver. Then there could be a dsl that returns Either or something else. No .bind() needed. Something like

fun getMyDouble(): Either<ExactError, PositiveDouble> = either {
    val a = PositiveDouble.fromDouble(0.5)
    val b = PositiveDouble.fromDouble(-5.0)
    a + b
}

[ILIYANGERMANOV]

From the arrow docs:

Typically, when working in Kotlin, you should prefer working with nullable types over Option as it is more idiomatic. However, when writing generic code, we sometimes need Option to avoid the nested nullability issues, or when working with libraries that don't support null values such as Project Reactor or RxJava.

There is no nested nullability or other libraries here. Also, Option or null implies the absence of value, which is not the case. It should be at least Either with a typed error clearly stating what went wrong and which constraints got violated. Otherwise this information is lost and that is not good. If I understand arrow correctly these functions should return the desired non-nullable type without wrappers and have a Raise context receiver. Then there could be a dsl that returns Either or something else. No .bind() needed. Something like

fun getMyDouble(): Either<ExactError, PositiveDouble> = either {
    val a = PositiveDouble.fromDouble(0.5)
    val b = PositiveDouble.fromDouble(-5.0)
    a + b
}

Either with typed error sounds good to me 👍 This way, for the code snippet above, we can know exactly what went wrong - e.g. NonPositive value, Overflow, Infinity, etc. I suggested Option just because I personally don't care about the exact error. All I need to know is whether it's positive and finite number or not. That being said, Either/Raise might be a better option.

[vandeven]

I would also agree on an Either with full error information.
We could also add extension functions for the types to do unsafe parsing.
I can can also help a bit with making this lib if you like :) I wanted to contribute for quite some time

[ILIYANGERMANOV]

I would also agree on an Either with full error information. We could also add extension functions for the types to do unsafe parsing. I can can also help a bit with making this lib if you like :) I wanted to contribute for quite some time

@vandeven Ofc, any contribution is more than welcome! Feel free submit PRs / comments with prototypes, ideas - everything is welcome! :) And much appreciated!

[CLOVIS-AI]

We could have Constraint alongside IntConstraint, FloatConstraint etc and then value class Constrained<A, Cons: Constraint> and value class ConstrainedInt<Cons: IntConstraint> etc.

It has to be a running joke at this point that all Arrow projects start by complaining that type classes are not a language feature 😓

I'm a bit worried about library size and width, especially when creating those types yourself is not that hard. Plus, most use cases I can think of for this library are domain-specific (e.g. I have an Email and Password value classes, and for obvious reasons I want them to be implemented in the project itself).

To ensure we are talking about the same thing, here is an example of what my codebase currently looks like (assuming context receivers):

value class Password(val text: String) {

    init {
        require(text.length < Failures.TooShort.minLength) { Failures.TooShort.toString() }
        require(text !in Failures.TooSimple.blacklist) { Failures.TooSimple.toString() }
    }
    
    sealed class Failures {
        object TooShort : Failures {
            val minLength = 8
            
            override fun toString() = "A password must be at least $minLength characters long"
        }
        
        object TooSimple : Failures {
            val blacklist = listOf("123456", "123456789")
        
            override fun toString() = "The provided password is too simple"
        }
    }
}

context(Raise<Password.Failures>)
fun passwordOf(text: String): Password {
    ensure(text.length < Password.Failures.TooShort.minLength) { Failures.TooShort }
    ensure(text !in Password.Failures.TooSimple.blacklist) { Failures.TooSimple }

    return Password(text)
}

My primary goal here is to reduce the boilerplate to declare these kinds of types. What do you think?

@ILIYANGERMANOV Can you rewrite this example in your envisioned way to do things, to compare?

[ILIYANGERMANOV]

@CLOVIS-AI I have a similar use-case: constrained domain types validated by business rules (e.g. AssetCode, AccountName, Email) + built-in primitives like NonNegativeInt, PositiveDouble, NotBlankTrimmedString.

I don't know or have a strong opinion how the final design should look like but I want the following:

• Invariant: guarantees any arbitrary properties for the constrained type A
• Type-safety: raises compile-time error if something isn't A
• No boilerplate: easy to define, create and use new constrained types

Your Password is a very good example. I see two possible paths forward Code generation (KSP) vs No code generation.

Code generation (KSP)

Definition

@ArrowExactType(name = "Password")
object PasswordDefiniton {
  context(Raise<Password.Failure>)
  fun constrain(value: String): String {
     ensure(text.length > MIN_LENGTH) { Failure.TooShort }
     ensure(text !in blacklist) { Failure.TooSimple }

     // add any transformations that you might want to do
     return value 
  }
}

Generated

// Generated by KSP
@JvmInline
value class Password private constructor(val value: String) {
   companion object {
      context(Raise<ArrowExactError<Password.Failure>>)
      fun from(val value: String): Password {
         // ...
      }

      fun unsafe(value: String): Password = recover(::from) {
         throw ArrowExactRuntimeError(...)
      }

      operator fun invoke(value: String): Password = unsafe(value)
   }
}

context(Raise<ArrowExactError<Password.Failure>>)
fun Password.fmap(f: (String) -> String): Password

Usage:

val password = Password("123abcd!") // unsafe, runtime err - use only when certain

recover({
  Password.from(input) // Raise API, safe
}) { error -> // ArrowExactError<Password.Failure>
  when(error.val) {
     Password.Failure.TooShort -> // ...
     // ...
  }
}

How do you imagine it to work? What do you think?

P.S. This is a pseudo-code written at 1am from my phone for the purpose of illustration and discussion. Don't take it literally - please share your honest thoughts, crazy ideas or any use-cases that you want this library to solve for you :)

[nomisRev]

Hey all,

Sorry for the late (and short) reply. After KotlinConf I am again travelling for work, and had little time to invest into this. It's a quite interesting discussion above here ☺️

1. On the topic of Option vs nullable

IMO, nullable is the right choice here. Rationale is that nothing here will suffer from the nested nullable problem as explained here. You can still benefit of the DSL using nullable, and you can quite easily lift to null using A?.toOption().

2. Design

I think exposing multiple signatures would be ideal, and potentially we can support xOrThrow, xOrNull, xOrNone, xwithRaiseand perhapsgetOrLeftforEitheror similar. Wrapping inIllegalStateException, RefinedNotMatchedorExactExceptionfororThrow` makes sense.

I think x is ideally a named method like create or from. This would be most flexible, but it removes the ability to use constructor syntax.

I personally am very interested in Krefy it looks quite nice, and it allows composing different refinements and predicate 🤔 Sadly it seems a development has stopped on the library.

[CLOVIS-AI]

@nomisRev Indeed, Krefy looks quite good. Using it in libraries will be hard though, because it must be part of the public API (because of inheritance). It also (probably?) stops you from implementing abstract classes (though I guess our idea with value class does too).

@ILIYANGERMANOV I'm not a fan of KSP. It's extremely hard to debug for downstream users when it goes wrong, and it often has setup problems with IDEA & multiplatform.

[ILIYANGERMANOV]

@CLOVIS-AI yeah totally, I'm still struggling to make Arrow Optics' KSP work on my multiplatform project. So definitely, we should avoid it.

@nomisRev good points 👍 I agree with them and indeed Krefty has a nice design and powerful composition. I might give Krefty a try in Ivy Wallet.

Only the infix extension refined feels weird to me and the fact that it throws an exception. IMO, a library that I'd like to use has similar mechanism for defining new types like Krefty + the xOr??? that you mentioned. Also, I'd like it to ship common built-in ones like for example PositiveDouble.

I haven't look at Krefty in detail but does it allow transformations of the wrapped value? On top of predicates, it would make my life easier if I can define types like NotBlankTrimmedString which will automatically do the trimming and then check for blankness. This trimming in Android is literally on every input field.

[ILIYANGERMANOV]

It would be nice if we can invite Ustits, Krefty's creator to collaborate on this. As a user, I have a big need for such a library because we practice DDD, and constraining our types is a boilerplate we write in every project.

[nomisRev]

I can invite @Ustits, I'm. not sure if he's still interested on working on this or what his thoughts are on Krefty's.

I haven't look at Krefty in detail but does it allow transformations of the wrapped value?

I'm not sure, how do you currently deal with this? You need to re-check the properties on map, right?

it would make my life easier if I can define types like NotBlankTrimmedString which will automatically do the trimming

I think the idea is that we can compose functions, or "type-level operations" into a new type. Ideally, we could do something like. @JvmInline value class NotBlankTrimmedString(val value: String) : Constrain<All<NotBlank, Trimmed>, String>, or:

value class  NotBlankTrimmedString private constructor(val value: String)  {
  companion object : Constrain<All<NotBlank, Trimmed>, String, NotBlankTrimmedString>
}

Where Constrain<All<NotBlank, Trimmed>, String, NotBlankTrimmedString> projects a DSL over the companion object to expose create, createOrThrow, createOrRaise, createOrEither, createOrNull, ...

This is what I have in my mind, but I'm not sure how we can currently implement such a thing. I think it's possible but I haven't been able to dig my hands into it since I'm quite overloaded 😵

[ILIYANGERMANOV]

I'm not sure, how do you currently deal with this? You need to re-check the properties on map, right?

Yes, every mapping may break the Invariant and recheck is required. What I do is simply first map and then automatically check the predicates again.

---

@nomisRev your proposals look good to me! 👍 I like the companion object DSL especially. Just as an idea, @pacher also suggested having positive { ... } DSL where the ... block of code must return a PositiveX constrained type.

Regarding the implementation of

value class  NotBlankTrimmedString private constructor(val value: String)  {
  companion object : Constrain<All<NotBlank, Trimmed>, String, NotBlankTrimmedString>
}

It might be possible but I don't know exactly how we can do that. IMO, worst case we can use KSP. With KSP we'll have the freedom to generate tons of helpful stuff w/o limitations. The downside is that it'll make the library integration pain in the ass.

[ustitc]

Hi all, very pleased that my refinement experiments in krefty have been noticed, considering the fact that the library was mostly inspired by refined-types from arrow-meta 😄 Will be glad to take part in the project .

@nomisRev I like the idea of companion object and that the scope of x functions is limited. Figured out that it becomes annoying when you can call the refined function on any type. But don’t quite understand what is the difference between create and createOrThrow? It seems to me that create must eagerly make a constraint check, return a desired object or throw an exception

@ILIYANGERMANOV

...which will automatically do the trimming and then check for blankness

That is a good point, probably Constraint can do both validation and conversion.

---

One more possible design:

abstract class Exact<T, R>(
        private val constraint: Constraint<T>,
        private val constructor: (T) -> R
) {

    fun create(value: T): R {}
    fun createOrNull(): R? {}
}

value class NotBlankTrimmedString private constructor(val value: String) {

    companion object : Exact<String, NotBlankTrimmedString>(
            constraint = NotBlank() and Trimmed(),
            constructor = { NotBlankTrimmedString(it) }
    )
}

Don't like the notion of constructor but for now don't have a clear understanding how to solve it differently without reflection and KSP

[ILIYANGERMANOV]

Awesome! @ustits glad to hear that you're interested in collaborating on this :)
I like your last proposed design. IMO, it's a good time to start implementing a 0.0.1 version and taking the discussion into PRs and actual code.

Whoever has some free time and enthusiasm can start. I vote for the last proposed design by @ustits:

interface Constraint<T, R> {
   fun constrain(value: T): R // <-- can be a transformation (mapping), a predicate or both
}

abstract class Exact<T, R>(
        private val constraint: Constraint<T>,
        private val constructor: (T) -> R
) {
    fun create(value: T): R {}
    fun createOrNull(): R? {}
}

value class NotBlankTrimmedString private constructor(val value: String) {

    companion object : Exact<String, NotBlankTrimmedString>(
            constraint = NotBlank() and Trimmed(),
            constructor = ::NotBlankTrimmedString
    )
}

To recap based on our discussion, we're looking for the following properties:

• composing constraints to form new types
• being to do mapping, e.g. NotBlankTrimmedString
• low boilerplate required to define new types, otherwise the library will be useless
• (personal opinion) Commonly used pre-defined types like in Krefty. If the library size (mental or physical) is a concern we can have arrow-exact-core and arrow-exact-common.

What are your thoughts? @nomisRev are we ready to start? 🚀 I want it to be fun to use in practice while making strong guarantees about the data model it defines.

[CLOVIS-AI]

I don't have much free time at the moment, but I completely agree with @ILIYANGERMANOV's comment above.

[ustitc]

I am trying a couple of alternative designs, will create a PR when I would have time

[ustitc]

Hey everyone, created a first draft, feel free to leave comments about the design #6