[meta issue] make all Nim APIs generic: ctor, init, to, streams, types, etc

[timotheecour]

All these will simplify some API's by making them generic with less duplication:

• constructors: https://github.com/nim-lang/Nim/issues/7474 newFooT => Foo.NewT
• initializers: (TODO: file issue) initFooT => Foo.InitT
• converters/parsers: https://github.com/nim-lang/Nim/issues/7430 a.parseBool => a.to[bool] or a.parse[bool]
• streams: Add ByteStream and generic read proc to streams Nim#7481 readBool => read[bool]
• macros.nim: nnkInt32Lit => nnkLit[int32] (eg will simplify newLit, etc)
• typeinfo.nim: getUInt32 => getType[uint32]
• json: newJBool => newJType[bool], getBool => getType[bool]

NOTE about generic meaning

Nim generics can be overloaded (eg: proc foo[T:int|bool]() or even with proc foo[T:isConceptBar]() unlike java generics, so the term can be confusing for some and indeed it tripped a few people recently (eg https://github.com/nim-lang/Nim/issues/7430#issuecomment-377473190 , nim-lang/Nim#3502 (comment) ), maybe Nim docs could clarify this aspect. The point is we want to have generic (in Nim's sense of overloadable generics) APIs as much as possible, as legacy API's like parseBool make generic code hard to use (API writer needs special case for each type eg bool), reminiscent of C and go.

tell-tell sign that generic api's are needed:

lots of very similar code blocks repeated for several distinct types

eg: in macros.nim:

proc newLit*(i: int32): NimNode {.compileTime.} =
  ## produces a new integer literal node.
  result = newNimNode(nnkInt32Lit)
  result.intVal = i

proc newLit*(i: int64): NimNode {.compileTime.} =
  ## produces a new integer literal node.
  result = newNimNode(nnkInt64Lit)
  result.intVal = i
# etc (lots of overloads)

could become simply:

proc newLit*[T](i: T): NimNode {.compileTime.} =
  result = NimNode.New(nnkLit[T])
  result.val[T] = i

motivation

non-generic API's (eg json newJBool) is viral: it causes calling functions to also be non-generic, or pushes the complexity on them to wrap them into a generic API

how to rewrite API's (typedesc vs generics) ?

https://github.com/nim-lang/Nim/issues/7517 [RFC] guidelines for when to use typedesc vs generics

[siliconvoodoo]

yes this is the thing that surprised me the most when using the json module. I thought it was a joke.

[Araq]

I prefer a langauge with a low barrier to entry though. A standard library does not have to be a strange beast that uses every feature the language offers.

[metagn]

For json just use %.

proc toJsonNode[T](val: T): JsonNode = %val

I wouldn't know about how necessary rewriting the newLit procs is, but if you want a nnk -> type transform you can just do

proc litNodeFromType(T: typedesc): NimNodeKind {.compiletime.} =
  var default: T
  let node = newLit(default)
  return node.kind

static:
  echo litNodeFromType(int)

And you could rewrite them anyway with the language as is

const litNodeToType: array[nnkCharLit..nnkStrLit, typedesc] = [
  char, int, int8, int16, int32, int64, uint, uint8,
  uint16, uint32, uint64, float, float32, float64,
  (when compiles(float128): float128 else: BiggestFloat), string
]

proc litNodeFromType(T: typedesc): NimNodeKind {.compiletime.} =
  let i = litNodeToType.find(T)
  if i != -1:
    result = litNodeToType[low(litNodeToType) + i]

proc newLit[T](i: T): NimNode {.compiletime.} =
  result = newNimNode(litNodeFromType(T))
  result.val = i

static:
  echo repr(newLit(3))
  echo repr(newLit('a'))

[timotheecour]

@Araq

I prefer a langauge with a low barrier to entry though. A standard library does not have to be a strange beast that uses every feature the language offers.

Low barrier of entry is good thing. However:

• genericity is not a strange beast, languages that support metaprogramming all make use of it in the standard library
• The standard library is often setting the standard in terms of best practices for other libraries.
• Nim's standard library should eat it's own dogfood. If it's hard to use in places, it should be fixed

Here's some concrete data points I hope you'll appreciate:

• the top complaint for go is lack of generics, eg https://blog.golang.org/survey2016-results (What changes would improve Go most?) + tons of other posts link
• in D's standard library phobos which makes heavy use generic API's centered around ranges (and std.conv.to etc), I can't remember anyone ever complaining about too much genericity (and I follow closely their forums). In the 2018 State of D survey ( https://rawgit.com/wilzbach/state-of-d/master/report.html), this issue wasn't even mentioned in "What do you dislike about Phobos (D's Standard Library)?" question. I also went through all 75 free-form answers to that question (see link ) and could only find 1 answer complaing about it: "Over-reliant on templates"
• the modules ppl keep complaining about are the "old-style" ones (non generic or not using ranges), eg std.json same survey (mentioned 17 times here ) which is non-generic ; ppl usually recommend using stdx.data.json (slated for std.json's replacement), which is written in generic style link

Certain features should definitely only be used sparingly, eg when alternatives are worse, eg from worst to mildest:

• 1 dirty macros
• 2 macros
• 3 templates
• 4 procs with typedesc
• 5 procs with generics

IMO whenever there's code duplication that just depends on type, it's a good candidate for generics

[dom96]

You're applying the results of a D survey to Nim, perhaps we should look at the results of the Nim survey?

If this is a pain point in Nim then we should see plenty of mentions of "generic" in there, but the only ones I could find are to do with stability, i.e. "please stabilise generics and concepts!". There is one comment which states "My only negative observation is that Nim's generics are not as good as D's." but that's it.

Perhaps we can interpret that as meaning "Nim should offer more generic APIs in the stdlib" but I'm not so certain. Happy to grep for anything else.

[lightness1024]

If you use a voting system one day, think about the "randomized Condorcet voting"
https://github.com/oscar6echo/randomized-condorcet-voting-system

[Bulat-Ziganshin]

As I wrote here, I propose:

1. use "str".parse(bool) syntax for operation opposite to '$'
2. use longer 'convertTo' name instead of bare 'to' for data type conversions which keeps data value, but changes "the shape" - such as list/seq, or seq[pair]/Dict, or various number types

[PMunch]

I'd really like to see this be adopted to the entire stdlib for Nim v2. It makes it easier to write macros and generics, and it looks much cleaner. The last part is quite subjective of course but I find var x = "100".parse(int) much nicer than var x = "100".parseInt(), same goes for var x = new MyType vs var x = newMyType(). It also ties in fairly well with how $ is just a procedure which exists just as a procedure which takes a type and returns a string. Similarly these procedures would feel much more like the same concept applied to different types. For macros and generics the benefit is pretty obvious, being able to have the type separate from the function name makes everything easier to compose. A generic procedure for example could support reading any kind of data from a stream with a simple someObject.genericField = myStream.read(T) (for reading data into SomeObject[T] = object; genericField: T).

EDIT: By the way I believe we should do this with typedesc instead of generics. That's the best way to get cleanest interface out of this.