API documentation for Jsonnet libraries

[steinybot]

This may be a bit premature but are there plans to come up with some sort of API documentation for the Jsonnet libraries, in particular the Kubernetes (ksonnet) library?

I find it really difficult to use without lots of string searching. IDE support would help but most plugins are limited. I just found http://g.bryan.dev.hepti.center/ which looks pretty cool. How do others manage to use it?

BTW thank you for planning to take over ksonnet-lib, this is awesome news.

[sh0rez]

Hi, FYI our fork of ksonnet-lib is available at https://github.com/kube-jsonnet/k. Keep in mind it's still alpha, some things might now work. Happy to receive feedback!

Regarding docs, we have no proper solution at the moment. We ourselves have to frequently view the source code of libraries to figure out what's available, so it's one of our biggest priorities to get an adequate solution here.

[sbarzowski]

Regarding docs, we have no proper solution at the moment. We ourselves have to frequently view the source code of libraries to figure out what's available, so it's one of our biggest priorities to get an adequate solution here.

Do you have any thoughts on how you would want this to work? For example what structure such docs should have? Or how to describe objects in the documentation? It's interesting to us, because we hope to provide an upstream solution one day.

FYI @sparkprime has a work-in-progress effort to extract information from the source code comments. I started investigating the other side of this - generating the HTML documentation using Jsonnet based on structured JSON input (I think the first target will be the stdlib documentation).

[sh0rez]

I do have a very very ugly prototype (https://github.com/sh0rez/tkdoc), which is highly opinionated:

• expects the exported object of the library to contain only functions
• these are then printed with function signature and comment*
• no comments at the moment (couldn't find them in the ast)

The overall idea is the following:

• libraries define a single entry file (main.libsonnet), which exposes the entire API. Example: https://github.com/sh0rez/grafana.libsonnet
• this file are the source for the documentation

With such a clear entrypoint, it is in theory easy to generate the docs.

BUT: The result of jsonnet.SnippetToAST is not really usable for that, because it literally represents the file contents, without possible changes due to object merging, etc. which are part of the evaluation.

So I think we need to somehow get a representation of the evaluation result right before rendering to json. IE all object merges already done, references resolved. Is that possible @sbarzowski?

[sh0rez]

That of course would be heavily opinionated for the use-case of Kubernetes but could get us to a starting point which we can build on later

[sbarzowski]

So I think we need to somehow get a representation of the evaluation result right before rendering to json.

Yes, but it wouldn't necessarily be what you want. The result would be values (object, number, function), not AST nodes (object literal, function definition). So in particular it is stripped of any comments at this point.

[sh0rez]

@sbarzowski that's a bummer, it won't help us then.

As you know the language internals far better than me, what's your recommendation? Consider the following example:

// foo.libsonnet
{
  new():: { apiVersion: "v1", kind: "Service" },
  spec:: { 
    // `withPorts` returns a patch to set `spec.ports` to `ports`
    withPorts(ports):: { spec+: { ports: ports }}
  }
}

// main.libsonnet
(import "foo.libsonnet") + {
  // `new` returns a new instance of `Service`
  new(name):: super.new() + { metadata+: { name: name }}
}

I'd like to generate the following docs from it:

---

main.libsonnet

fn new

new(name):: {}

new returns a new instance of Service.

fn spec.withPorts

withPorts(ports):: {}

withPorts returns a patch to set spec.ports to ports

---

[sbarzowski]

Well, to support something like that, we'll need static analysis which performs something like limited evaluation, but also keeps track of the comments and original source code locations. It will have a very limited set of operations (mostly + and locals I think). The most tricky part which may be needed is some support for self and super.

It is not entirely out of question to extend the normal evaluation to carry more source information around, but this may be pretty complicated (unclear how to handle list comprehensions, function calls etc.) and may have negative performance implications. And it's way more invasive.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jwenz723]

I don't know how this documentation is generated, but it seems like it would make sense to generate docs similar to the jsonnet stdlib: https://jsonnet.org/ref/stdlib.html

But maybe the better option is to move forward with using https://jsonnet-libs.github.io/k8s-alpha/1.18/ ?

[sh0rez]

https://github.com/jsonnet-libs/docsonnet is now a thing and powers the docs of https://jsonnet-libs.github.io/k8s-alpha/.

Closing this issue as the general problem is resolved. Improvements welcome :)