GraphQL Annotations - Updated (as metadata)

[clintwood]

This PR builds on PR#265 (also discussed here: #114, #180, #264) by bringing it up to date with the latest commits in this repo. This PR has assumed using @@ as the prefix for annotations but can be easily changed.

Summary
This PR also extends the concept of annotations further to fully enable annotations on Object Type Definitions GraphQLObjectType and Object Field Definitions (see examples below) with introspection of annotations on __type and __type { fields } (also via __schema { types { fields } }). This PR also enables annotations on GraphQL schema language (including schema type extension system) as well as on GraphQL query language constructs including Query/Mutation/Subscription/Fragment and Field Definitions (see examples below). What's not covered yet (and can be easily added) is adding annotation fields to the introspection query to support various utilities. And of course there are other areas that could have been missed :).

Details - Annotations give us Metadata
Annotations are different from Directives in that directives may and are usually expected to affect the output result of GraphQL requests and GraphQL service behavior whereas Annotations provide a mechanism to attach metadata to schema definitions, request and schema language constructs. This metadata can be used by build-time tooling or at runtime at any point through the request pipeline. Annotations are much more akin to formalised comments than to actual language constructs and as such become part of the request or schema AST via an annotations property.

Rationale and Use Cases
Currently (including the latest Directive RFC's) there is no way to associate metadata with various GraphQL constructs for use by runtime systems or build time tooling. Adding the ability to annotate various GraphQL constructs opens up the possibility for a number of interesting use cases. In my particular case I have two concrete use cases that would (and actually do via this fork) make use of annotations on GraphQL Schema Definitions as well as on GraphQL query language.

Annotations on GraphQL Schema Definition Use Case:
As shown in the example below annotations on schema definitions are being used for dynamic database query generation at runtime where the annotation metadata can describe the collections and other database attributes needed for mapping purposes. This annotation metadata on schema definitions is also useful for build time transforms using tools like babel (much like relay does with babel-relay-plugin client side). A nice spin-off of this is that the schema definition resolve implementations can be decoupled from database/store specific implementations by handing off metadata to a store layer to interpret and materialize required data however it desires.

Annotation on GraphQL 'query' (request) language Use Case:
Also shown below, having annotations on query (request) language constructs enables both runtime and build time use of annotation metadata for dynamic query composition, generation of result accessors by client code - I have a more concrete use case which is similar in some respects to relay which I hope to release at some point.

Examples
Schema Annotations Example - Annotations for database query generation at runtime:

// User type
export const User = new GraphQLObjectType({
  name: 'User',
  description: 'User Type.',
  annotations: {
    db: { collection: 'Users' }
  },
  fields: () => ({
    id: {
      type: new GraphQLNonNull(GraphQLID),
      description: 'User ID.',
      annotations: {
        db: { isKey: true, fieldName: '_id' }
      }
    },
    name: {
      type: new GraphQLNonNull(GraphQLString),
      description: 'User name.',
    },
    friends: {
      type: new GraphQLList(Friend),
      description: 'Users\'s friend list.',
      annotations: {
        db: {
          isConnection: true,
          edgeCollection: 'UserFriends',
          edgeDirection: 'outbound',
        },
        auth: {
          roles: ['user', 'admin'],
        },
      },
      args: {
        ...rangeArgs,
        ...filterArgs,
      },
      resolve(root, args, authContext, info) {
        // e.g. get annotations on User
        let userAnnotations =
          info.parentType.annotations;
        // e.g. get annotations on this field
        let fieldAnnotation = 
          info.parentType.getFields()[info.fieldName].annotations;

        // handoff to store to generate query required to resolve friends
        return info.rootValue.store(root, args, authContext, info);
      }
    },
  }),
});

The above annotations can also be queried via introspection:

`# query
query {
  __type(name: "User") {
    name
    annotations {
      name
      args {
        name
        value
      }
    }
    fields {
      name
      annotations {
        name
        args { 
          name
          value
        }
      }
    }
  }
}`


// Result
  "data": {
    "__type": {
      "name": "User",
      "annotations": [
        {
          "name": "db",
          "args": [
            {
              "name": "collection",
              "value": "Users"
            }
          ]
        }
      ],
      "fields": [
        {
          "name": "id",
          "annotations": [
            {
              "name": "db",
              "args": [
                {
                  "name": "isKey",
                  "value": "true"
                },
                {
                  "name": "fieldName",
                  "value": "_id"
                }
              ]
            }
          ]
        },
        {
          "name": "name",
          "annotations": []
        },
        {
          "name": "friends",
          "annotations": [
            {
              "name": "db",
              "args": [
                {
                  "name": "isConnection",
                  "value": "true"
                },
                {
                  "name": "edgeCollection",
                  "value": "UserFriends"
                },
                {
                  "name": "edgeDirection",
                  "value": "outbound"
                }
              ]
            },
            {
              "name": "auth",
              "args": [
                {
                  "name": "roles",
                  "value": "user,admin"
                }
              ]
            }
          ]
        },
      ]
    }
  }

GraphQL 'query' (request) language Annotations Example - Annotations used client side:

const request = `
@@clientCache(ttl: 3600)
query ($id: ID!) {
  viewer {
    id
    @@prop
    name
  }
  someUser: userById(id: $id) {
  viewer {
    id
    @@prop
    name
    ...friends
  }
}

@@prop(name: "someUserFriends")
fragment friends on User {
  friends {
    # friend fields
  }
}
`;

// NOTE: annotation metadata from a request like the above is available in the AST
// from parsed requests (and also accessible via fieldASTs in the resolve function
// in the schema)
// Typically AST would be accessed using visit(...) function but can be accessed
// via other means e.g.:
const ast = parse(request);
let clientCacheTTL = ast.definitions[0].annotations[0].arguments[0].value.value;

// Hypothetical example using above request in property generation mechanism to a
// create relay like container (note this is not relay!)
const container = createContainer(request);
await container.fetch('viewer');
let viewerName = container.viewer.name; // get viewer#name

await container.fetch('someUser', { id: "user123" });
let userName = container.someUser.name; // get someUser#name
let friends = container.someUserFriends;
let friendCount = friends.length;

Caveats of this PR:

• Some anomilies with flow were not resolved but noted
• Need a better way to restrict annotations to scalar types and/or validate JS scalar values
• Mentioned previously the introspection query should include annotation fields and dependant utility functions need to be updated.

Lastly this PR is not intended to take away from the work done by others in the referenced PR's and issues but hopefully to take the concept further.

[facebook-github-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla - and if you have received this in error or have any questions, please drop us a line at cla@fb.com. Thanks!

[coveralls]

Coverage increased (+0.01%) to 99.457% when pulling 1933963 on clintwood:annotations into 3974438 on graphql:master.

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

[leebyron]

Thanks for your work on this! This is still something I'm interested in adding and discussing further.

A few points of feedback based on what's been discussed so far:

I think the concepts of "directives" and "annotations" should be combined. In my opinion the grammatical variations between the two are more confusing than useful. In fact, a change was made to the spec for introspection which opens the way for this: graphql/graphql-spec#152

Another rough area to resolve is to determine whether the purpose of annotations is for .graphql files to be annotated as an input to graphql tools (validators, executors, code-generators) or whether annotations should become metadata added to elements of a graphql schema, where the annotation is just simply a way to syntactically represent that information when printed using the schema language. Your use cases imply both:

Annotations on GraphQL Schema Definition Use Case:

As shown in the example below annotations on schema definitions are being used for dynamic database query generation at runtime where the annotation metadata can describe the collections and other database attributes needed for mapping purposes.

I'm curious if this kind of information is necessary to provide to a client (like an iOS or Relay app) or if it's simply a server-side implementation detail, in which case should it really be exposed via introspection? What's a client to do with that information?

Annotation on GraphQL 'query' (request) language Use Case:

Also shown below, having annotations on query (request) language constructs enables both runtime and build time use of annotation metadata for dynamic query composition, generation of result accessors by client code.

This was the exact original motivation for adding Directives to the language, as a means to annotate portions of requests so that they can be treated differently at whatever part of the system, whether that's by a client-side system generating code (and then stripped before being sent to a server) or sent through to a server where execution behavior can be modified (@include and @skip being the built-in examples).

[clintwood]

@leebyron, thank you for spending time on this!

I believe Directives and Annotations are different and therefore have different use cases which I'll try better explain below.

Directives:
My understanding of Directives as they are in GraphQL today is that they need to be defined using the GraphQLDirective class (as is done with the built in include and skip directives) then added to the schema via directives element/prop. These directives can now be used in GraphQL query/request language (.graphql files, etc.) to 'direct' behavior through the execution pipeline. They also seem to lean toward GraphQL server side services.

This is different from what is being proposed in this PR in the following ways:

Annotations:
Annotations on Schema is informal metadata on selected 'elements' (GraphQLObjectType, GraphQLFieldDefinition) of a schema definition defined in a formal way via the annotations field. In JavaScript terms this is effectively JSON tacked onto selected elements (GraphQLObjectType, GraphQLFieldDefinition, etc.) in a way that can be validated against a predefined structure. This is a new concept which is not related to Directives.

This schema metadata can then be accessed and used via the resolve function or by other API's in the resolve pipeline.

In my specific use case I use the schema both server side and client side (with some variation for offline data) and pass this metadata to the store provider (in resolve) which it uses to generate data access functionality. The store providers are obviously different for client and server side.

Annotations on GraphQL query/request language is informal metadata on selected language constructs of GraphQL query/request language (and .graphql files, etc.) with a formal structure. Informal in the sense that these annotations do not need to be defined in the schema before use in the query language (as is done for Directives). The formal structure is similar to Directives e.g. @@name(a: "arg1" b: "arg2") (Token Name Arguments where arguments are optional) and the placement is before language constructs (unlike Directives). These are much more like formalised comments that are recognised by the parser.

This query/request language metadata is available in the parsed AST.

I have several use cases for this metadata, however, my primary use is client side is for annotating the relationship between root operation fields (and parent fragments) and fragments where fragments may be added dynamically at run-time (and therefore cannot be specified using a fragment spread in the root operaton field since this is not know beforehand). I also use this metadata to annotate what 'props' to expose (in a non-react world) for binding to UI elements. Requests pushed to the server are generated by manipulating ASTs that have the non-relevant annotations stripped out. There is a lot more to this but you get the idea...

Once again I appreciate your considering this. I'd also be interested in whether other users see Annotations as specified here a useful feature!?

[coveralls]

Changes Unknown when pulling e3b22d6 on clintwood:annotations into * on graphql:master*.

[helfer]

I think annotations would be incredibly useful. I think @leebyron is right that they're pretty similar to directives when it comes to queries. Personally, I don't think it matters whether we call them directives or annotations, so long as there's something that lets you add metadata to schema language, and not just to queries.

I'm not sure if it's a good idea to make those completely free-form though, where there's no validation whatsoever, because that will make it harder to detect and fix errors. To address that maybe we could pass annotation/directive definitions to the parser, which could then do some validation on them. Maybe those definitions could more or less take the shape of plugins that could modify the AST in certain ways (e.g. by adding nodes to it), or modify the schema during construction. That would allow for things such as automatic construction of resolve functions for REST endpoints, simple mocking, and many more.

[clintwood]

@ helfer, the plugin idea sounds great - in reality all I'm looking for is a way to extend the AST (both query & schema languages) via annotations to add custom metadata for my internal use... If that can be achieved with plugins of some form I'd be very interested!

In this PR, annotations have some loose type checking to ensure argument values are scalars (JavaScript) - but this is incomplete. If GraphQL formalised (included) a GraphQLJSONType it would make sense to validate annotations against that.

[helfer]

@clintwood @leebyron I wrote down some of my thoughts on schema decorators. They are similar to directives, but go beyond annotations in that they don't just add metadata, but can actually transform the schema. Would love to get your thoughts on it: ardatan/graphql-tools#43

The main difference between annotations or directives and decorators is that decorators are intended to be self-contained components that can quickly be dropped into any server that provides the right hooks.

That's different from directives, because support for each directive currently has to be hard-coded one by one into the server.

It's also different from annotations, because annotations don't have any semantic meaning, they would just be a way to add structured metadata to a schema. The semantics would have to be implemented separately by each project using them, thus limiting the potential for reuse.

PS: Directives and decorators are so similar in syntax that they should be merged if possible, but there are two things that would have to be resolved for that to happen:

1. Directives currently have to be defined in the spec to have any semantic meaning, and implemented separately by each server. That means they can't be added to a server by installing a package.
2. Directives come after the thing they modify, decorators come before. I tried putting them after, but it really didn't look nice.

[freiksenet]

@helfer I think schema decorators unnecessarily lock the schema into the implementation. The good part about having just the metadata style decorators is that different engines can interpret them in different way.

What you have suggested is implementable with just metadata decorators/directives and a custom processor on top of schema parser, why build a new language construct just for that?

[helfer]

@freiksenet Yes, I'm aware that the metadata and some pre/post processor is sufficient for this. What I'm trying to do is specify what form this metadata should have, and provide a reusable way of defining them, the purpose explicitly being that different implementations preserve the semantic meaning of these decorators.

There's the generic part that goes into the schema language, but there must also always be a part that interprets the decorator/annotation, and that part is specific to the implementation. In my examples I've assumed GraphQL-JS for that part.

[freiksenet]

OK, makes sense then.

I think we should really agree on something. Original metadata annotation idea is from last year, there has been couple of suggestions but we never managed to actually get any of them in. I think preservable metadata in schema done in any way is better than no metadata situation that we have now :)

[clintwood]

Closing in favor of #376.