[Question] local relative $ref

[dkarlovi]

I'm using this with Behat to validate a JSON response for APIs. Using 5.2.1.

If my schema is embedded inside a Behat feature file (say features/my.feature), how do I reference a definition inside schemas/entity.json? Would that definition be able to reference some other definition (schemas/other-entity.json)?

[erayd]

Local references within the same document must start with /, where / refers to the root of the schema in which the $ref occurs.

If you're wanting to refer to another file, then you simply concatenate the file and the reference to the part of the file you wish to reference, separated by #. For example, "$ref": "schemas/other-entity.json#/definitions/my/entity". If you want to reference the entire file, then the filename alone is sufficient.

There is no limit to how many layers of referencing you can use. Definitions are dereferenced as needed during validation.

[dkarlovi]

@erayd Thank you very much for such a quick reply! 👍

Let's just use an example to be more clear, sorry about the verbosity:

1. I have a schema embedded into Behat feature file features/my.feature:

    Scenario: Fetching a list of existing images when images match
        When I send a "GET" request to "/api/images"
        Then the JSON should be valid according to this schema:
        """
        {
            "definitions": {
                "user": {
                    "type": "object",
                    "required": ["id", "username"],
                    "properties": {
                        "id": {"type": "string", "format": "udid"},
                        "username": {"type": "string"}
                    },
                    "additionalProperties": false
                },
                "image": {
                    "type": "object",
                    "required": ["id", "name"],
                    "properties": {
                        "id": {"type": "string", "format": "udid"},
                        "name": {"type": "string"},
                        "created_at": {"type": "string", "format": "date-time"},
                        "created_by": {"$ref": "#/definitions/user"},
                        "updated_at": {"type": "string", "format": "date-time"},
                        "updated_by": {"$ref": "#/definitions/user"}
                    },
                    "additionalProperties": false
                }
            },
            "type": "array",
            "items": {"$ref": "#/definitions/image"},
            "minItems": 30,
            "maxItems": 30,
            "uniqueItems": true,
            "additionalItems": false
        }
        """

2. I'd like to extract user and image definitions to schemas/user.json and schemas/image.json respectively, ie.

{
  "title": "user",
  "type": "object",
  "required": ["id", "username"],
  "properties": {
    "id": {"type": "string", "format": "udid"},
    "username": {"type": "string"}
  },
  "additionalProperties": false
}

{
  "title": "image",
  "type": "object",
  "required": ["id", "name"],
  "properties": {
    "id": {"type": "string", "format": "udid"},
    "name": {"type": "string"},
    "created_at": {"type": "string", "format": "date-time"},
    "created_by": {"$ref": "user.json"},
    "updated_at": {"type": "string", "format": "date-time"},
    "updated_by": {"$ref": "user.json"}
  },
  "additionalProperties": false
}

3. this would make my feature file more dry and makes writing tests much easier:

    Scenario: Fetching a list of existing images when images match
        When I send a "GET" request to "/api/images"
        And the JSON should be valid according to this schema:
        """
        {
            "type": "array",
            "items": {"$ref": "schemas/image.json"},
            "minItems": 30,
            "maxItems": 30,
            "uniqueItems": true,
            "additionalItems": false
        }
        """

Obviously, this doesn't work currently so my questions is how would one do this properly?

[erayd]

It's 1:34am here, so I might be missing something, but what you've proposed in your most recent post looks correct.

When you say it doesn't work, what happens instead? Can you post the error(s) you are seeing?

[dkarlovi]

It says

Unable to resolve URI 'schemas/image.json' from base '' (JsonSchema\Exception\UriResolverException)

[erayd]

That means that you haven't told the validator where your original schema lives, and you also haven't loaded the referenced schemas manually (i.e. you are relying on the autoloader). That is fine, however as your $ref paths are relative, it needs to know what they are relative to in order to resolve them.

If you add "id": "file:///path/to/features/my.feature" to the base schema, and ensure that your references are valid relative to that, then it will work. Note that you may need to prefix them with ../ or similar if the schemas directory doesn't live in the same place as my.feature.

[dkarlovi]

I think I get it, makes perfect sense. 👍 I'm currently trying to get it to work with your suggestion, getting the same error but will try to track it down with this new found knowledge.

Will report here if I succeed, thanks again for your time.

[erayd]

I wish you the best of luck :-). If you get stuck and need to get it working ASAP, loading the schemas manually is an alternative approach. And if you think you've found a bug, please shout so we can fix it - the code is a mess with stuff from many years ago and from many people who are no longer around. We're cleaning it up, but it's taking a while to get through.

I really need to get some sleep now, followed by a long drive - so further replies are likely to be delayed - however please feel free to ask if you have any more questions about this. I'm aware that the documentation for this library is in pretty poor shape at present.

[dkarlovi]

@erayd I think this doesn't work (as expected), I've tried with your id suggestion.

Here's a repo reproducing the issue, if you could point me some sort of solution, that would be great: https://github.com/dkarlovi/json-schema-behat-external-ref

[erayd]

It should work - if it's not working for you, then something is wrong. It may be a bug.

I'll investigate this now; stand by.

[erayd]

Can confirm there is a bug. Stand by for a fix.

[erayd]

@dkarlovi I need to think about this in more detail. The bug touches on some of the core $ref logic, and I need to think how best to fix this without breaking backwards-compatibility so that we can do a backported fix for 5.x.x. Watch this space, and bug #447.

[dkarlovi]

Thanks for having a look at this so quickly, appreciate it. Will be paying attention here and there.

[dkarlovi]

@erayd I can confirm this works as expected in 5.2.2, see my commit.

[dkarlovi]

@erayd I might be wrong, but the relative resolution seems off to me.

For example, in my repo files are layout like:

features/
    schemas/
        image.json
    my.feature

So, if I say id: features/my.feature, it should be $ref: schemas/image.json (as in, relative to features/, which would mean features/schemas/image.json), but currently I must specify ../schemas/image.json (as in, <root>/schemas/image.json) which doesn't exist.

This is more visible in my actual project where I have layout like:

features/
    images/
        images_fetch.feature
    schemas/
        image.json

and the working reference is:

            "id": "features/images/images_fetch.feature",
            "items": {"$ref": "../../../schemas/image.json"},

If I specify "$ref": "../schemas/image.json" (which should be correct), it says:

      file_get_contents(file://features/images/features/schemas/image.json): failed to open stream: No such file or directory (JsonSchema\Exception\ResourceNotFoundException)

[erayd]

Hmm.... I did test this, before I did the PR, but it's possible I may have screwed something up. Could you let me know what the working directory is in your example? The initial id is set relative to that.

[erayd]

Looks like I missed a condition in my original PR - this is fixed in #452. Thanks for spotting the issue.

[erayd]

@dkarlovi Hold fire on 5.2.3 (the release went out with the wrong branch - see #454).

[dkarlovi]

@erayd I can confirm this works as expected in 5.2.4. Thank you very much for your effort on this matter and a quick resolution.

[erayd]

@dkarlovi You're most welcome :-).

[dkarlovi]

BTW it's pretty weird collaborating with you on this fix and getting a workstation system update (Fedora Linux, Remi) installing the new version. 👍

[erayd]

Haha, I had no idea this library was packaged outside of composer! Very speedy too... I wonder if it's an automated build?

[erayd]

*outside of packagist

[dkarlovi]

I don't think it's fully automated, see: https://git.remirepo.net/cgit/rpms/php/justinrainbow/php-justinrainbow-json-schema5.git/