Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata to data being sent to API endpoint in JSON dump registration backend #5012

Closed
viktorvanwijk opened this issue Jan 10, 2025 · 3 comments · Fixed by #5053
Closed

Comments

@viktorvanwijk
Copy link
Contributor

Follow-up of #4908

Suggestion is to add metadata to the data being sent. In the JSON dump configuration options, any variable (component, user-defined, or static) can be selected to add to this metadata, with several pre-selected variables such as:

  • Form name (form_name)
  • Form ID (form_id)
  • Submission ID (submission_id)
  • Now (now)
  • Authentication type (auth_type)
  • Others...?

Example

An example form with the following configuration:

  • Authentication via DigiD, capturing a BSN
  • First form step with text fields firstName and lastName
  • Second form step with a user upload attachment

Should lead to a JSON object being sent with the shape:

{
    "metadata":
    {
        "form_name": "Form 1",
        "form_id": "b78d063c-e249-4217-99e3-eaf3cf1fd590",
        "submission_id": "6e2fd868-ba06-4faf-a8ec-c4d27d55cea8"
        "now": "2025-01-10T15:28:00+01:00",
        "auth_type": "bsn",
    }
    "metadata_schema":
    {
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "type": "object",
        "properties":
        {
            "form_name":
            {
                "title": "Form name",
                "type": "string"
            }
            "form_id":
            {
                "title": "Form identifier",
                "type": "string",
                "format": "uuid"
            }
            "submission_id":
            {
                "title": "Submission identifier",
                "description": "UUID of the submission",
                "type": "string",
                "format": "uuid",
            }
            "now":
            {
                "title": "Current date time",
                "type": "string",
                "format": "date-time"
            },
            "auth_type": 
            {
                  "title": "Authentication type",
                  "type": "string",
                  "enum": ["bsn", "kvk", "pseudo", "employee_id"]
            },
    }
    "values":
    {
        "auth_bsn": "123456782",
        "firstName": "Donnie",
        "lastName": "Darko",
        "attachment": "<base64 encoded data>"
    },
    "values_schema":
    {
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "type": "object",
        "properties":
        {
            "auth_bsn":
            {
                "type": "string",
                "pattern": "^\\d{9}$"
            },
            "firstName":
            {
                "type": "string"
            },
            "lastName":
            {
                "type": "string"
            },
            "attachment":
            {
                "type": "string",
                "contentEncoding": "base64"
            }
        },
        "required":
        [
            "auth_bsn",
            "firstName",
            "lastName"
        ],
        "additionalProperties": false
    }
}
@viktorvanwijk viktorvanwijk added enhancement topic: registration waiting for approval An estimate was made but the stakeholder still needs to approve it. owner: venlo labels Jan 10, 2025
@viktorvanwijk viktorvanwijk self-assigned this Jan 10, 2025
@viktorvanwijk viktorvanwijk added the triage Issue needs to be validated. Remove this label if the issue considered valid. label Jan 10, 2025
@joeribekker
Copy link
Contributor

Add reference number to metadata
Rename "now" to "registration_timestamp"
Remove "form_id" and "submission_id"

@joeribekker joeribekker added approved and removed triage Issue needs to be validated. Remove this label if the issue considered valid. waiting for approval An estimate was made but the stakeholder still needs to approve it. labels Jan 13, 2025
@joeribekker
Copy link
Contributor

Discussed with JVS and they agree with the suggestion made by you @viktorvanwijk

@viktorvanwijk
Copy link
Contributor Author

Also add form version

@viktorvanwijk viktorvanwijk moved this from Todo to In Progress in Development Jan 22, 2025
@viktorvanwijk viktorvanwijk added this to the Release 3.1.0 milestone Jan 22, 2025
viktorvanwijk added a commit that referenced this issue Jan 28, 2025
viktorvanwijk added a commit that referenced this issue Jan 29, 2025
viktorvanwijk added a commit that referenced this issue Jan 30, 2025
Also add support for an extra static variables registry in generate_json_schema
viktorvanwijk added a commit that referenced this issue Jan 30, 2025
If there is no authentication, the value can be an empty string
viktorvanwijk added a commit that referenced this issue Jan 30, 2025
viktorvanwijk added a commit that referenced this issue Jan 30, 2025
viktorvanwijk added a commit that referenced this issue Jan 30, 2025
Also add support for an extra static variables registry in generate_json_schema
viktorvanwijk added a commit that referenced this issue Jan 30, 2025
If there is no authentication, the value can be an empty string
viktorvanwijk added a commit that referenced this issue Jan 30, 2025
viktorvanwijk added a commit that referenced this issue Feb 4, 2025
viktorvanwijk added a commit that referenced this issue Feb 4, 2025
If there is no content (in case of an HTTP 204 response for example), converting the response to json results in an error, so we just return an empty string
viktorvanwijk added a commit that referenced this issue Feb 4, 2025
* Improve UX by introducing a (collapsed) fieldset which contains a table of fixed variables, and a variable selection box to add additional variables.
* Fix imports
* Add TODO: currently, this is tricky because the form registration options are saved before the (user defined) form variables are persisted. If this field is limited to static/registration variables, however, it will be possible as they are defined in the code. At this point in time, though, not sure how strict we should make this.
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
Also add support for an extra static variables registry in generate_json_schema
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
If there is no authentication, the value can be an empty string
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
If there is no content (in case of an HTTP 204 response for example), converting the response to json results in an error, so we just return an empty string
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
* Improve UX by introducing a (collapsed) fieldset which contains a table of fixed variables, and a variable selection box to add additional variables.
* Fix imports
* Add TODO: currently, this is tricky because the form registration options are saved before the (user defined) form variables are persisted. If this field is limited to static/registration variables, however, it will be possible as they are defined in the code. At this point in time, though, not sure how strict we should make this.
* Fix help texts of JSON dump option fields. They are sentences that should end with a period
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
Also add support for an extra static variables registry in generate_json_schema
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
If there is no authentication, the value can be an empty string
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
If there is no content (in case of an HTTP 204 response for example), converting the response to json results in an error, so we just return an empty string
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
* Improve UX by introducing a (collapsed) fieldset which contains a table of fixed variables, and a variable selection box to add additional variables.
* Fix imports
* Add TODO: currently, this is tricky because the form registration options are saved before the (user defined) form variables are persisted. If this field is limited to static/registration variables, however, it will be possible as they are defined in the code. At this point in time, though, not sure how strict we should make this.
* Fix help texts of JSON dump option fields. They are sentences that should end with a period
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
* Improve UX by introducing a (collapsed) fieldset which contains a table of fixed variables, and a variable selection box to add additional variables.
* Fix imports
* Add TODO: currently, this is tricky because the form registration options are saved before the (user defined) form variables are persisted. If this field is limited to static/registration variables, however, it will be possible as they are defined in the code. At this point in time, though, not sure how strict we should make this.
* Fix help texts of JSON dump option fields. They are sentences that should end with a period
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
* Improve UX by introducing a (collapsed) fieldset which contains a table of fixed variables, and a variable selection box to add additional variables.
* Fix imports
* Add TODO: currently, this is tricky because the form registration options are saved before the (user defined) form variables are persisted. If this field is limited to static/registration variables, however, it will be possible as they are defined in the code. At this point in time, though, not sure how strict we should make this.
* Fix help texts of JSON dump option fields. They are sentences that should end with a period
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
Also add support for an extra static variables registry in generate_json_schema
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
If there is no authentication, the value can be an empty string
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
If there is no content (in case of an HTTP 204 response for example), converting the response to json results in an error, so we just return an empty string
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
* Improve UX by introducing a (collapsed) fieldset which contains a table of fixed variables, and a variable selection box to add additional variables.
* Fix imports
* Add TODO: currently, this is tricky because the form registration options are saved before the (user defined) form variables are persisted. If this field is limited to static/registration variables, however, it will be possible as they are defined in the code. At this point in time, though, not sure how strict we should make this.
* Fix help texts of JSON dump option fields. They are sentences that should end with a period
viktorvanwijk added a commit that referenced this issue Feb 5, 2025
@github-project-automation github-project-automation bot moved this from Implemented to Done in Development Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment