ResponseValidator: how to handle mime-type with character encoding details?

[erikbos]

While using ResponseValidator.validate I noticed validation is sensitive to the content-type header including character encoding details.

In case an HTTP response includes content-type: application/json; charset=utf-8 validation fails: MediaTypeNotFound(mimetype='application/json; charset=utf-8', availableMimetypes=['application/json'])

(As quick work I tried adding application/json; charset=utf-8 to the openapi spec as additional mime-type but that results in vague errors such as raised in #128)

I can manually overwrite mimetype of the response before validating but that feels a bit wrong .. one could reason that application/json; charset=utf-8 should be accepted and treated equal as application/json. Any comment?

See below for code that reproduces this situation.

import json
from openapi_core import create_spec
from openapi_core.validation.request.validators import RequestValidator
from openapi_core.validation.response.validators import ResponseValidator
from openapi_core.testing import MockRequest
from openapi_core.testing import MockResponse

def validate(spec_dict, mime_type):
    spec = create_spec(spec_dict)

    openapi_request = MockRequest('localhost', 'get', '/v1/cars')
    validator = RequestValidator(spec)
    result = validator.validate(openapi_request)
    request_errors = result.errors

    data = json.dumps({
        'cars' :[
            {
                'name': 'car1',
            },
            {
                'name': 'car2',
            }
        ]
    })
    openapi_response = MockResponse(data, mimetype=mime_type)

    # Workaround
    #if openapi_response.mimetype == 'application/json; charset=utf-8':
    #     openapi_response.mimetype = 'application/json'

    validator = ResponseValidator(spec)
    result = validator.validate(openapi_request, openapi_response)
    response_errors = result.errors

    print('Request errors: {} Response errors: {}'.format(request_errors, response_errors))


spec = {
    'openapi': '3.0.0',
    'info': {
        'version': '0.1',
        'title': 'List of objects',
        'description': 'Test for list of objects'
    },
    'paths': {
        '/v1/cars': {
            'get': {
                'description': 'Retrieve all cars',
                'responses': {
                    '200': {
                        'description': 'Successfully retrieved all cars',
                        'content': {
                            'application/json': {
                                'schema': {
                                    'type': 'object',
                                    'properties': {
                                        'cars': {
                                            'type': 'array',
                                            'items': {
                                                '$ref': '#/components/schemas/Car'
                                            }
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    },
    'components': {
        'schemas': {
            'Car': {
                'type': 'object',
                'properties': {
                    'name': {
                        'type': 'string'
                    }
                }
            }
        }
    }
}
validate(spec, 'application/json')
validate(spec, 'application/json; charset=utf-8')

[NasaGeek]

Personally, I worked around this by specifying the response mimetype in the spec as "application/json; UTF-8", e.g.

--- snip ---
      responses:
        '200':
          description: OK
          content:
            application/json; charset=UTF-8:
              schema:
                properties:
--- snip ---

I see that you mentioned trying that and encountering errors. Perhaps they're being caused by something else?

[erikbos]

Not sure if such work around actually works, hence I provided a test:

Run "as is":

• the first validate() call works, no errors, as expected.
• the second validate() fails as there is no matching content-type, as expected.

after the updating the content-type in test spec from application/json to application/json; charset=utf-8:

• the first validate() fails as there is no matching content-type, as expected.
• second one now fails with a schema validation error: schema_errors=(<ValidationError: '\'{"cars": [{"name": "car1"}, {"name": "car2"}]}\' is not of type object'>

By updating the spec I would expect the validation for the second validate() call to work, but that does not appear to be the case. Am I overlooking something?

[NasaGeek]

Looking back, I see there was one more element I needed for my workaround:

validator = ResponseValidator(
        spec,
        custom_media_type_deserializers={"application/json; charset=UTF-8": json.loads},
)