"TypeError: TypedDict does not support instance and class checks" for TypedDict field

[kpberry]

Bug

Models with TypedDict fields currently result in an error when they are instantiated, due to the fact that TypedDict does not support isinstance/issubclass checks.

from typing import TypedDict

from pydantic import BaseModel

TD = TypedDict('TD', {'a': int})


class ExampleModel(BaseModel):
    td: TD


m = ExampleModel(td={'a': 1})

# output:
# Traceback (most recent call last):
#   ...
#   File "pydantic/main.py", line 246, in pydantic.main.ModelMetaclass.__new__
#   File "pydantic/fields.py", line 310, in pydantic.fields.ModelField.infer
#   File "pydantic/fields.py", line 272, in pydantic.fields.ModelField.__init__
#   File "pydantic/fields.py", line 364, in pydantic.fields.ModelField.prepare
#   File "pydantic/fields.py", line 405, in pydantic.fields.ModelField._type_analysis
#   File "/usr/lib/python3.8/typing.py", line 1728, in _check_fails
#     raise TypeError('TypedDict does not support instance and class checks')
# TypeError: TypedDict does not support instance and class checks

Previously, models could have TypedDict fields without any problems. This error seems to have been introduced in commit #1266.

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

pydantic version: 1.5.1
            pydantic compiled: False
                 install path: /home/kpberry/Downloads/pydantic/pydantic
               python version: 3.8.2 (default, Mar 13 2020, 10:14:16)  [GCC 9.3.0]
                     platform: Linux-5.4.0-26-generic-x86_64-with-glibc2.29
     optional deps. installed: ['typing-extensions', 'email-validator', 'devtools']

[samuelcolvin]

afaik TypedDict is supported by pydantic, see #760.

Anything that does work will be flakey, will not perform proper validation and could break at any time.

I think best to explicitly raise an error whenever a TypedDict is used, saying

TypedDict is not yet supported, see #760

[kpberry]

I think TypedDict fields were usable with pydantic==1.4, but, as far as I can tell, the above TypeError does occur in pydantic>=1.5.

I think you're right, though, and it probably is best to raise an explicit error when TypedDict is used as a model field until TypedDict is fully supported.

[vbrinnel]

I confirm TypedDict were usable with pydantic 1.4

[samuelcolvin]

There's no logic for validating TypedDict, so while you might have been able to use them, I very much doubt it was a good idea.

We should either support them fully or raise a sensible exception explaining that they don't work.

This is an issue about raising a better error, #760 is about supporting them fully.

[vbrinnel]

I'm aware that no logic validation is currently performed by pydantic and it is fine with me.

We should either support them fully or raise a sensible exception explaining that they don't work.

There is also the option of treating them as Dict[Any, Any] for validation (they are dict at run-time after all). That way, one could still get proper mypy warnings during development and pydantic validation would just reduce to isinstance(variable, dict)

v1.4 behaved like this:

In []: from typing import TypedDict                                                                                                                                              
In []: from pydantic import BaseModel
                                                                                                                 
In []: class MyDict(TypedDict): 
    ...:     a: int
                                                                                                                                                 
In []: class A(BaseModel): 
    ...:     d: MyDict                                                                                                                                                                       

In []: A(d={})                                                                                                                                                                   
Out[]: A(d={})

In []: A(d=12)                                                                                                                                                                   
---------------------------------------------------------------------------
ValidationError: 1 validation error for A
d
  value is not a valid dict (type=type_error.dict)

[Granitosaurus]

There's no logic for validating TypedDict

Actually TypedDict does contain validation instructions

>>> from typing import TypedDict

>>> class Person(TypedDict):
...     name: str
...     age: int

>>> Person.__annotations__
{'name': <class 'str'>, 'age': <class 'int'>}

So technically Pydantic should be able to validate dictionaries that are type hinted with TypedDicts

[samuelcolvin]

Yes, it's possible, now someone just needs to go and implement the feature. 😄

[kpberry]

I currently have a working solution for this which does validation based on the TypedDict annotations as suggested. I'm currently writing test cases and will post an update when I've got full coverage.

[kpberry]

My solution is here, but there are parts of it that I think could probably be done better:
https://github.com/kpberry/pydantic/commits/typed-dict-support

1. TypedDict does not support issubclass checks (which was the initial issue in this thread), so in order to check if a type is a subtype of TypedDict in ModelField._type_analysis, it seems like we need to check if it is a subclass of TypedDict's metaclass. Unfortunately, _TypedDictMeta is protected in both the typing and typing_extensions module, so the mypy check won't allow it to be imported (probably for good reason). For now, I check self.type_.__class__.__name__ == '_TypedDictMeta', but I think there must be a better way to do this.
2. I'm not sure what the best spot is in ModelField._type_analysis to do the TypedDict subclass check. I put it before the origin checks to avoid the issue at the top of this thread, but there might be a better spot for it.
3. I added each TypedDict value type to self.sub_fields in ModelField._type_analysis, since it fit the pattern for validating Tuples, etc. However, since each value corresponds to a specific named key, I had to add a key_name attribute to ModelField with the key name in order to implement ModelField._validate_typeddict. I'm not sure if it's a good idea to add new attributes to ModelField for this, especially considering that it's a relatively niche feature.

Would appreciate feedback/suggestions before trying to make a pull request.

Edit: Forgot to mention, all tests are passing and all lines that I added should be covered.
Edit 2: I noticed that there's an optional total parameter to TypedDict, so I added support and tests for that.