AssertionError Serializer #120

danilocastelhano1 · 2020-07-10T20:04:57Z

i have an serializer who give me an exception, follow my traceback:

File "./manage.py", line 17, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line
381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line
375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323,
 in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364,
 in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/management/commands/specta
cular.py", line 44, in handle
    schema = generator.get_schema(request=None, public=True)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/generators.py", line 161,
in get_schema
    paths=self.parse(request, public),
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/generators.py", line 141,
in parse
    operation = view.schema.get_operation(path, path_regex, method, self.registry)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 77, in g
et_operation
    operation['responses'] = self._get_response_bodies()
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 822, in
_get_response_bodies
    return {'200': self._get_response_for_code(response_serializers)}
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 847, in
_get_response_for_code
    component = self.resolve_serializer(serializer, 'response')
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 919, in
resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 580, in
_map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 639, in
_map_basic_serializer
    schema = self._map_serializer_field(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 398, in
_map_serializer_field
    component = self.resolve_serializer(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 919, in
resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 580, in
_map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 639, in
_map_basic_serializer
    schema = self._map_serializer_field(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 403, in
_map_serializer_field
    component = self.resolve_serializer(field.child, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 919, in
resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 580, in
_map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 639, in
_map_basic_serializer
    schema = self._map_serializer_field(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 403, in
_map_serializer_field
    component = self.resolve_serializer(field.child, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 919, in
resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 580, in
_map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 639, in
_map_basic_serializer
    schema = self._map_serializer_field(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 398, in
_map_serializer_field
    component = self.resolve_serializer(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 919, in
resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 580, in
_map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 639, in
_map_basic_serializer
    schema = self._map_serializer_field(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 403, in
_map_serializer_field
    component = self.resolve_serializer(field.child, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 919, in
resolve_serializer
    component.schema = self._map_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 580, in
_map_serializer
    schema = self._map_basic_serializer(serializer, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 639, in
_map_basic_serializer
    schema = self._map_serializer_field(field, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 403, in
_map_serializer_field
    component = self.resolve_serializer(field.child, direction)
  File "/usr/local/lib/python3.7/site-packages/drf_spectacular/openapi.py", line 907, in
resolve_serializer
    assert is_serializer(serializer)
AssertionError

The text was updated successfully, but these errors were encountered:

tfranzel · 2020-07-11T02:14:48Z

hey! it is nearly impossible to guess what went wrong there. all i can see that it is probably a heavily nested serializer (which should be ok) and that the assert got hit by something that was not a serializer. this should be a very unlikely event. i would need some kind of serializer example or reproduction on how to provoke the error.

jayvdb · 2020-07-13T12:33:33Z

I just hit this.

I suggest that assert is never used without a second arg which explains the failed assertion. There are code quality tools which can enforce this. If you want these assertions to cause issues to be raised, it helps to have assertion messages let people raise bugs which can be used to identify the problem.

Since this component is py36+, it could also use should_be instead of assert, which auto-voodoo creates useful assertion error messages.

And as usual, I would prefer these are warnings. In my case it is a custom lazy-ily defined serializer and it could be safely ignored as inspecting it wont work anyway. The following worked for me.

-        assert is_serializer(serializer)
+        if not is_serializer(serializer):
+            return

tfranzel · 2020-07-13T12:44:43Z

certainly something to consider. i did not anticipate this being hit, except for very rare cases. i'll look into it whether a more meaningful assert message or a non-fatal error/warn is more appropriate.

should_be looks nice but i don't want to add 2 deps for this one line.

@danilocastelhano1 which library with customized serializer logic did you use?

tfranzel · 2020-07-13T20:13:38Z

@jayvdb i had another look and i remember now. that assertion is supposed to be just that,a is a basic assumption that "must" hold true. (Almost) all callers of that function ensure that assumption and if not throw a warning and do a fallback. The only way i can see to get there unchecked is by either using PolymorphicSerializer or PolymorphicProxySerializer.

can you elaborate on how you got that error? that would be most helpful. i might have missed a case somewhere.

danilocastelhano1 · 2020-07-14T11:59:12Z

@tfranzel i use docusign library, there is a big serializer in there

tfranzel · 2020-07-14T12:18:47Z

@danilocastelhano1 i see.

i cannot find a docusign lib that has a serializer in it. can you provide a link?
did you use PolymorphicSerializer or PolymorphicProxySerializer in combination with that serializer?
can you post a snippet on how you provoke the error? i guess how you use @extend_schema there.

danilocastelhano1 · 2020-07-14T12:34:01Z

in docusign, I use customizable serializers, I believe this is what is causing the exception, if I change my serializer to one that is: MySerializer (serializers.ModelSerializer), it works correctly, but if I create a customizable serializer: eg: MyCustomSerializer ( serializers.Serializer), generate the mentioned error.

tfranzel · 2020-07-14T12:47:41Z

ahh i understand, but customized serializers should never provoke this error, as long as they are based on serializers.BaseSerializer. that includes serializers.Serializer!

pretty much every test we have is using this, where it has never been a problem. example: class LikeSerializer(serializers.Serializer):

also the check is pretty trivial:

drf-spectacular/drf_spectacular/plumbing.py

Lines 102 to 107 in 7600cbe

    
           def is_serializer(obj) -> bool: 
        
               from drf_spectacular.serializers import OpenApiSerializerExtension 
        
               return ( 
        
                   isinstance(force_instance(obj), serializers.BaseSerializer) 
        
                   or bool(OpenApiSerializerExtension.get_match(obj)) 
        
               )

are you sure you are actually using rest_framework.serializers.Serializer as base class? something must be different. i'm a bit puzzled.

danilocastelhano1 · 2020-07-14T13:15:07Z

i think i found the error, it is in:

my_field = ListSerializer(required=False, child=serializers.CharField(required=False))

when i use ListSerializer, gives the error above, ListSerializer belongs to rest_framework->serializers.py
i'm using this version of djangorestframework==3.11.0

jayvdb · 2020-07-14T15:50:25Z

can you elaborate on how you got that error? that would be most helpful. i might have missed a case somewhere.

You have missed the case where the serializer doesnt inherit from those classes. :-)
https://github.com/peopledoc/django-formidable/blob/master/formidable/serializers/child_proxy.py is the one I encountered a few days ago.

tfranzel · 2020-07-14T17:01:15Z

@danilocastelhano1 ... ahh that makes absolute sense! now we are getting to the bottom of it. DRF automatically translates XSerializer(many=True) to ListSerializer(child=XSerializer()) internally. so the assumption is that child needs to be a serializer and not a field. i have to check if what you are doing is actually officially alllowed by DRF. i'm not so sure.

https://www.django-rest-framework.org/api-guide/serializers/#listserializer
The ListSerializer class provides the behavior for serializing and validating multiple objects at once. You won't typically need to use ListSerializer directly, but should instead simply pass many=True when instantiating a serializer.

tfranzel · 2020-07-14T17:23:04Z

@jayvdb well i wouldn't call that missed. that is on purpose. DRF required the Serializer interface from objects that are supposed to be "serializers". an arbitrary object will not do here. it is a safeguard against crazy and literally every non-fringe DRF app implements that interface. There is a lot of magic going on in Views and Serializers that we rely on. Same goes for DRF itself.

We have PolymorphicProxySerializer (not derived from BaseSerializer) which does similar proxy magic. special things that deviate from the norm require a Extensions like PolymorphicProxySerializerExtension. Otherwise all bets are off. We have to draw a line somewhere 😄

danilocastelhano1 · 2020-07-14T17:24:39Z

I think it is official, because in drf-yasg works good, and in drf-spectacular gives the error

You can create an serializer:
class TestSerializer(serializers.Serializer):
# Works fine!
field1 = serializers.CharField(required=False)
# Works fine!
field2 = serializers.IntegerField(required=False)
# Error in fied 3!
field3 = ListSerializer(required=False, child=serializers.CharField(required=False))

and a viewset:
class TestViewSet(viewsets.GenericViewSet, mixins.ListModelMixin,
mixins.RetrieveModelMixin):
queryset = Test.objects.all()
permission_classes = [IsAuthenticated]
serializer_class = TestSerializer

tfranzel · 2020-07-14T18:18:39Z

@danilocastelhano1 well that was a surprise! i validated that it works on the API as you said. today i learnt something new 😄 ListSerializer accepts Field as child, which means it is indeed a bug. still i think its unusual usage. i'll come up with a bugfix. thanks for digging in.

danilocastelhano1 · 2020-07-14T18:40:16Z

Glad to help and improve the tool. tks.

tfranzel · 2020-07-14T21:36:35Z

@danilocastelhano1 can you check if that works for you? close if issue is resolved.

fwiw, my confusion came from the fact that there is a dedicated class for exactly that purpose: serializers.ListField. that would have worked right off the bat. However, DRF code mentions that they are closely related, but since your usage does also work, we will support both cases.

jayvdb · 2020-07-15T00:30:19Z

Otherwise all bets are off. We have to draw a line somewhere

I agree. I just wish you didnt force failure on my entire API because of one part of it that I dont want or expect to be exposed in the OpenAPI.

tfranzel · 2020-07-15T10:20:27Z

@jayvdb lets move your problem to another issue (#126). they hit the same assert but because of different reasons.

danilocastelhano1 · 2020-07-15T12:27:07Z

morning guys, how can i test? by pip install drf-spectacular? first i ran pip uninstall drf-spectacular then i ran pip install drf-spectacular so the error persists, i noted that file: openapi.py do not changed, in github the file (https://github.com/tfranzel/drf-spectacular/blob/master/drf_spectacular/openapi.py) has 943 lines vs 933 when i ran pip install drf-spectacular

tfranzel · 2020-07-15T12:47:38Z

morning!

pip uninstall drf-spectacular
pip install git+https://github.com/tfranzel/drf-spectacular

should do it. then you have the current master state

danilocastelhano1 · 2020-07-15T12:59:34Z

Nice @tfranzel Woked good thanks a lot!

tfranzel added the bug Something isn't working label Jul 14, 2020

tfranzel added a commit that referenced this issue Jul 14, 2020

allow for field child on ListSerializer. #120

7690b33

tfranzel mentioned this issue Jul 15, 2020

AssertionError on resolve_serializer #126

Closed

danilocastelhano1 closed this as completed Jul 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AssertionError Serializer #120

AssertionError Serializer #120

danilocastelhano1 commented Jul 10, 2020

tfranzel commented Jul 11, 2020

jayvdb commented Jul 13, 2020

tfranzel commented Jul 13, 2020

tfranzel commented Jul 13, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

jayvdb commented Jul 14, 2020

tfranzel commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

jayvdb commented Jul 15, 2020

tfranzel commented Jul 15, 2020

danilocastelhano1 commented Jul 15, 2020

tfranzel commented Jul 15, 2020

danilocastelhano1 commented Jul 15, 2020

AssertionError Serializer #120

AssertionError Serializer #120

Comments

danilocastelhano1 commented Jul 10, 2020

tfranzel commented Jul 11, 2020

jayvdb commented Jul 13, 2020

tfranzel commented Jul 13, 2020

tfranzel commented Jul 13, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

jayvdb commented Jul 14, 2020

tfranzel commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

danilocastelhano1 commented Jul 14, 2020

tfranzel commented Jul 14, 2020

jayvdb commented Jul 15, 2020

tfranzel commented Jul 15, 2020

danilocastelhano1 commented Jul 15, 2020

tfranzel commented Jul 15, 2020

danilocastelhano1 commented Jul 15, 2020