Structuring fails when using subclass union inside another subclass union

[brunokim]

• cattrs version: 23.2.3
• Python version: 3.10.9
• Operating System: 6.8.0-40-generic #40~22.04.3-Ubuntu

Description

Full gist: https://gist.github.com/brunokim/1545287ec50c1383c2ed9aabfcd45753

I'm using inheritance to build two type hierarchies: Expr and Command, which includes Expr.

@frozen
class Expr:
    pass


@frozen
class Literal(Expr):
    value: int | str = field()


@frozen
class Symbol(Expr):
    name: str = field()


@frozen
class Command:
    tag = ""


@frozen
class Select(Command):
    tag = "select"

    exprs: tuple[Expr, ...] = field(converter=tuple)


@frozen
class Where(Command):
    tag = "where"

    expr: Expr = field()

Exprs can use the built-in field disambiguation for structuring, but there are more Commands than those in this MWE, which require a union strategy using the tag field

def _configure_expr_converter(converter):
    include_subclasses(Expr, converter)
    converter.register_structure_hook(Literal, lambda d, t: Literal(d["value"]))


def _configure_command_converter(converter):
    def union_strategy(union, converter):
        return configure_tagged_union(
            union, converter, tag_generator=lambda cls: cls.tag, tag_name="cmd"
        )

    include_subclasses(Command, converter, union_strategy=union_strategy)

Weirdly enough, I'm able to successfully structure fields in Select, where Exprs are within a tuple, but not in Where, which chooses the base empty Expr() type!

>>> _command_converter.structure({"cmd": "select", "exprs": [{"value": 1}, {"name": "foo"}]}, Command)
Select(exprs=(Literal(value=1), Symbol(name='foo')))
>>> _command_converter.structure({"cmd": "where", "expr": {"value": 1}}, Command)
Where(expr=Expr())

I'd expect to see Where(expr=Literal(value=1)) instead.

What I Did

I tried walking the conversion function, and it seems that the generated Where structuring function is hardcoded to use the base Expr, instead of attempting to use the subclass dispatcher.

  1  ->	def structure_Where(o, _, __cl=__cl, __c_cve=__c_cve, __c_avn=__c_avn, __c_structure_expr=__c_structure_expr, __c_type_expr=__c_type_expr):
  2  	  res = {}
  3  	  errors = []
  4  	  try:
  5  	    res['expr'] = __c_structure_expr(o['expr'], __c_type_expr)
  6  	  except Exception as e:
  7  	    e.__notes__ = getattr(e, '__notes__', []) + [__c_avn("Structuring class Where @ attribute expr", "expr", __c_type_expr)]
  8  	    errors.append(e)
  9  	  if errors: raise __c_cve('While structuring ' + 'Where', errors, __cl)
 10  	  try:
 11  	    return __cl(
(Pdb) 
 12  	      **res,
 13  	    )
 14  	  except Exception as exc: raise __c_cve('While structuring ' + 'Where', [exc], __cl)

  1  ->	def structure_Expr(o, _, __cl=__cl, __c_cve=__c_cve, __c_avn=__c_avn):
  2  	  res = {}
  3  	  errors = []
  4  	  if errors: raise __c_cve('While structuring ' + 'Expr', errors, __cl)
  5  	  try:
  6  	    return __cl(
  7  	      **res,
  8  	    )
  9  	  except Exception as exc: raise __c_cve('While structuring ' + 'Expr', [exc], __cl)

Thanks for any insight on what I should be doing!

[Tinche]

You can just flip the order of calling _configure_expr_converter and _configure_command_converter - the simpler one (_configure_expr_converter) should be called first. When I do that, the output becomes:

Literal(value=1)
Symbol(name='foo')
Select(exprs=(Literal(value=1), Symbol(name='foo')))
Where(expr=Literal(value=1))

The reason for this is that some hook factories (the attrs one being a prime example) fetch subhooks for all fields eagerly (at the time of hook creation). This is mostly for speed reasons, however it makes the order of hook creation matter. You can look at #560 (comment) for more details. We really need more docs here ;)

Let me know if you have further questions.

[brunokim]

Do you have any direction on where in the docs you'd like to draw attention to this order dependency? I'd like to contribute.