XyzOutputTypeDef not usable for XyzTypeDef

[sodul]

After upgrading to boto3 1.28 from 1.26 we noticed that some output types have become incompatible as input types, raising mypy errors.

For example:

incompatible type "list[TagOutputTypeDef]"; expected "list[TagTypeDef] | None" 
incompatible type "list[UserIdGroupPairOutputTypeDef]"; expected "Sequence[UserIdGroupPairTypeDef]"
incompatible type "list[IpRangeOutputTypeDef]"; expected "Sequence[IpRangeTypeDef]"

If you look at the type annotation I would assume that they should be compatible:

IpRangeOutputTypeDef = TypedDict(
    "IpRangeOutputTypeDef",
    {
        "CidrIp": str,
        "Description": str,
    },
)


IpRangeTypeDef = TypedDict(
    "IpRangeTypeDef",
    {
        "CidrIp": str,
        "Description": str,
    },
    total=False,
)

But somehow you can't use a IpRangeOutputTypeDef as input for a IpRangeTypeDef which is less strict. All the keys being the same with the only difference being that ``IpRangeTypeDefhastotal=False` which is less strict.

I'm not sure if this should be filed as a bug or if we are suppose to handle these things differently so I posted it under Discussions.

[vemel]

Thank you for the report.

I will investigate if I can fix this error for mypy.

[vemel]

Fixed in mypy_boto3_builder 7.15.2

[sodul]

Thanks for the quick fix.

I installed the latest boto and stubs (short list here, but all the versions are the same):

> pip freeze | grep boto
boto3==1.28.12
boto3-stubs==1.28.12
botocore==1.31.12
botocore-stubs==1.29.162

I'm seeing different but very similar errors:

error: Incompatible return value type (got "list[TargetOutputTypeDef]", expected "list[TargetTypeDef]")  [return-value]
error: Incompatible return value type (got "list[InventoryConfigurationOutputTypeDef]", expected "list[InventoryConfigurationTypeDef]")  [return-value]
error: Argument "DistributionConfig" to "update_distribution" of "CloudFrontClient" has incompatible type "DistributionConfigOutputTypeDef"; expected "DistributionConfigTypeDef"  [arg-type]

I'm also seeing this for dynamoDB:

error: Incompatible return value type (got "list[dict[str, AttributeValueTableTypeDef]]", expected "list[dict[str, bytes | bytearray | str | int | Decimal | bool | set[int] | set[Decimal] | set[str] | set[bytes] | set[bytearray] | Sequence[Any] | Mapping[str, Any] | None]]")  [return-value]

But that last one is probably because the DynamoDB types are now better handled in the stubs and we had our own private type.

[vemel]

I just released mypy-boto3-dynamodb 1.28.15.post1 with the fix included. You can help me if you test this new package with your code.

Please let me know if it works as it should.

[vemel]

@sodul please let me know if the new version works for you. Feel free to close the discussion if it does, or report any issues you encountered.

[sodul]

Tested today with:

boto3==1.28.16
boto3-stubs==1.28.16.post1
botocore==1.31.16
botocore-stubs==1.29.162

Got these errors:

error: Argument "DistributionConfig" to "create_distribution" of "CloudFrontClient" has incompatible type "dict[str, object]"; expected "DistributionConfigTypeDef"  [arg-type]
error: Incompatible types in assignment (expression has type "GetDistributionResultTypeDef", variable has type "CreateOriginAccessControlResultTypeDef")  [assignment]
error: TypedDict "CreateOriginAccessControlResultTypeDef" has no key "Distribution"  [typeddict-item]
error: Incompatible return value type (got "CreateOriginAccessControlResultTypeDef", expected "dict[str, Any]")  [return-value]
error: Argument "DistributionConfig" to "update_distribution" of "CloudFrontClient" has incompatible type "DistributionConfigOutputTypeDef"; expected "DistributionConfigTypeDef"  [arg-type]
error: Incompatible return value type (got "list[TargetOutputTypeDef]", expected "list[TargetTypeDef]")  [return-value]
error: Incompatible return value type (got "list[InventoryConfigurationOutputTypeDef]", expected "list[InventoryConfigurationTypeDef]")  [return-value]

Let me try 1.28.15.post1.

[vemel]

Use 1.28.16. it is the latest.

[sodul]

With:

boto3==1.28.15
boto3-stubs==1.28.15.post1
botocore==1.31.15
botocore-stubs==1.29.162

I get:

error: Argument "DistributionConfig" to "create_distribution" of "CloudFrontClient" has incompatible type "dict[str, object]"; expected "DistributionConfigTypeDef"  [arg-type]
error: Argument "DistributionConfig" to "update_distribution" of "CloudFrontClient" has incompatible type "DistributionConfigOutputTypeDef"; expected "DistributionConfigTypeDef"  [arg-type]
error: Incompatible return value type (got "CreateOriginAccessControlResultTypeDef", expected "dict[str, Any]")  [return-value]
error: Incompatible return value type (got "list[InventoryConfigurationOutputTypeDef]", expected "list[InventoryConfigurationTypeDef]")  [return-value]
error: Incompatible return value type (got "list[TargetOutputTypeDef]", expected "list[TargetTypeDef]")  [return-value]
error: Incompatible types in assignment (expression has type "GetDistributionResultTypeDef", variable has type "CreateOriginAccessControlResultTypeDef")  [assignment]

So the issues are present with both 1.28.15 and 1.28.16.

[sodul]

@vemel Everything is passing as expected with the latest version:

boto3==1.28.17
boto3-stubs==1.26.17
botocore==1.31.17
botocore-stubs==1.29.162

Thanks a lot for resolving this.

[sodul]

For reference these are relevant issues in mypy:
python/mypy#12142
python/mypy#12200

[alicederyn]

I am working on a PEP for this kind of situation; input would be greatly appreciated.

Current PEP proposal: https://peps.python.org/pep-0705/
Discussion thread: https://discuss.python.org/t/pep-705-typedmapping/24827/

[vemel]

@alicederyn Thank you! I would be happy to help.

[sodul]

@vemel FYI mypy has improved the issue in the 0.981 release python/mypy#13311