Support enriched metadata

[brettcannon]

Take raw metadata and wrap data as appropriate with more enriched representations (e.g., requirements.Requirement for Requires-Dist).

Part of #570

[brettcannon]

Docs are in! pypa/packaging.python.org#1283 will let me go back in and add links to the core metadata fields that I couldn't link to.

All of CI is now passing, so does anyone want to give this a once-over?

[pradyunsg]

Did a quick skim, LGTM!

[brettcannon]

@pradyunsg thanks for the skim!

I'll give @dstufft some time to look it over before I merge.

[brettcannon]

What do we want to do about core metadata 1.1 and its fields that only existed in that version (i.e., requires, provides, and obsoletes)? Right now the fields are considered a validation failure since there just isn't a concept of those fields in the class. Should I just assign that data to the roughly corresponding core metadata 1.2 fields? Should I just not support core metadata 1.0 and 1.1 (or just 1.1)?

[dstufft]

I don't think we've ever removed a field? The PEP for Metadata 1.2 or 2.1 never said that those fields were removed, just deprecated.

[dstufft]

It looks like pypa/packaging.python.org#386 just missed documenting those fields, because PEP 345 doesn't document them (other than as a footnote to say they're deprecated), and AFAIK most or all existing implementations of metadata reading that attempt to be comprehensive (so not things trying to get a single field or something) support them still.

[dstufft]

This approach looks good to me, I think it covers all the bases/invariants that I was trying to handle in my original PR, looks great!

[brettcannon]

I don't think we've ever removed a field? The PEP for Metadata 1.2 or 2.1 never said that those fields were removed, just deprecated.

Ah, that makes sense. I have been following the spec as "the spec", and so figured the fields were gone. I will add the missing ones in as str-only fields and that should take care of things! I'll also open an issue on packaging.python.org about the missing fields.

[brettcannon]

Actually, Donald reported the doc bug in pypa/packaging.python.org#1107 and there's a PR at pypa/packaging.python.org#1138 waiting on Donald's review. 😅

[dstufft]

Donald reported the doc bug in pypa/packaging.python.org#1107

Welp.

[brettcannon]

I wrote a script that tried to pull a METADATA file from every project on PyPI. The issues that I saw, in order of most to least frequent, were:

1. Using core metadata version 2.0
2. Improper description content type
3. Invalid extras name (people definitely want a name to specify development-only requirements)
4. Multi-line summaries
5. Repeated fields (in one case because they messed up their Description formatting and made it a blank line and then followed with the whole description in the body)

I need to make a change to make it more descriptive what triggered when the name is invalid (since it's reusing utils.canonicalize_name() it has a generic exception message), but otherwise I think that's the only change left!

[brettcannon]

OK, I updated the appropriate code so that all failed field validations raise InvalidMetadata (with appropriate __cause__ values). That way you only have to worry about catching a single exception type instead of having to know exactly what code was used to perform the validation or transformation of code.

And with that, I think this PR is ready! I'm waiting for CI to finish, so hopefully I can merge this tomorrow.