Support type annotations in Returns and Yields sections

[marvinpfoertner]

Currently, when using numpydoc-style docstrings alongside Python type annotation, the types of returns or yields must be specified twice, which is tedious and also error prone. I would like to propose a backward-compatible addition to the docstring specification, which allows returns/yields to be specified without return types:

Returns
-------
return1_name :
    Description of `return1_name`.
return2_name :
    Description of `return2_name`.

Note the addition of the colons to distinguish return names and return types in a backward-compatible way.
Additionally, for a single return value with a Python return type annotation, it would be great if the following would be possible:

Returns
-------
Description of the return

or

Returns
-------
    Description of the return

The same suggestions should also work for the Yields section

[rgommers]

This is already an option in Sphinx itself, but IIRC not very usable yet. I agree this would be nice to have. Can you please look into how Sphinx does this, and if anything is needed from Numpydoc to integrate with that?

[marvinpfoertner]

From experimenting with the sphinx/napoleon feature you mentioned, I believe that it is sort of broken, since it does not really adhere to the specification and behaves differently for one and multiple return values. I can follow up with specifics once I find the time to set up a reproducible example if you want.

I think it would be a good start to define something like my proposal above in the specification of the numpydoc-style in order to define the semantics and desired behavior in the presence of PEP 484 type hints.

This will make a consistent and compatible implementation in tools like sphinx and pylint easier.

[rgommers]

So it should work with autodoc_typehints: https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_typehints
There's also https://pypi.org/project/sphinx-autodoc-typehints/#description, which seems actively developed right now and relevant.

If that could be made to work with

Returns
-------
return1_name :
    Description of `return1_name`.
return2_name :
    Description of `return2_name`.

then I think we're good, and I'm +1 on accepting that feature. That's the main target I'd think.

I'm less sure about the single return format you propose; certainly seems lower prio.

[marvinpfoertner]

@rgommers I opened an issue in the Sphinx repo. I'd be happy if you could chime in in case you have any objections to my description there.

[marvinpfoertner]

I believe we've reached a deadlock situation, since the sphinx maintainers don't want to add this feature, unless it is added to the numpydoc specification. See sphinx-doc/sphinx#10134 (comment)

[rgommers]

Okay, then let's approve this as part of the Numpydoc spec I'd say (the part in #356 (comment)). Any objections @numpy/numpydoc-devs?

[marvinpfoertner]

Just to make sure, since #356 (comment) only mentions the Returns documentation: Is this going to be added for both the Returns and Yields sections?

Personally, I'm strongly in favor of adding it to both sections.

[rossbar]

Adding the following pattern to the standard:

Returns
-------
name : 
    description

seems fine to me. As noted, it's backward-compatible from the perspective of docstring parsing as this will parse fine w/ numpydoc 1.2:

>>> def foo():
...     """foo's description
... 
...     Returns
...     -------
...     my_float : 
...         a value of 1.0
...     """
...     my_float = 1.0
...     return my_float
>>> from numpydoc.docscrape import NumpyDocString
>>> foo_nds = NumpyDocString(foo.__doc__)
>>> foo_nds["Returns"]
[Parameter(name='my_float', type='', desc=['a value of 1.0'])]

It's just not explicitly mentioned in the standard.

It will require a change to the validation, as RT02 is more strict than the docstring parser and checks that the Returns section matches one of the two patterns mentioned in the standard. I think this is fine as long as the change is documented in the release notes.

IMO I don't much see the utility of having the Returns section list the name of the object instead of the type, since the name for returned object isn't relevant outside the function scope. I'd propose a different solution to the problem of duplicated type info in the signature and docstring (namely extracting typing info directly from the docstring), but that's a separate discussion. Since it's already technically supported, seems to be desired in multiple places, and shouldn't break anyone I have no strong objections.

[larsoner]

IMO I don't much see the utility of having the Returns section list the name of the object instead of the type, since the name for returned object isn't relevant outside the function scope.

To me there are a couple of reasons. First, it's often useful as a quick shorthand, because you can more quickly get the variable's purpose, e.g. I often can get it immediately from a quick glance at the name if I'm familiar with the function and just can't remember the order for example:

Returns
--------
n_excluded : int
    The number of things that were excluded.
n_included : int
    The number of things that were included.

This to me is faster than where I have to read some or most of a sentence:

Returns
---------
int
    The number of things that were excluded.
int
    The number of things that were included.

Second, it helps with code uniformity (at least in some places and projects), as it gives users (and devs for internal use!) some hint about what good names would be to standardize around for the returned variables where it's reasonable to do so.

[BobDotCom]

What's the status of this?