Support for examples section

[jaimergp]

Hi! I would like to start by saying this project has made mkdocs usable for our team and we are SO excited about finally ditching RST! I cannot emphasize this enough: Thank. You.

The only thing we are missing is that, currently, mkdocstrings doesn't support an Examples section, which normally include doctest formatted blocks. For example:

def example_function(arg1, kwarg=None) -> object:
    """
    Example function to demonstrate how APIs are rendered

    Parameters:
        arg1 (dict): Some description for this argument.
            This type (in parenthesis) is ignored.
        kwarg: Some more descriptions

    Returns:
        A description for the returned value

    Examples:

        >>> 2 + 2 == 4
        False

    """
    pass

gets rendered as:

The style is not bad, but it could be even better! Notice how pytest is able to parse the block just fine.

I can get what I want by using this other docstring:

def example_function(arg1, kwarg=None) -> object:
    """
    Example function to demonstrate how APIs are rendered

    Parameters:
        arg1 (dict): Some description for this argument.
            This type (in parenthesis) is ignored.
        kwarg: Some more descriptions

    Returns:
        A description for the returned value

    __Examples__

    ```python
    >>> 2 + 2 == 4
    False

    ```
    """
    pass

, but I find it a bit annoying (in order of importance):

1. Manually hardcoding the Examples heading style
2. Having to specify the code fences
3. Needing an extra blank line before the closing triple backticks so pytest can get the doctests correctly.

Maybe a good compromise would be to just add Examples to the recognized headers? That way I could write:

def example_function(arg1, kwarg=None) -> object:
    """
    Example function to demonstrate how APIs are rendered

    Parameters:
        arg1 (dict): Some description for this argument.
            This type (in parenthesis) is ignored.
        kwarg: Some more descriptions

    Returns:
        A description for the returned value

    Examples:

        ```python
        >>> 2 + 2 == 4
        False

        ```
    """
    pass

, which currently does not work as intended:

I'd be happy to help and contribute a PR if needed, but if I understood correctly, you are currently refactoring the backend and I don't know if this is the best moment to add a feature.

Thanks a lot!

[pawamoy]

Hi @jaimergp, thank you very much for the kind words and the very detailed issue! I'm glad that your team is enjoying this project 🙂

I totally agree that examples must be supported, and I think the preferred way would be the first you wrote, but it's open to discussion of course.

This feature request is one more thing in favor of implementing a proper "google-style docstring markdown extension". We currently parse docstrings in mkdocstrings (and soon in another project responsible for loading the docs only), but it would definitely be way better to have an extension doing that. It would have its own challenges but I think I came with a good design. This design is unfortunately still only available in my mind, but I plan on writing it down here, in a pinned issue, with a "needs help" tag. This would be great if contributors could help with it!

So, as you said in the last paragraph, now is not a good time for such a feature, but I hope I'll be able to come soon with the desired architecture, which will allow us to iteratively improve mkdocstrings.

You'll hear from me when things have moved forward 🙂

[pawamoy]

Hey, I'm close to a new release of mkdocstrings with the new architecture. The code responsible for parsing docstrings is now hosted at https://github.com/pawamoy/py-tkdocs, so I'm transfering this issue over there!

[jaimergp]

Oh, great news! Yes, of course, no probs!

Do you need help testing stuff out?

[pawamoy]

Well sure, I'll publish a new version of mkdocstrings, v0.9.0, and if you can test it to see if there are any bugs and report them it would be great 🙂

Then we'll be able to improve the handling of Examples section 😄

[pawamoy]

Hey again @jaimergp

Do you still want to contribute a PR to support the examples sections? I can guide you through the code if needed.

[jaimergp]

Sure! Where should I start looking in?

[pawamoy]

All the changes should happen in src/pytkdocs/parsers/docstrings.py.

• add a TITLES_EXAMPLES constant at the top, like TITLES_RETURN
• add a type to the Section.Type "enum"
• add a read_examples_section method to the DocstringParser class. It should be similar to read_return_section
• handle such sections in the parse method. Again, should be similar to how the return sections are handled

Now this would be the pytkdocs part. Then in mkdocstrings we would need to update the docstring.html template to support these type of sections 🙂

[pawamoy]

Any progress @jaimergp? If not, and if you don't mind, I'll start working on this 🙂

[jaimergp]

Sorry @pawamoy, I've been very slow to start new side projects and haven't been able to contribute time to this. Please go ahead and take over! Let me know if I can help you test the usage once it's ready!

[igonro]

Hey @pawamoy, how is this feature going? I would be interested too. I think I could help if needed.

[pawamoy]

I didn't start yet, so you can send a PR @igonro! Take a look at my comment above explaning how this can be done 🙂

There were some changes, the module is now src/pytkdocs/parsers/docstrings/google.py, and the method is parse_sections.

[igonro]

Hey @pawamoy, I'm starting this feature right now! But I found a little problem with the make setup command, it didn't find the pipcommand. In order to fix this, I changed pip and put instead python -m pip. I think this is a safer approach. So later I can send this change as PR if you want.

[pawamoy]

Ah, sorry about that, thanks for reporting that issue. I'll update it myself, thanks (I update my projects from template so it'll be easier to fix the template directly).

[igonro]

Man, I'm struggling to implement this functionality. I'm trying to write the read_examples_section, but I don't know very well what type of Section I have to return.

Edit: Nevermind, I've just realized that I didn't edit the docstring.html in mkdocstrings.

[pawamoy]

A new one! Just add a type in the Section.Type enum, in the base module, like EXAMPLE = "example".

EDIT: oh okay 🙂

[pawamoy]

I think we'll have to be a bit smarter in mkdocstrings for these examples section. If the codehilite extension is listed in the config, we should override its guess_lang option to True to make sure code blocks are highlighted as Python code. But we'll see that once pytkdocs can read such sections 🙂

Also don't hesitate to open your PR early. You can set it as "draft" of "wip", and this will allow us to discuss about the code 🙂

[igonro]

Yes. I think I will do that, because I'm a little bit blocked. 😞

[pawamoy]

Oh, I might have forgotten to mention the serialization: https://github.com/pawamoy/pytkdocs/blob/master/src/pytkdocs/serializer.py#L150 😅

[pawamoy]

We really should have a proper JSON encoder instead of these functions, but that's another story.

[igonro]

Ok. I will be working the following days in this feature when I can dedicate some time. It might take me a few days, but step by step I will eventually get it done 😃

[pawamoy]

Of course, thank you very much for contributing to mkdocstrings 🙂