Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate and publish Python API docs #77

Open
ScottTodd opened this issue Aug 13, 2024 · 6 comments
Open

Generate and publish Python API docs #77

ScottTodd opened this issue Aug 13, 2024 · 6 comments
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@ScottTodd
Copy link
Member

(Issue migrated from nod-ai/SHARK-ModelDev#106)


Users would find it helpful to view API documentation pages instead of just needing to read through source files, samples, and tests. I would also like to link to API docs from https://iree.dev/guides/ml-frameworks/pytorch/.

IREE Python used readthedocs (with mixed results): https://iree-python-api.readthedocs.io/en/latest/

Could see what PyTorch uses for https://pytorch.org/docs/stable/index.html (sample page: torch.nn.Linear)

Even if we don't end up publishing API docs, going through and adding docstrings and help text could still be useful.

@ScottTodd ScottTodd added the documentation Improvements or additions to documentation label Aug 13, 2024
@ScottTodd
Copy link
Member Author

Bumping up priority for this now that there is more code in https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/kernel/wave that we'd like documentation for.

I'm currently thinking we can use Sphinx + readthedocs, hosted at a URL like iree-turbine.readthedocs.io, with a few pages containing larger blocks of guide text (example: https://github.com/iree-org/iree/blob/main/docs/api_docs/python/compiler/tools.rst --> https://iree-python-api.readthedocs.io/en/latest/compiler/tools.html).

Once we have more content and users/customers, we can decide if we want to merge or split documentation websites. We could for example:

  • split TKW from iree-turbine
  • merge iree-turbine API docs with IREE API docs
  • host API docs on https://iree.dev/

@ScottTodd ScottTodd self-assigned this Jan 17, 2025
@ScottTodd
Copy link
Member Author

ScottTodd commented Jan 24, 2025

I've started setting up scaffolding for a documentation site on my branch here: https://github.com/ScottTodd/iree-turbine/tree/docs-sphinx . Instructions for working on the site locally are in this README: https://github.com/ScottTodd/iree-turbine/tree/docs-sphinx/docs/api_docs

Some docstring warnings were surfaced during the build:

iree-turbine\iree\turbine\aot\compiled_module.py:docstring of iree.turbine.aot.compiled_module.CompiledModule.create_from_dict:7: ERROR: Unexpected indentation.
iree-turbine\iree\turbine\aot\compiled_module.py:docstring of iree.turbine.aot.compiled_module.CompiledModule.create_from_dict:5: WARNING: Inline literal start-string without end-string.
iree-turbine\iree\turbine\aot\compiled_module.py:docstring of iree.turbine.aot.compiled_module.CompiledModule.create_from_dict:8: WARNING: Block quote ends without a blank line; unexpected unindent.
iree-turbine\iree\turbine\aot\compiled_module.py:docstring of iree.turbine.aot.compiled_module.CompiledModule.create_from_dict:8: WARNING: Inline literal start-string without end-string.
iree-turbine\iree\turbine\aot\compiled_module.py:docstring of iree.turbine.aot.compiled_module.CompiledModule.create_from_dict:8: WARNING: Inline interpreted text or phrase reference start-string without end-string.
iree-turbine\iree\turbine\aot\fx_programs.py:docstring of iree.turbine.aot.fx_programs.FxProgramsBuilder:11: ERROR: Unexpected indentation.
iree-turbine\iree\turbine\aot\fx_programs.py:docstring of iree.turbine.aot.fx_programs.FxProgramsBuilder:9: WARNING: Inline literal start-string without end-string.
iree-turbine\iree\turbine\aot\fx_programs.py:docstring of iree.turbine.aot.fx_programs.FxProgramsBuilder:17: ERROR: Unexpected indentation.
iree-turbine\iree\turbine\aot\fx_programs.py:docstring of iree.turbine.aot.fx_programs.FxProgramsBuilder:19: WARNING: Inline literal start-string without end-string.
iree-turbine\iree\turbine\aot\fx_programs.py:docstring of iree.turbine.aot.fx_programs.FxProgramsBuilder:19: WARNING: Inline interpreted text or phrase reference start-string without end-string.
iree-turbine\iree\turbine\aot\__init__.py:docstring of iree.turbine.aot.exporter.export:16: WARNING: Inline emphasis start-string without end-string.
iree-turbine\iree\turbine\aot\builtins\jittable.py:docstring of iree.turbine.aot.builtins.jittable.jittable:1: WARNING: duplicate object description of iree.turbine.aot.builtins.jittable.jittable, other instance in aot/aot, use :no-index: for one of them

Next steps, in no particular order:

@ScottTodd
Copy link
Member Author

ScottTodd commented Jan 24, 2025

A few screenshots of what is currently generated:

Image

Image

For the second screenshot, this code is pulling in all of https://github.com/iree-org/iree-turbine/blob/main/iree/turbine/kernel/ops/wave_ops.py right now, hence the "settle on a website layout" point. Going to need to split that into a few subpages to make it manageable.

.. automodule:: iree.turbine.kernel
  :imported-members:
  :members:
  :undoc-members:

ScottTodd added a commit that referenced this issue Jan 31, 2025
Progress on #77. There is
still plenty to iterate on, but this gets something started that
developers can start using and improving.

> [!IMPORTANT]
> **A demo website is hosted at
https://iree-turbine-scotttoddtest.readthedocs.io/en/latest to help with
review.**

I also claimed https://app.readthedocs.org/projects/iree-turbine/ and
will configure it once this is merged. The final hosted site will be at
https://iree-turbine.readthedocs.io/en/latest/ and will update after
each commit to the `main` branch in this repository.

## Notes

* I've structured the site into a few main sections:
  * API reference material
    * "IREE Turbine Core"
    * "TK and TKW"
  * Project documentation
    * Infrastructure
* Some module setups are conflicting with how
[`autodoc`](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html)
wants modules to be defined. I worked with what we have now, but we may
want to change how modules are defined and imports are used. I'm fairly
inexperienced at this level, but I think a reliance on relative imports
like these is leading to packages that can't easily be used in
isolation:
https://github.com/iree-org/iree-turbine/blob/a7c333236f37fefffd513e8b97eac4085d0bf72e/iree/turbine/kernel/wave/codegen.py#L83-L94

This show up in the docs where something like `automodule::
iree.turbine.aot.builtins` for a given module is missing significant
classes and functions and code paths like
https://github.com/iree-org/iree-turbine/tree/main/iree/turbine/kernel/compiler
have no `__init__.py` to create a package.
@ScottTodd
Copy link
Member Author

The initial website is now live at https://iree-turbine.readthedocs.io/en/latest/ . I've also configured a webhook in this repository so it should update after each push to the default branch.

@ScottTodd
Copy link
Member Author

Source code and instructions for working on the website can be found at https://github.com/iree-org/iree-turbine/tree/main/docs.

ScottTodd added a commit that referenced this issue Jan 31, 2025
Progress on #77.

Note that the README is included at
https://pypi.org/project/iree-turbine/ too.

* Status badge showing readthedocs build status
* Link to https://iree-turbine.readthedocs.io/
* Switch pytorch install instructions off of the `test` channel
* Document readthedocs dashboard and publishing mechanism
ScottTodd added a commit to iree-org/iree that referenced this issue Feb 3, 2025
Progress on iree-org/iree-turbine#77.

* Drop warning about PyTorch support being unstable while under
development. I think it is mature enough now.
* Add links to new documentation at https://iree-turbine.readthedocs.io/
* Make the link to https://iree-python-api.readthedocs.io/ more
prominent
ita9naiwa pushed a commit to ita9naiwa/iree that referenced this issue Feb 4, 2025
Progress on iree-org/iree-turbine#77.

* Drop warning about PyTorch support being unstable while under
development. I think it is mature enough now.
* Add links to new documentation at https://iree-turbine.readthedocs.io/
* Make the link to https://iree-python-api.readthedocs.io/ more
prominent

Signed-off-by: Hyunsung Lee <ita9naiwa@gmail.com>
@ScottTodd
Copy link
Member Author

We should make a pass through our docstrings and check that they all render correctly. Some new ones added in #418 (comment) could use some tweaks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

1 participant