Add autoloading tutorial

[shink]

Fixes #3039

This PR adds a tutorial about the out-of-tree extension autoloading mechanism introduced in pytorch/pytorch#122468 and implemented in pytorch/pytorch#127074

cc: @jgong5 @Yikun @hipudding @FFFrog @albanD

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3037

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[svekars]

Please make sure to review the Tutorial Submission Policy

[albanD]

Thanks for writing a tutorial on this. I think it is indeed a good topic to cover.
I think it would be good to take a step back and clarify who the audience is for this and focus the content on providing the information that audience is looking for.

In particular, my expectation is that the audience are the third party extension writers who should use this feature to improve seamless integration within PyTorch. In that case,
I will keep the intro at the top but make the "How to apply this to out-of-tree extensions the main section after that.

The How it works is covered in the RFC IIRC, maybe linking to it would be enough? I do agree that pointing out the env variable to disable this feature is a good thing to mention here though.

For the examples section, that makes sense to have this at the end as "example projects using this feature as well" but I would limit it to code pointers into how they did the integration on their github repos code. I don't think the code samples add much?

[shink]

@svekars @albanD Thanks so much for your comments!

Please make sure to review the Tutorial Submission Policy

Oh I'm so sorry, I will make sure to review this policy and adhere to it.

Thanks for writing a tutorial on this. I think it is indeed a good topic to cover. I think it would be good to take a step back and clarify who the audience is for this and focus the content on providing the information that audience is looking for.

In particular, my expectation is that the audience are the third party extension writers who should use this feature to improve seamless integration within PyTorch. In that case, I will keep the intro at the top but make the "How to apply this to out-of-tree extensions the main section after that.

The How it works is covered in the RFC IIRC, maybe linking to it would be enough? I do agree that pointing out the env variable to disable this feature is a good thing to mention here though.

For the examples section, that makes sense to have this at the end as "example projects using this feature as well" but I would limit it to code pointers into how they did the integration on their github repos code. I don't think the code samples add much?

Thanks for your professional review! I totally agree with your suggestion. I'm making changes.

[albanD]

Re-shuffle looks good. Thanks for the quick update.
Only the TODO in the examples section but the rest sounds good to me at a high level. Will let @svekars review in case there is any finer grain issue needed.

[shink]

@jgong5 Please review the HPU example section, thanks!

[jgong5]

The diagram seems missing a pointer from import torch to the entry points to load - the loading of entry points is triggered by import torch, right?

[shink]

yes, I will make some changes to this diagram

[shink]

@jgong5 updated, please have a look

[shink]

@jgong5 should the implementation example of this mechanism be added here?

[jgong5]

Yes, please leave it here for now. @bsochack is the habana bridge ready for autoloading?

[bsochack]

It is implemented and verified. Not yet released.

[shink]

@bsochack we need an implementation example here, can you provide one? Thanks!

[bsochack]

Any suggestion how it should be provided?

The only challenge that we had is the circular dependencies. We solved it in the way below. Other than that, it worked out of box.

habana_frameworks/init.py:

import os

is_loaded = False  # A member variable of habana_frameworks module to track if our module has been imported

def __autoload():
    # This is an entrypoint for pytorch autoload mechanism
    # If the following confition is true, that means our backend has already been loaded, either explicitly
    # or by the autoload mechanism and importing it again should be skipped to avoid circular imports
    global is_loaded
    if is_loaded:
        return
    import habana_frameworks.torch

habana_frameworks/torch/init.py:

import os

# This is to prevent torch autoload mechanism from causing circular imports
import habana_frameworks

habana_frameworks.is_loaded = True

and one addition in setup.py:

entry_points={
        "torch.backends": [
            "device_backend = habana_frameworks:__autoload",
        ],
},

[shink]

@bsochack thanks so much for your quick reply!

The circular dependency issue is worth noting, I will state this in this doc.

I noticed habana_frameworks is not an open-source project, so are there any public links about the autoloading mechanism? It's OK if not.

[bsochack]

Right, habana_frameworks is not open source and there is no official release with the above change.
Btw, it requires PT 2.5 (not yet released) so it is hard to provide a real example.

[shink]

@bsochack OK it's alright

[shink]

@bsochack updated, please take a look at the HPU section

[bsochack]

lgtm

[shink]

@svekars it's ready for review, please take a look, thanks!

[svekars]

Sorry for the delay in reviewing this. I added editorial suggestion. Please make sure you have What you will learn and the Prerequisites sections - I added how they could look like in a comment. Let me know if you have any questions!

[svekars]

I'm not sure if we should link to a pull request. Maybe you can link to the file - it looks like the changes have landed already.

[shink]

This section talks about the implementation of autoloading, I think it makes sense to put a PR here.
PR can be used to let users know what has changed and to follow discussions about the implementation.

[shink]

@svekars Thanks so much for your professional and patient review! I made some changes which can be found at: https://github.com/pytorch/tutorials/pull/3037/files/4d44a78d348b918a5f6a3ff60f804b330e983546..e425fe90aeca2b734c390ddd4e0fadc1f0f715fd

[shink]

Hi @svekars, just a gentle reminder about this PR. I'm hoping your review and thanks for your time!