Skip to content
This repository has been archived by the owner on Oct 13, 2023. It is now read-only.

Dynamically find available training backends #52

Open
MJafarMashhadi opened this issue Jan 19, 2021 · 0 comments
Open

Dynamically find available training backends #52

MJafarMashhadi opened this issue Jan 19, 2021 · 0 comments

Comments

@MJafarMashhadi
Copy link

Currently, the training backends are hard coded in hydra's code. There are a few drawbacks with this approach:

  • All the dependencies for all the training platforms must be installed even though one might not need some of them. otherwise the imports on top of cli.py will fail. (I will elaborate on this issue in a separate feature request issue)
  • if statements like this: https://github.com/georgianpartners/hydra/blob/7c733fd0eb7f9a9081ecdd2b5e3a5db2bdc282d4/hydra/cli.py#L129-L142 can turn up messy and error prone very fast. In an object oriented design this is not a good practice.
  • The user is not able to easily add a new training backend or customize the existing ones (by subclassing them or AbstractPlatform)

I propose to have a code that dynamically discovers the AbstractPlatform subclasses in runtime and registers them with hydra. It will get rid of switch-case-like if statements such as the one above and https://github.com/georgianpartners/hydra/blob/7c733fd0eb7f9a9081ecdd2b5e3a5db2bdc282d4/hydra/cli.py#L74-L77
This discovery interface should also provide the users with an appropriate way (such as a method or a decorator) for users to register and load their custom training platforms manually.

--
I went ahead of myself and started working on a PR before opening the issue. I'll submit it once it's finalized.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant