Oss doc

[blefaudeux]

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
• Did you read the contributor guideline?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

Moving #63 to the right direction, hopefully.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[blefaudeux]

note: I resurrected an old branch for that, and did not realize that the old commits would show up here

[msbaines]

Docs stuff looks OK. Can you rebase or merge with master to avoid extraneous diffs due to changes to benchmarks/oss.py

[blefaudeux]

@msbaines the benchmark changes were on purpose actually, fixing the broken all_reduce/loss.backward() + minor readability. I hope that works for you, landed to get some of the API documentation up ?