Ade20k benchmark

[A-Jacobson]

ade20k benchmark in the style of the resnet benchmark. definitely not generated by gpt.

Notes:

• Added download_ade20k.py - considered adding a download=True flag to the main script but it felt complicated to include along with the streaming object. Thought this may be a more simple compromise but am open to suggestions.
• mcloud run config still points to my fork. will need to be changed once this is merged.
• light on tests.. probably?

Additionally:

Did some renaming and refactoring to keep this in line with the resnet blog. Examples include:

1. local dataset splits arg from training, validation, test - > train, val, test
2. build_ade20k_dataloader -> build_ade20k_dataspec - this is was the resnet example is doing and these functions are actually retuning dataspecs
3. composer_deeplab -> build_composer_deeplab - now takes pytorch initialization functions rather than composer Initializer objects

[Landanjs]

This was a lot to review haha, great job getting this up!

Do you wandb runs for this?

Requested big changes:

1. Combine build_ade20k_dataspec and build_streaming_ade20k_dataspec, then add a is_streaming argument. This isn't perfect, but it does reduce the data.py and main.py code a bit.
2. Should the synthetic ADE20k dataset be removed?
3. I think we should remove the option for v1 streaming since this will be deprecated in the future.
4. Can we remove the plus argument for the model and associated code? Reduces the code a bit and I don't have a good reason to keep the code around anymore.
5. I think a smoke test (e.g. test_trainer in the ResNet benchmark) is a main component for benchmarks. Would it be possible to add this? @dblalock for clarification on its importance

Nits:

• Add newline between docstring TL;DR and Args:, between Args: and Return: (mostly in data.py I think?).
• Double check that there is 1 newline at the end of each file

[dblalock]

This is awesome. I'm so excited to finally train a segmentation model.

But anyway, LGTM other than some local nitpicking. We'll also have to dedup the MCLI vs standalone configs, but that's a separate PR.

[dblalock]

Do we need sync batchnorm? Just looking for ways to simplify what we give to users + customers.

[A-Jacobson]

it seems that we aren't using it but that's very much batch size dependent. It's likely going to be useful if someone has a smaller gpu or large enough image size to force their per device batch size down to ~4 or less.