Add special device mapping

[FindHao]

To reuse the inference batch sizes already set in models' meta.yaml file for different kind of devices like AMD GPUs, this PR adds a SPECIAL_DEVICE_MAPPING and got the corresponding batch size when executes determine_batch_size ()

Run

You can use the following command to run model resnet18 inference by this userbenchmark.

python run_benchmark.py rocm-test  -t eval -m resnet18 

You can run all models without -m resnet18.

Results

The result json file is generated in the location benchmark/.userbenchmark/rocm-test/metrics-20230706XX.json. The content is like the following.

{
    "name": "rocm-test",
    "environ": {
        "pytorch_git_version": "e9ebda29d87ce0916ab08c06ab26fd3766a870e5",
        "device": "AMD Instinct MI210"
    },
    "metrics": {
        "test_eval[resnet18-cuda-eager]_latency": 23.336431
    }
}

Acknowlegement

This userbenchmark's code is mostly copied from userbenchmark/torch-nightly, some functionalities may not be available even though there are related codes.

[facebook-github-bot]

@FindHao has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[xuzhao9]

Can we have an example command and result json in the PR body? For example, the result of just running a single model.

[FindHao]

Can we have an example command and result json in the PR body? For example, the result of just running a single model.

updated.

[xuzhao9]

Thanks!

[facebook-github-bot]

@FindHao merged this pull request in 9df5215.