Skip to content

Commit

Permalink
docs: [Automated] Regenerating documenation from
Browse files Browse the repository at this point in the history
Signed-off-by: TRTorch Github Bot <trtorch.github.bot@nvidia.com>
  • Loading branch information
TRTorch Github Bot committed Mar 17, 2021
1 parent dc4d966 commit 5b6bd4c
Show file tree
Hide file tree
Showing 8 changed files with 222 additions and 21 deletions.
2 changes: 1 addition & 1 deletion docs/_notebooks/Resnet50-example.html
Original file line number Diff line number Diff line change
Expand Up @@ -690,7 +690,7 @@
</div>
</div>
<p>
<img alt="832f033f9337487ba7868b45ff52e9fe" src="http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png"/>
<img alt="0904ed52551d45acab1d028795d47df1" src="http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png"/>
</p>
<h1 id="notebooks-resnet50-example--page-root">
TRTorch Getting Started - ResNet 50
Expand Down
2 changes: 1 addition & 1 deletion docs/_notebooks/lenet-getting-started.html
Original file line number Diff line number Diff line change
Expand Up @@ -784,7 +784,7 @@
</div>
</div>
<p>
<img alt="a469a5f1dc3245c091eb0097581de724" src="http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png"/>
<img alt="eb83af0673044a89b9357bb998acd927" src="http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png"/>
</p>
<h1 id="notebooks-lenet-getting-started--page-root">
TRTorch Getting Started - LeNet
Expand Down
2 changes: 1 addition & 1 deletion docs/_notebooks/ssd-object-detection-demo.html
Original file line number Diff line number Diff line change
Expand Up @@ -804,7 +804,7 @@
</div>
</div>
<p>
<img alt="6fee1a4f09854ba1a11aed1c38759f74" src="http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png"/>
<img alt="092b8936ef17484f92965a48c4cdea98" src="http://developer.download.nvidia.com/compute/machine-learning/frameworks/nvidia_logo.png"/>
</p>
<h1 id="notebooks-ssd-object-detection-demo--page-root">
Object Detection with TRTorch (SSD)
Expand Down
75 changes: 70 additions & 5 deletions docs/_sources/tutorials/ptq.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,17 @@ the TensorRT calibrator. With TRTorch we look to leverage existing infrastructur
calibrators easier.

LibTorch provides a ``DataLoader`` and ``Dataset`` API which steamlines preprocessing and batching input data.
This section of the PyTorch documentation has more information https://pytorch.org/tutorials/advanced/cpp_frontend.html#loading-data.
These APIs are exposed via both C++ and Python interface which makes it easier for the end user.
For C++ interface, we use ``torch::Dataset`` and ``torch::data::make_data_loader`` objects to construct and perform pre-processing on datasets.
The equivalent functionality in python interface uses ``torch.utils.data.Dataset`` and ``torch.utils.data.DataLoader``.
This section of the PyTorch documentation has more information https://pytorch.org/tutorials/advanced/cpp_frontend.html#loading-data and https://pytorch.org/tutorials/recipes/recipes/loading_data_recipe.html.
TRTorch uses Dataloaders as the base of a generic calibrator implementation. So you will be able to reuse or quickly
implement a ``torch::Dataset`` for your target domain, place it in a DataLoader and create a INT8 Calibrator
which you can provide to TRTorch to run INT8 Calibration during compliation of your module.

.. _writing_ptq:
.. _writing_ptq_cpp:

How to create your own PTQ application
How to create your own PTQ application in C++
----------------------------------------

Here is an example interface of a ``torch::Dataset`` class for CIFAR10:
Expand Down Expand Up @@ -132,14 +135,76 @@ Then all thats required to setup the module for INT8 calibration is to set the f
auto trt_mod = trtorch::CompileGraph(mod, compile_spec);

If you have an existing Calibrator implementation for TensorRT you may directly set the ``ptq_calibrator`` field with a pointer to your calibrator and it will work as well.

From here not much changes in terms of how to execution works. You are still able to fully use LibTorch as the sole interface for inference. Data should remain
in FP32 precision when it's passed into `trt_mod.forward`. There exists an example application in the TRTorch demo that takes you from training a VGG16 network on
CIFAR10 to deploying in INT8 with TRTorch here: https://github.com/NVIDIA/TRTorch/tree/master/cpp/ptq

.. _writing_ptq_python:

How to create your own PTQ application in Python
----------------------------------------

TRTorch Python API provides an easy and convenient way to use pytorch dataloaders with TensorRT calibrators. ``DataLoaderCalibrator`` class can be used to create
a TensorRT calibrator by providing desired configuration. The following code demonstrates an example on how to use it

.. code-block:: python
testing_dataset = torchvision.datasets.CIFAR10(root='./data',
train=False,
download=True,
transform=transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.4914, 0.4822, 0.4465),
(0.2023, 0.1994, 0.2010))
]))
testing_dataloader = torch.utils.data.DataLoader(testing_dataset,
batch_size=1,
shuffle=False,
num_workers=1)
calibrator = trtorch.ptq.DataLoaderCalibrator(testing_dataloader,
cache_file='./calibration.cache',
use_cache=False,
algo_type=trtorch.ptq.CalibrationAlgo.ENTROPY_CALIBRATION_2,
device=torch.device('cuda:0'))
compile_spec = {
"input_shapes": [[1, 3, 32, 32]],
"op_precision": torch.int8,
"calibrator": calibrator,
"device": {
"device_type": trtorch.DeviceType.GPU,
"gpu_id": 0,
"dla_core": 0,
"allow_gpu_fallback": False,
"disable_tf32": False
}
}
trt_mod = trtorch.compile(model, compile_spec)
In the cases where there is a pre-existing calibration cache file that users want to use, ``CacheCalibrator`` can be used without any dataloaders. The following example demonstrates how
to use ``CacheCalibrator`` to use in INT8 mode.

.. code-block:: python
calibrator = trtorch.ptq.CacheCalibrator("./calibration.cache")
compile_settings = {
"input_shapes": [[1, 3, 32, 32]],
"op_precision": torch.int8,
"calibrator": calibrator,
"max_batch_size": 32,
}
trt_mod = trtorch.compile(model, compile_settings)
If you already have an existing calibrator class (implemented directly using TensorRT API), you can directly set the calibrator field to your class which can be very convenient.
For a demo on how PTQ can be performed on a VGG network using TRTorch API, you can refer to https://github.com/NVIDIA/TRTorch/blob/master/tests/py/test_ptq_dataloader_calibrator.py
and https://github.com/NVIDIA/TRTorch/blob/master/tests/py/test_ptq_trt_calibrator.py

Citations
^^^^^^^^^^^

Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images.

Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
Binary file modified docs/objects.inv
Binary file not shown.
2 changes: 1 addition & 1 deletion docs/py_api/trtorch.html
Original file line number Diff line number Diff line change
Expand Up @@ -1011,7 +1011,7 @@ <h2 id="functions">
<span class="sig-paren">
)
</span>
→ &lt;torch._C.ScriptClass object at 0x7fb41dd1c618&gt;
→ &lt;torch._C.ScriptClass object at 0x7fb23f032618&gt;
<a class="headerlink" href="#trtorch.TensorRTCompileSpec" title="Permalink to this definition">
</a>
Expand Down
2 changes: 1 addition & 1 deletion docs/searchindex.js

Large diffs are not rendered by default.

Loading

0 comments on commit 5b6bd4c

Please sign in to comment.