-
Notifications
You must be signed in to change notification settings - Fork 356
/
Copy pathindex.rst
210 lines (162 loc) · 5.19 KB
/
index.rst
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
.. Torch-TensorRT documentation master file, created by
sphinx-quickstart on Mon May 4 13:43:16 2020.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Torch-TensorRT
==============
In-framework compilation of PyTorch inference code for NVIDIA GPUs
--------------------------------------------------------------------------
Torch-TensorRT is a inference compiler for PyTorch, targeting NVIDIA GPUs via NVIDIA's TensorRT Deep Learning Optimizer and Runtime.
It supports both just-in-time (JIT) compilation workflows via the ``torch.compile`` interface as well as ahead-of-time (AOT) workflows.
Torch-TensorRT integrates seamlessly into the PyTorch ecosystem supporting hybrid execution of optimized TensorRT code with standard PyTorch code.
More Information / System Architecture:
* `Torch-TensorRT 2.0 <https://www.nvidia.com/en-us/on-demand/session/gtcspring23-s51714/>`_
Getting Started
----------------
* :ref:`installation`
.. toctree::
:caption: Getting Started
:maxdepth: 1
:hidden:
getting_started/installation
getting_started/getting_started_with_windows
Dynamo Frontend
----------------
* :ref:`torch_compile`
* :ref:`dynamo_export`
.. toctree::
:caption: Dynamo Frontend
:maxdepth: 1
:hidden:
dynamo/torch_compile
dynamo/dynamo_export
TorchScript Frontend
-----------------------
* :ref:`creating_a_ts_mod`
* :ref:`getting_started_with_python_api`
* :ref:`getting_started_cpp`
* :ref:`use_from_pytorch`
.. toctree::
:caption: TorchScript Frontend
:maxdepth: 1
:hidden:
ts/creating_torchscript_module_in_python
ts/getting_started_with_python_api
ts/getting_started_with_cpp_api
ts/use_from_pytorch
FX Frontend
------------
* :ref:`getting_started_with_fx`
.. toctree::
:caption: FX Frontend
:maxdepth: 1
:hidden:
fx/getting_started_with_fx_path
User Guide
------------
* :ref:`dynamic_shapes`
* :ref:`ptq`
* :ref:`saving_models`
* :ref:`runtime`
* :ref:`using_dla`
.. toctree::
:caption: User Guide
:maxdepth: 1
:hidden:
user_guide/dynamic_shapes
user_guide/ptq
user_guide/saving_models
user_guide/runtime
user_guide/using_dla
Tutorials
------------
* :ref:`torch_tensorrt_tutorials`
* :ref:`serving_torch_tensorrt_with_triton`
* :ref:`notebooks`
.. toctree::
:caption: Tutorials
:maxdepth: 3
:hidden:
tutorials/serving_torch_tensorrt_with_triton
tutorials/notebooks
tutorials/_rendered_examples/dynamo/torch_compile_resnet_example
tutorials/_rendered_examples/dynamo/torch_compile_transformers_example
tutorials/_rendered_examples/dynamo/torch_compile_advanced_usage
tutorials/_rendered_examples/dynamo/torch_compile_stable_diffusion
Python API Documenation
------------------------
* :ref:`torch_tensorrt_py`
* :ref:`torch_tensorrt_logging_py`
* :ref:`torch_tensorrt_ptq_py`
* :ref:`torch_tensorrt_dynamo_py`
* :ref:`torch_tensorrt_ts_py`
* :ref:`torch_tensorrt_fx_py`
.. toctree::
:caption: Python API Documenation
:maxdepth: 0
:hidden:
py_api/torch_tensorrt
py_api/logging
py_api/ptq
py_api/dynamo
py_api/ts
py_api/fx
C++ API Documenation
----------------------
* :ref:`namespace_torch_tensorrt`
* :ref:`namespace_torch_tensorrt__logging`
* :ref:`namespace_torch_tensorrt__ptq`
* :ref:`namespace_torch_tensorrt__torchscript`
.. toctree::
:caption: C++ API Documenation
:maxdepth: 1
:hidden:
_cpp_api/torch_tensort_cpp
_cpp_api/namespace_torch_tensorrt
_cpp_api/namespace_torch_tensorrt__logging
_cpp_api/namespace_torch_tensorrt__torchscript
_cpp_api/namespace_torch_tensorrt__ptq
CLI Documentation
---------------------
* :ref:`torchtrtc`
.. toctree::
:caption: CLI Documenation
:maxdepth: 0
:hidden:
cli/torchtrtc
Contributor Documentation
--------------------------------
* :ref:`system_overview`
* :ref:`dynamo_converters`
* :ref:`writing_dynamo_aten_lowering_passes`
* :ref:`ts_converters`
* :ref:`useful_links`
.. toctree::
:caption: Contributor Documentation
:maxdepth: 1
:hidden:
contributors/system_overview
contributors/dynamo_converters
contributors/writing_dynamo_aten_lowering_passes
contributors/ts_converters
contributors/useful_links
Indices
----------------
* :ref:`supported_ops`
* :ref:`genindex`
* :ref:`search`
.. toctree::
:caption: Indices
:maxdepth: 1
:hidden:
indices/supported_ops
Legacy Further Information (TorchScript)
-------------------------------------------
* `Introductory Blog Post <https://developer.nvidia.com/blog/accelerating-inference-up-to-6x-faster-in-pytorch-with-torch-tensorrt/>`_
* `GTC 2020 Talk <https://www.nvidia.com/en-us/on-demand/session/gtcsj20-s21671/>`_
* `GTC 2020 Fall Talk <https://www.nvidia.com/en-us/on-demand/session/gtcfall20-a21864/>`_
* `GTC 2021 Talk <https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31864/>`_
* `GTC 2021 Fall Talk <https://www.nvidia.com/en-us/on-demand/session/gtcfall21-a31107/>`_
* `PyTorch Ecosystem Day 2021 <https://assets.pytorch.org/pted2021/posters/I6.png>`_
* `PyTorch Developer Conference 2021 <https://s3.amazonaws.com/assets.pytorch.org/ptdd2021/posters/D2.png>`_
* `PyTorch Developer Conference 2022 <https://pytorch.s3.amazonaws.com/posters/ptc2022/C04.pdf>`_