Skip to content

Commit

Permalink
chore: format changes in documentation
Browse files Browse the repository at this point in the history
Signed-off-by: Bo Wang <wangbo1995ee@163.com>
  • Loading branch information
bowang007 committed May 5, 2021
1 parent b6bf716 commit 78e67cc
Showing 1 changed file with 12 additions and 12 deletions.
24 changes: 12 additions & 12 deletions core/partitioning/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,22 @@ TRTorch partitioning phase is developed to support automatic fallback feature in
default until the automatic fallback feature is enabled.

On a high level, TRTorch partitioning phase does the following:
- Segmentation. Go through the set of operators in order and verify if there is converter for each operator. Then,
- `Segmentation`. Go through the set of operators in order and verify if there is converter for each operator. Then,
roughly separate the graph into parts that TRTorch can support and parts TRTorch cannot.
- Dependency Analysis. For every to be compiled operator there is a "complete dependency graph", which means that
- `Dependency Analysis`. For every to be compiled operator there is a "complete dependency graph", which means that
every input can to traced back to an input as Tensor or TensorList. Go through all segments after segmentation then
do dependency analysis to ensure that there are only Tensor/TensorList inputs and outputs for TensorRT segments.
- Shape Analysis. For each segments, figure out the input and outputs shapes starting from the provided input shape
- `Shape Analysis`. For each segments, figure out the input and outputs shapes starting from the provided input shape
from the user. Shapes can be calculated by running the graphs with JIT.
- Conversion. Every TensorRT segments will be converted to TensorRT engine. This part is done in compiler.cpp, but
- `Conversion`. Every TensorRT segments will be converted to TensorRT engine. This part is done in compiler.cpp, but
it's still a phase in our partitioning process.
- Stitching. Stitch all TensorRT engines with PyTorch nodes altogether.
- `Stitching`. Stitch all TensorRT engines with PyTorch nodes altogether.

Here is the brief description of functionalities of each file:
- PartitionInfo.h/cpp: The automatic fallback APIs that is used for partitioning.
- SegmentedBlock.h/cpp: The main data structures that is used to maintain information for each segments after segmentation.
- shape_analysis.h/cpp: Code implementation to get the shapes for each segments by running them in JIT.
- partitioning.h/cpp: APIs and main code implementation for partitioning phase.
- `PartitionInfo.h/cpp`: The automatic fallback APIs that is used for partitioning.
- `SegmentedBlock.h/cpp`: The main data structures that is used to maintain information for each segments after segmentation.
- `shape_analysis.h/cpp`: Code implementation to get the shapes for each segments by running them in JIT.
- `partitioning.h/cpp`: APIs and main code implementation for partitioning phase.

### Automatic Fallback
To enable automatic fallback feature, you can set following attributes in Python:
Expand All @@ -39,10 +39,10 @@ To enable automatic fallback feature, you can set following attributes in Python
}
})
```
- enabled: By default automatic fallback will be off. It is enabled by setting it to True.
- min_block_size: The minimum number of consecutive operations that must satisfy to be converted to TensorRT. For
- `enabled`: By default automatic fallback will be off. It is enabled by setting it to True.
- `min_block_size`: The minimum number of consecutive operations that must satisfy to be converted to TensorRT. For
example, if it's set to 3, then there must be 3 consecutive supported operators then this segments will be converted.
- forced_fallback_ops: A list of strings that will be the names of operations that the user explicitly want to be in
- `forced_fallback_ops`: A list of strings that will be the names of operations that the user explicitly want to be in
PyTorch nodes.

To enable automatic fallback feature in C++, following APIs could be uses:
Expand Down

0 comments on commit 78e67cc

Please sign in to comment.