-
Notifications
You must be signed in to change notification settings - Fork 312
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Summary: Pull Request resolved: #5385 Differential Revision: D62727993
- Loading branch information
1 parent
0f61e4f
commit 0c73f6b
Showing
6 changed files
with
50 additions
and
47 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
# Introduction to the ExecuTorch Developer Tools | ||
|
||
ExecuTorch has been designed with [productivity](./intro-overview.md) as one of its core objectives and the ExecuTorch Developer Tools enable this through the comprehensive suite of tools it provides users to help them profile, debug, and visualize models that they have onboarded onto ExecuTorch. | ||
|
||
All the components of the Developer Tools have been designed from the ground up with deep integration in both the export process and the runtime. This enables us to provide unique features such as linking back operator execution in the runtime to the line of code in the original eager model that this operator originated from. | ||
|
||
## Developer Tools Features | ||
|
||
The ExecuTorch Developer Tools support the following features: | ||
|
||
- **BundledProgram** is a utility tool for exporting the model bundled with a sample set of (representative) inputs and expected outputs, so that during runtime users can validate that the actual output is in fact the same as the expected output. | ||
- **Profiling** models with operator level breakdown of performance stats | ||
- Linking back operator performance stats to source code and module hierarchy | ||
- Model loading and execution time | ||
- **Delegate Integration** - Surfacing performance details from delegate backends | ||
- Link back delegate operator execution to the nodes they represent in the edge dialect graph (and subsequently linking back to source code and module hierarchy) | ||
- **Debugging** - Intermediate outputs and output quality analysis | ||
- **Visualization** - Coming soon | ||
|
||
## Fundamental components of the Developer Tools | ||
|
||
In order to fully understand and leverage the power of the Developer Tools in this section, the fundamental components that power the Developer Tools will be detailed. | ||
|
||
### ETRecord | ||
ETRecord (ExecuTorch Record) is an artifact generated during the export process that stores the graphs and other metadata that is critical for the Developer Tools to be able to link back the performance/debug data sourced from the runtime to the source code of the eager model. | ||
|
||
To draw a rough equivalence to conventional software development ETRecord can be considered as the binary built with debug symbols that is used for debugging in GNU Project debugger (gdb). | ||
|
||
More details are available in the [ETRecord documentation](sdk-etrecord.rst) on how to generate and store an ETRecord. | ||
|
||
### ETDump | ||
ETDump (ExecuTorch Dump) is the binary blob that is generated by the runtime after running a model. Similarly as above, to draw a rough equivalence to conventional software development, ETDump can be considered as the coredump of ExecuTorch, but in this case within ETDump we store all the performance and debug data that was generated by the runtime during model execution. | ||
|
||
```{note} | ||
If you only care about looking at the raw performance data without linking back to source code and other extensive features, an ETDump alone will be enough to leverage the basic features of the Developer Tools. For the full experience, it is recommended that the users also generate an ETRecord. | ||
``` | ||
|
||
More details are available in the [ETDump documentation](sdk-etdump.md) on how to generate and store an ETDump from the runtime. | ||
|
||
|
||
### Inspector APIs | ||
The Inspector Python APIs are the main user enrty point into the Developer Tools. They join the data sourced from ETDump and ETRecord to give users access to all the performance and debug data sourced from the runtime along with linkage back to eager model source code and module hierarchy in an easy to use API. | ||
|
||
More details are available in the [Inspector API documentation](sdk-inspector.rst) on how to use the Inspector APIs. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,44 +1,3 @@ | ||
# Introduction to the ExecuTorch Developer Tools | ||
|
||
ExecuTorch has been designed with [productivity](./intro-overview.md) as one of its core objectives and the ExecuTorch Developer Tools enable this through the comprehensive suite of tools it provides users to help them profile, debug, and visualize models that they have onboarded onto ExecuTorch. | ||
|
||
All the components of the Developer Tools have been designed from the ground up with deep integration in both the export process and the runtime. This enables us to provide unique features such as linking back operator execution in the runtime to the line of code in the original eager model that this operator originated from. | ||
|
||
## Developer Tools Features | ||
|
||
The ExecuTorch Developer Tools support the following features: | ||
|
||
- **BundledProgram** is a utility tool for exporting the model bundled with a sample set of (representative) inputs and expected outputs, so that during runtime users can validate that the actual output is in fact the same as the expected output. | ||
- **Profiling** models with operator level breakdown of performance stats | ||
- Linking back operator performance stats to source code and module hierarchy | ||
- Model loading and execution time | ||
- **Delegate Integration** - Surfacing performance details from delegate backends | ||
- Link back delegate operator execution to the nodes they represent in the edge dialect graph (and subsequently linking back to source code and module hierarchy) | ||
- **Debugging** - Intermediate outputs and output quality analysis | ||
- **Visualization** - Coming soon | ||
|
||
## Fundamental components of the Developer Tools | ||
|
||
In order to fully understand and leverage the power of the Developer Tools in this section, the fundamental components that power the Developer Tools will be detailed. | ||
|
||
### ETRecord | ||
ETRecord (ExecuTorch Record) is an artifact generated during the export process that stores the graphs and other metadata that is critical for the Developer Tools to be able to link back the performance/debug data sourced from the runtime to the source code of the eager model. | ||
|
||
To draw a rough equivalence to conventional software development ETRecord can be considered as the binary built with debug symbols that is used for debugging in GNU Project debugger (gdb). | ||
|
||
More details are available in the [ETRecord documentation](sdk-etrecord.rst) on how to generate and store an ETRecord. | ||
|
||
### ETDump | ||
ETDump (ExecuTorch Dump) is the binary blob that is generated by the runtime after running a model. Similarly as above, to draw a rough equivalence to conventional software development, ETDump can be considered as the coredump of ExecuTorch, but in this case within ETDump we store all the performance and debug data that was generated by the runtime during model execution. | ||
|
||
```{note} | ||
If you only care about looking at the raw performance data without linking back to source code and other extensive features, an ETDump alone will be enough to leverage the basic features of the Developer Tools. For the full experience, it is recommended that the users also generate an ETRecord. | ||
``` | ||
|
||
More details are available in the [ETDump documentation](sdk-etdump.md) on how to generate and store an ETDump from the runtime. | ||
|
||
|
||
### Inspector APIs | ||
The Inspector Python APIs are the main user enrty point into the Developer Tools. They join the data sourced from ETDump and ETRecord to give users access to all the performance and debug data sourced from the runtime along with linkage back to eager model source code and module hierarchy in an easy to use API. | ||
|
||
More details are available in the [Inspector API documentation](sdk-inspector.rst) on how to use the Inspector APIs. | ||
Please update your link to <https://pytorch.org/executorch/main/developer-tools-overview.html>. This URL will be deleted after v0.4.0. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters