Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[doc][devtools][rename] New URL for developer tools overview page #5385

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@ Techniques for performing computations and memory accesses on tensors with lower

The ExecuTorch runtime executes models on edge devices. It is responsible for program initialization, program execution and, optionally, destruction (releasing backend owned resources).

## [Developer Tools](./sdk-overview.md)
## [Developer Tools](./devtools-overview.md)

A collection of tools users need to profile, debug and visualize programs that are running with ExecuTorch.

Expand Down
44 changes: 44 additions & 0 deletions docs/source/devtools-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Introduction to the ExecuTorch Developer Tools

ExecuTorch has been designed with [productivity](./intro-overview.md) as one of its core objectives and the ExecuTorch Developer Tools enable this through the comprehensive suite of tools it provides users to help them profile, debug, and visualize models that they have onboarded onto ExecuTorch.

All the components of the Developer Tools have been designed from the ground up with deep integration in both the export process and the runtime. This enables us to provide unique features such as linking back operator execution in the runtime to the line of code in the original eager model that this operator originated from.

## Developer Tools Features

The ExecuTorch Developer Tools support the following features:

- **BundledProgram** is a utility tool for exporting the model bundled with a sample set of (representative) inputs and expected outputs, so that during runtime users can validate that the actual output is in fact the same as the expected output.
- **Profiling** models with operator level breakdown of performance stats
- Linking back operator performance stats to source code and module hierarchy
- Model loading and execution time
- **Delegate Integration** - Surfacing performance details from delegate backends
- Link back delegate operator execution to the nodes they represent in the edge dialect graph (and subsequently linking back to source code and module hierarchy)
- **Debugging** - Intermediate outputs and output quality analysis
- **Visualization** - Coming soon

## Fundamental components of the Developer Tools

In order to fully understand and leverage the power of the Developer Tools in this section, the fundamental components that power the Developer Tools will be detailed.

### ETRecord
ETRecord (ExecuTorch Record) is an artifact generated during the export process that stores the graphs and other metadata that is critical for the Developer Tools to be able to link back the performance/debug data sourced from the runtime to the source code of the eager model.

To draw a rough equivalence to conventional software development ETRecord can be considered as the binary built with debug symbols that is used for debugging in GNU Project debugger (gdb).

More details are available in the [ETRecord documentation](sdk-etrecord.rst) on how to generate and store an ETRecord.

### ETDump
ETDump (ExecuTorch Dump) is the binary blob that is generated by the runtime after running a model. Similarly as above, to draw a rough equivalence to conventional software development, ETDump can be considered as the coredump of ExecuTorch, but in this case within ETDump we store all the performance and debug data that was generated by the runtime during model execution.

```{note}
If you only care about looking at the raw performance data without linking back to source code and other extensive features, an ETDump alone will be enough to leverage the basic features of the Developer Tools. For the full experience, it is recommended that the users also generate an ETRecord.
```

More details are available in the [ETDump documentation](sdk-etdump.md) on how to generate and store an ETDump from the runtime.


### Inspector APIs
The Inspector Python APIs are the main user enrty point into the Developer Tools. They join the data sourced from ETDump and ETRecord to give users access to all the performance and debug data sourced from the runtime along with linkage back to eager model source code and module hierarchy in an easy to use API.

More details are available in the [Inspector API documentation](sdk-inspector.rst) on how to use the Inspector APIs.
3 changes: 3 additions & 0 deletions docs/source/devtools-tutorial.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
## Developer Tools Usage Tutorial

Please refer to the [Developer Tools tutorial](./tutorials/devtools-integration-tutorial) for a walkthrough on how to profile a model in ExecuTorch using the Developer Tools.
2 changes: 1 addition & 1 deletion docs/source/getting-started-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,6 @@ _Executor_ is the entry point to load the program and execute it. The execution

## Developer Tools

It should be efficient for users to go from research to production using the flow above. Productivity is essentially important, for users to author, optimize and deploy their models. We provide [ExecuTorch Developer Tools](./sdk-overview.md) to improve productivity. The Developer Tools are not in the diagram. Instead it's a tool set that covers the developer workflow in all three phases.
It should be efficient for users to go from research to production using the flow above. Productivity is essentially important, for users to author, optimize and deploy their models. We provide [ExecuTorch Developer Tools](./devtools-overview.md) to improve productivity. The Developer Tools are not in the diagram. Instead it's a tool set that covers the developer workflow in all three phases.

During the program preparation and execution, users can use the ExecuTorch Developer Tools to profile, debug, or visualize the program. Since the end-to-end flow is within the PyTorch ecosystem, users can correlate and display performance data along with graph visualization as well as direct references to the program source code and model hierarchy. We consider this to be a critical component for quickly iterating and lowering PyTorch programs to edge devices and environments.
8 changes: 4 additions & 4 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ Topics in this section will help you get started with ExecuTorch.
tutorials/export-to-executorch-tutorial
running-a-model-cpp-tutorial
extension-module
tutorials/sdk-integration-tutorial
tutorials/devtools-integration-tutorial
apple-runtime
demo-apps-ios
demo-apps-android
Expand Down Expand Up @@ -196,15 +196,15 @@ Topics in this section will help you get started with ExecuTorch.
:caption: Developer Tools
:hidden:

sdk-overview
devtools-overview
sdk-bundled-io
sdk-etrecord
sdk-etdump
sdk-profiling
sdk-debugging
sdk-inspector
sdk-delegate-integration
sdk-tutorial
devtools-tutorial

.. toctree::
:glob:
Expand Down Expand Up @@ -247,7 +247,7 @@ ExecuTorch tutorials.
:header: Using the ExecuTorch Developer Tools to Profile a Model
:card_description: A tutorial for using the ExecuTorch Developer Tools to profile and analyze a model with linkage back to source code.
:image: _static/img/generic-pytorch-logo.png
:link: tutorials/sdk-integration-tutorial.html
:link: tutorials/devtools-integration-tutorial.html
:tags: devtools

.. customcarditem::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ Since weight packing creates an extra copy of the weights inside XNNPACK, We fre
When executing the XNNPACK subgraphs, we prepare the tensor inputs and outputs and feed them to the XNNPACK runtime graph. After executing the runtime graph, the output pointers are filled with the computed tensors.

#### **Profiling**
We have enabled basic profiling for XNNPACK delegate that can be enabled with the following compiler flag `-DENABLE_XNNPACK_PROFILING`. With ExecuTorch's Developer Tools integration, you can also now use the Developer Tools to profile the model. You can follow the steps in [Using the ExecuTorch Developer Tools to Profile a Model](./tutorials/sdk-integration-tutorial) on how to profile ExecuTorch models and use Developer Tools' Inspector API to view XNNPACK's internal profiling information.
We have enabled basic profiling for XNNPACK delegate that can be enabled with the following compiler flag `-DENABLE_XNNPACK_PROFILING`. With ExecuTorch's Developer Tools integration, you can also now use the Developer Tools to profile the model. You can follow the steps in [Using the ExecuTorch Developer Tools to Profile a Model](./tutorials/devtools-integration-tutorial) on how to profile ExecuTorch models and use Developer Tools' Inspector API to view XNNPACK's internal profiling information.


[comment]: <> (TODO: Refactor quantizer to a more official quantization doc)
Expand Down
2 changes: 1 addition & 1 deletion docs/source/sdk-inspector.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ APIs:
* By accessing the `public attributes <#inspector-attributes>`__ of the ``Inspector``, ``EventBlock``, and ``Event`` classes.
* By using a `CLI <#cli>`__ tool for basic functionalities.

Please refer to the `e2e use case doc <tutorials/sdk-integration-tutorial.html>`__ get an understanding of how to use these in a real world example.
Please refer to the `e2e use case doc <tutorials/devtools-integration-tutorial.html>`__ get an understanding of how to use these in a real world example.


Inspector Methods
Expand Down
43 changes: 1 addition & 42 deletions docs/source/sdk-overview.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,3 @@
# Introduction to the ExecuTorch Developer Tools

ExecuTorch has been designed with [productivity](./intro-overview.md) as one of its core objectives and the ExecuTorch Developer Tools enable this through the comprehensive suite of tools it provides users to help them profile, debug, and visualize models that they have onboarded onto ExecuTorch.

All the components of the Developer Tools have been designed from the ground up with deep integration in both the export process and the runtime. This enables us to provide unique features such as linking back operator execution in the runtime to the line of code in the original eager model that this operator originated from.

## Developer Tools Features

The ExecuTorch Developer Tools support the following features:

- **BundledProgram** is a utility tool for exporting the model bundled with a sample set of (representative) inputs and expected outputs, so that during runtime users can validate that the actual output is in fact the same as the expected output.
- **Profiling** models with operator level breakdown of performance stats
- Linking back operator performance stats to source code and module hierarchy
- Model loading and execution time
- **Delegate Integration** - Surfacing performance details from delegate backends
- Link back delegate operator execution to the nodes they represent in the edge dialect graph (and subsequently linking back to source code and module hierarchy)
- **Debugging** - Intermediate outputs and output quality analysis
- **Visualization** - Coming soon

## Fundamental components of the Developer Tools

In order to fully understand and leverage the power of the Developer Tools in this section, the fundamental components that power the Developer Tools will be detailed.

### ETRecord
ETRecord (ExecuTorch Record) is an artifact generated during the export process that stores the graphs and other metadata that is critical for the Developer Tools to be able to link back the performance/debug data sourced from the runtime to the source code of the eager model.

To draw a rough equivalence to conventional software development ETRecord can be considered as the binary built with debug symbols that is used for debugging in GNU Project debugger (gdb).

More details are available in the [ETRecord documentation](sdk-etrecord.rst) on how to generate and store an ETRecord.

### ETDump
ETDump (ExecuTorch Dump) is the binary blob that is generated by the runtime after running a model. Similarly as above, to draw a rough equivalence to conventional software development, ETDump can be considered as the coredump of ExecuTorch, but in this case within ETDump we store all the performance and debug data that was generated by the runtime during model execution.

```{note}
If you only care about looking at the raw performance data without linking back to source code and other extensive features, an ETDump alone will be enough to leverage the basic features of the Developer Tools. For the full experience, it is recommended that the users also generate an ETRecord.
```

More details are available in the [ETDump documentation](sdk-etdump.md) on how to generate and store an ETDump from the runtime.


### Inspector APIs
The Inspector Python APIs are the main user enrty point into the Developer Tools. They join the data sourced from ETDump and ETRecord to give users access to all the performance and debug data sourced from the runtime along with linkage back to eager model source code and module hierarchy in an easy to use API.

More details are available in the [Inspector API documentation](sdk-inspector.rst) on how to use the Inspector APIs.
Please update your link to <https://pytorch.org/executorch/main/devtools-overview.html>. This URL will be deleted after v0.4.0.
2 changes: 1 addition & 1 deletion docs/source/sdk-profiling.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ We provide access to all the profiling data via the Python [Inspector API](./sdk
- Through the Inspector API, users can do a wide range of analysis varying from printing out performance details to doing more finer granular calculation on module level.


Please refer to the [Developer Tools tutorial](./tutorials/sdk-integration-tutorial.rst) for a step-by-step walkthrough of the above process on a sample model.
Please refer to the [Developer Tools tutorial](./tutorials/devtools-integration-tutorial.rst) for a step-by-step walkthrough of the above process on a sample model.
2 changes: 1 addition & 1 deletion docs/source/sdk-tutorial.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
## Developer Tools Usage Tutorial

Please refer to the [Developer Tools tutorial](./tutorials/sdk-integration-tutorial) for a walkthrough on how to profile a model in ExecuTorch using the Developer Tools.
Please update your link to <https://pytorch.org/executorch/main/devtools-tutorial.html>. This URL will be deleted after v0.4.0.
Loading
Loading