How to check what operators are being used by a model

[vt911001]

Hi,

Good day,
I am trying to figure out how much of the total operations are running on DML or falling back to CPU.
This might sound unwise, but I tried to compare the operations against the operators supported by DML to see if they are correlated.

Some of them looks similar but I am really not sure about it.

Below are some operations dumped from SSDMobilenetV1 model:
image_tensor ToFloat Preprocessor/map/Shape Preprocessor/map/TensorArrayUnstack/Shape MultipleGridAnchorGenerator/div_19/_188__cf__191 MultipleGridAnchorGenerator/strided_slice/_187__cf__190 MultipleGridAnchorGenerator/Meshgrid_17/ExpandedShape/concat/_186__cf__189 MultipleGridAnchorGenerator/Meshgrid_17/ExpandedShape_1/concat/_185__cf__188 MultipleGridAnchorGenerator/mul_55/_184__cf__187 MultipleGridAnchorGenerator/strided_slice_1/_183__cf__186 MultipleGridAnchorGenerator/Meshgrid_16/ExpandedShape/concat/_182__cf__185 MultipleGridAnchorGenerator/Meshgrid_16/ExpandedShape_1/concat/_181__cf__184 MultipleGridAnchorGenerator/assert_equal/All/_180__cf__183 MultipleGridAnchorGenerator/strided_slice_2/_179__cf__182 BoxPredictor_0/BoxEncodingPredictor/weights/read/_178__cf__181 BoxPredictor_0/BoxEncodingPredictor/biases/read/_177__cf__180 BoxPredictor_1/BoxEncodingPredictor/weights/read/_176__cf__179 BoxPredictor_1/BoxEncodingPredictor/biases/read/_175__cf__178 BoxPredictor_2/BoxEncodingPredictor/weights/read/_174__cf__177 BoxPredictor_2/BoxEncodingPredictor/biases/read/_173__cf__176 BoxPredictor_3/BoxEncodingPredictor/weights/read/_172__cf__175 BoxPredictor_3/BoxEncodingPredictor/biases/read/_171__cf__174 BoxPredictor_4/BoxEncodingPredictor/weights/read/_170__cf__173 BoxPredictor_4/BoxEncodingPredictor/biases/read/_169__cf__172 BoxPredictor_5/BoxEncodingPredictor/weights/read/_168__cf__171 BoxPredictor_5/BoxEncodingPredictor/biases/read/_167__cf__170 FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_0/BatchNorm/batchnorm/sub/_165__cf__168 FeatureExtractor/MobilenetV1/Conv2d_1_depthwise/depthwise_weights/read/_163__cf__166 FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_1_depthwise/BatchNorm/batchnorm/sub/_162__cf__165 FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_1_depthwise/BatchNorm/batchnorm/mul/_161__cf__164 FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/sub/_159__cf__162 FeatureExtractor/MobilenetV1/Conv2d_2_depthwise/depthwise_weights/read/_157__cf__160

Thanks,
Vincent

[jstoecker]

Hey Vincent,

The easiest way to find out where ops are placed is by using log_device_placement in a tf.ConfigProto. Some more details here. However, this option can be misleading because it's telling you the placement of ops in the model before it gets optimized: for example, ops may be fused together or eliminated entirely because they can be evaluated statically (constant folding). Regardless, this tool tells you the intended device to execute each op.

If you want to know the ops that get executed after the model's graph is optimized then you have to trace each execution. For example, you can trace a single session.run for training to collect stats. The tf.RunOptions object exposes two options that are useful here:

• trace_level can be set to FULL_TRACE to track each operator as an event.
• output_partition_graphs can be set to true to track optimized subgraphs that get evaluated by each device (e.g. CPU and DML).

You pass the tf.RunOptions to a tf.Session (or model.fit for Keras) and it will record data that is then retrieved through the tf.RunMetadata object. There's a short example of how to do this in this comment on TF repo.

Personally I think the output_partition_graphs will be more useful to you but it's not as commonly discussed. Here's a stack overflow comment with some code.

[vt911001]

Hi Justin,

Thanks for your help, I have tried your last suggestion and it works.
Really appreciate your help.

Thanks,
Vincent