You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am trying to run inference for the Anomaly Detection benchmark against the model with weights, activations, inputs, and outputs quantized. I am getting totally off results for the average AUC.
I changed nothing but the input handling before inference as the data have to be scaled down and converted to np.int8 (just like other benchmarks). Here's the code for that:
defrun_inference(model_path, data):
interpreter=tf.lite.Interpreter(model_path=model_path)
interpreter.allocate_tensors()
input_details=interpreter.get_input_details()
input_scale, zero_point=input_details[0]['quantization']
input_data=numpy.array(data/input_scale+zero_point, dtype=numpy.int8) # Just like other benchmarksoutput_details=interpreter.get_output_details()
output_data=numpy.empty_like(data)
foriinrange(input_data.shape[0]):
interpreter.set_tensor(input_details[0]['index'], input_data[i:i+1, :])
interpreter.invoke()
output_data[i:i+1, :] =interpreter.get_tensor(output_details[0]['index'])
returnoutput_data
The data parameter comes from the untouched inference code in 03_tflite_test.py and model_path is trained_models/model_ToyCar_quant_fullint_micro_intio.tflite.
The average AUC is 0.5564.
The same exact code (without re-scaling the input data type) works for the trained_models/model_ToyCar_quant_fullint_micro.tflite model.
I tried to scale the input representative dataset using the following code in the conversion script:
Hello,
I am trying to run inference for the Anomaly Detection benchmark against the model with weights, activations, inputs, and outputs quantized. I am getting totally off results for the average AUC.
I changed nothing but the input handling before inference as the data have to be scaled down and converted to np.int8 (just like other benchmarks). Here's the code for that:
The
data
parameter comes from the untouched inference code in03_tflite_test.py
andmodel_path
istrained_models/model_ToyCar_quant_fullint_micro_intio.tflite
.The average AUC is 0.5564.
The same exact code (without re-scaling the input data type) works for the
trained_models/model_ToyCar_quant_fullint_micro.tflite
model.I tried to scale the input representative dataset using the following code in the conversion script:
However, this makes the average AUC even worse: 0.4605.
Any hints would be appreciated,
Thanks
The text was updated successfully, but these errors were encountered: