Inference results visual comparison #316
Conversation
Results are written to a file after the benchmark complete Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
geti_sdk/benchmarking/benchmarker.py
Outdated
| writer.writerow(result_row) | ||
|
|
||
| # Write results to file | ||
| with open(results_file, "w", newline="") as csvfile: |
There was a problem hiding this comment.
The results are written to file after each benchmarking run, to avoid data loss in case of a crash. I'd prefer to keep it like that unless there's a very good reason to change it, the overhead is not that big so in my opinion it's worth it to prevent data loss.
There was a problem hiding this comment.
All parts of the code where we can get a crash are already under the try context, so we just keep the file open for an extended period. Anyway I can revert this one if you think that's right
There was a problem hiding this comment.
Ah yeah I see what you mean. Maybe just re-open the file after each run then, to avoid keeping it open the whole time
geti_sdk/benchmarking/benchmarker.py
Outdated
| workspace_id=self.geti.workspace_id, | ||
| project=self.project, | ||
| ) | ||
| sc_image = image_client.upload_image(numpy_image) |
There was a problem hiding this comment.
The method PredictionClient.predict_image will do image upload and prediction in one line, could save a couple of lines of code here.
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
Signed-off-by: Igor Davidyuk <igor.davidyuk@intel.com>
This PR introduces the
compare_predictionsmethod to the Benchmarker class to visually compare deployments predictions on a gridTODO: