Difficulty Extracting Detected String Values from Predicted Text Probabilities

[ronbitonn]

Description:

I am currently working with the Detectron2 library and trying to extract specific information from the model's predictions. I have extracted detection scores, prediction boxes, and prediction classes successfully, but I am having trouble extracting the detected string values from the predicted text probabilities. Below is the relevant portion of my code:

# Extract information from preds
detection_scores = preds.get("scores")
pred_boxes = preds.get("pred_boxes")
pred_classes = preds.get("pred_classes")
text_probs = preds.get("pred_text_prob")

# Printing the extracted information
print("\nDetection Scores:")
print(detection_scores)

print("\nPrediction Boxes:")
print(pred_boxes)

print("\nPrediction Classes:")
print(pred_classes)

print("\nText Probabilities:")
print(text_probs)

# Assuming the first prediction is the one you're interested in
detected_number = pred_classes[0].item() if len(pred_classes) > 0 else None
detection_score = detection_scores[0].item() if len(detection_scores) > 0 else None

# Attempting to get the highest text probability for the first predicted word
text_score = text_probs[0].max().item() if len(text_probs) > 0 else None

# Print the saved values
print("\nDetected Number:", detected_number)
print("Detection Score:", detection_score)
print("Text Score:", text_score)

While this code executes without errors, it does not produce the desired result. Specifically, I am trying to obtain the detected string value (e.g., "18" as shown in the attached image) from the predicted text probabilities.

Request:

Could you guide how to correctly extract the detected string values from the predicted text probabilities? Any advice or code snippets would be greatly appreciated.

Additional Information:

Below are the extracted values from my model's predictions:

Detection Scores:
tensor([0.9918, 0.2736], device='cuda:0')

Prediction Boxes:
RotatedBoxes(tensor([[ 594.0432, 1129.9354,  254.9043,  253.3743,   12.0006],
        [1271.8351, 1030.9031,   12.8350,   15.3923,   91.0112]],
       device='cuda:0'))

Prediction Classes:
tensor([0, 0], device='cuda:0')

Text Probabilities:
tensor([[[2.4674e-08, 9.4696e-08, 2.8828e-05,  ..., 1.8614e-07,
          1.7122e-07, 1.4470e-07],
         [5.7717e-08, 2.4107e-04, 1.0346e-03,  ..., 1.1237e-07,
          5.0156e-08, 6.6810e-07],
         [6.8935e-08, 9.9975e-01, 2.3456e-05,  ..., 1.1423e-07,
          4.3881e-07, 5.7719e-07],
         ...,
         [2.7530e-06, 6.1837e-02, 2.1865e-02,  ..., 7.1566e-06,
          6.2267e-06, 5.9245e-05],
         [3.7513e-06, 1.2308e-01, 1.7794e-02,  ..., 1.1299e-05,
          1.5661e-05, 6.3211e-05],
         [8.9842e-07, 5.7951e-03, 8.6867e-03,  ..., 1.8185e-06,
          2.2052e-06, 1.6607e-05]],

        [[3.4363e-04, 5.7468e-04, 1.8616e-03,  ..., 3.7598e-03,
          6.6891e-03, 4.0677e-04],
         [5.2373e-06, 9.8509e-01, 1.2281e-03,  ..., 2.9823e-05,
          6.0815e-05, 2.4332e-05],
         [1.2496e-05, 9.8383e-01, 3.8379e-04,  ..., 5.4723e-05,
          8.0950e-05, 6.0791e-05],
         ...,
         [1.2839e-04, 8.4761e-01, 1.8634e-03,  ..., 6.7326e-04,
          7.7782e-04, 9.0737e-04],
         [1.2892e-04, 8.4728e-01, 1.8580e-03,  ..., 6.7630e-04,
          7.8144e-04, 9.0697e-04],
         [1.2938e-04, 8.4700e-01, 1.8539e-03,  ..., 6.7892e-04,
          7.8478e-04, 9.0656e-04]]], device='cuda:0')

I appreciate your help!