Rollback (delete task) failing w/ "KeyError: 'field_track'"

[dmer]

          Hey @mjordan I'm running into an error that seems related to track files - I can put this in another (or new) issue if you prefer.

I'm just running a very basic rollback - the rollback.yml:

task: delete
host: https://site.edu
username: admin
password: pwd
input_dir: /mnt/ingest/
standalone_media_url: true
input_csv: rollback.csv

and the rollback csv has a list of about 30 node id's

This is a test set that I'm deleting so I can re-ingest w/ some changes - the only thing I did w/ these objects in Islandora was manually add a thumbnail to one object.

when I run the job I get this output:

"Delete" task started using config file rollback.yml.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /mnt/ingest/islandora_workbench/./workbench:2172 in <module>                                     │
│                                                                                                  │
│   2169 │   if config['task'] == 'update':                                                        │
│   2170 │   │   update()                                                                          │
│   2171 │   if config['task'] == 'delete':                                                        │
│ ❱ 2172 │   │   delete()                                                                          │
│   2173 │   if config['task'] == 'add_media':                                                     │
│   2174 │   │   add_media()                                                                       │
│   2175 │   if config['task'] == 'delete_media':                                                  │
│                                                                                                  │
│ /mnt/ingest/islandora_workbench/./workbench:621 in delete                                        │
│                                                                                                  │
│    618 │   │   │   for media in media_response_body:                                             │
│    619 │   │   │   │   if 'mid' in media:                                                        │
│    620 │   │   │   │   │   media_id = media['mid'][0]['value']                                   │
│ ❱  621 │   │   │   │   │   media_delete_status_code = remove_media_and_file(config, media_id)    │
│    622 │   │   │   │   │   if media_delete_status_code == 204:                                   │
│    623 │   │   │   │   │   │   media_messages.append("+ Media " + config['host'] + '/media/' +   │
│    624                                                                                           │
│                                                                                                  │
│ /mnt/ingest/islandora_workbench/workbench_utils.py:3601 in remove_media_and_file                 │
│                                                                                                  │
│   3598 │   media_bundle_name = get_media_response_body['bundle'][0]['target_id']                 │
│   3599 │   if media_bundle_name in config['media_track_file_fields']:                            │
│   3600 │   │   track_file_field = config['media_track_file_fields'][media_bundle_name]           │
│ ❱ 3601 │   │   for track_file in get_media_response_body[track_file_field]:                      │
│   3602 │   │   │   track_file_id = track_file['target_id']                                       │
│   3603 │   │   │   track_file_endpoint = config['host'] + '/entity/file/' + str(track_file_id)   │
│   3604 │   │   │   track_file_response = issue_request(config, 'DELETE', track_file_endpoint)    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'field_track'

Edit: noting that WB was updated fairly recently - when I do git log the most recent commit is from Aug. 3rd

Originally posted by @dmer in #373 (comment)

[mjordan]

@dmer do any of the media that would be deleted in this rollback have media track files?

[dmer]

@mjordan no, these were all files pulled via https from an S3 bucket. I did manually add one thumbnail file to one of the records prior to attempting to run the rollback, but that's the only variation I can think of.

[mjordan]

After ingesting some nodes and accompanying media, I can't replicate this error. Rollback works fine. I'd like to see if anyone else can replicate it. Regardless, any time Python code throws a KeyError, good code should check to ensure that key is present. I will add a defensive if check here to avoid this from happening in the future.

[mjordan]

After some further testing (including deleting a node+media without a track, and a node+media+track), I've pushed an update to main that should prevent the KeyError from happening. When you get a chance, can you try deleting a node + media to see if the exception still occurs?

[dmer]

I just pulled the latest Main and updated Python stuff and re-ran this rollback that failed and I got the same exception error. It happens when the rollback encounters a Video object - this object had an mp4 video file as Original File media, but there were no track files attached at all. When I deleted the media from that node manually and re-ran the job it picked up where it left off and continued deleting records and media. Pasting the error msg below in case that's helpful

 Traceback (most recent call last) ────────────────────────────────╮
│ /mnt/ingest/islandora_workbench/./workbench:2192 in <module>                                     │
│                                                                                                  │
│   2189 │   if config['task'] == 'update':                                                        │
│   2190 │   │   update()                                                                          │
│   2191 │   if config['task'] == 'delete':                                                        │
│ ❱ 2192 │   │   delete()                                                                          │
│   2193 │   if config['task'] == 'add_media':                                                     │
│   2194 │   │   add_media()                                                                       │
│   2195 │   if config['task'] == 'delete_media':                                                  │
│                                                                                                  │
│ /mnt/ingest/islandora_workbench/./workbench:611 in delete                                        │
│                                                                                                  │
│    608 │   │   │   for media in media_response_body:                                             │
│    609 │   │   │   │   if 'mid' in media:                                                        │
│    610 │   │   │   │   │   media_id = media['mid'][0]['value']                                   │
│ ❱  611 │   │   │   │   │   media_delete_status_code = remove_media_and_file(config, media_id)    │
│    612 │   │   │   │   │   if media_delete_status_code == 204:                                   │
│    613 │   │   │   │   │   │   media_messages.append("+ Media " + config['host'] + '/media/' +   │
│    614                                                                                           │
│                                                                                                  │
│ /mnt/ingest/islandora_workbench/workbench_utils.py:3830 in remove_media_and_file                 │
│                                                                                                  │
│   3827 │   media_bundle_name = get_media_response_body['bundle'][0]['target_id']                 │
│   3828 │   if media_bundle_name in config['media_track_file_fields']:                            │
│   3829 │   │   track_file_field = config['media_track_file_fields'][media_bundle_name]           │
│ ❱ 3830 │   │   if 'field_track' in get_media_response_body[track_file_field]:                    │
│   3831 │   │   │   for track_file in get_media_response_body[track_file_field]:                  │
│   3832 │   │   │   │   track_file_id = track_file['target_id']                                   │
│   3833 │   │   │   │   track_file_endpoint = config['host'] + '/entity/file/' + str(track_file_  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'field_track'

[mjordan]

Of course I tested only using an audio media. I'll restest with a video and let you know what I found.

[mjordan]

Same results. There must be a difference in our environments. If you visit /admin/structure/media/manage/video/fields, does your video media bundle have a "field_track"?

[dmer]

@mjordan I do not see a "field_track" on that media bundle - should I add one?

[mjordan]

I think so but you should not do so based only on my advice. My understanding is that both the standard audio and video media bundles have this field. I think we should ask in slack if that is the case before you do anything, Do you want me to ask that?

[mjordan]

Note to @mjordan : https://github.com/mjordan/islandora_workbench/blob/main/workbench_utils.py#L3830 should not be hard coded to 'field_track', it should take into account the field name defined in the media_track_file_fields config setting.

^^ Done.

[mjordan]

@dmer mind if I close this?

[dmer]

That's fine - thanks @mjordan