Create new large images for each item in a folder

[gabrielle6249]

This adds an endpoint that creates large images for an entire folder, as mentioned here. It iterates through the folder's items, skips over items with a large image already and creates a large image for the items without one. It has several options, one that recurses child folders, a second that creates a job for each large image item, and another that makes the jobs local.

This does not yet retry creating large images for items with an unfinished large image, but seemed to be a good start.

[manthey]

Add a limit=2 to the childFiles call; we don't want to retrieve any more data than needed for this.

[gabrielle6249]

Agreed. Changed that in this commit.

[manthey]

These lines can be simplied -- there is no need to load the file; we already have it. That ism you can get rid of largeImageFileId = str(files[0]['_id']) and just do ImageItem().createImageItem(item, files[0], createJob=createJobs, localJob=localJobs)

[gabrielle6249]

That's much cleaner. Change made here.

[manthey]

I'd like to add a basic test of this. Something like

@pytest.mark.usefixtures('unbindLargeImage')
@pytest.mark.plugin('large_image')
def testFolderCreateImages(server, admin, user, fsAssetstore):
    file = utilities.uploadExternalFile('sample_image.ptif', admin, fsAssetstore)
    itemId = file['itemId']
    item = Item().load(itemId, user=admin)
    folderId = str(item['folderId'])
    # Remove the large image from this item
    ImageItem().delete(item)
    # Ask to make all items in this folder large images
    resp = server.request(
        method='PUT', path=f'/large_image/folder/{folderId}/tiles', user=admin)
    assert utilities.respStatus(resp) == 200
    assert resp.json['largeImagesCreated'] == 1
    item = Item().load(itemId, user=admin)
    # Check that this item became a large image again
    assert 'largeImage' in item

could be added to the girder/test_girder/test_large_image.py file.

A better test set would also create a subfolder and put a file there to test recursion, and would test running the process a second time doesn't create any large images since they already exist and are fine.

[manthey]

This seems to be a fragment from the item tiles endpoint with the force flag. If this is a function renaming from the force flag, then it should go back to being called force for consistency and then internally, this goes back to createJob='always' if self.boolParam('force', params, default=False) else True. The third state (don't use a job no matter what) would require the endpoint's parameter to accept three states (functionally these are 'always', never (False), and if-needed (True)).

[gabrielle6249]

In order for the parameter to accept the three states, do you think it's best to make a string parameter and change createJobs accordingly? Should the flag still be called force?

[manthey]

We probably need to catch both TileSourceError and KeyError. Specifically, if a source has been uninstalled, it will throw a KeyError.

[gabrielle6249]

Good point. Change made here.

[manthey]

Let's extend this to test a few more conditions. Continuing within this function:

    # Hitting the endpoint again should skip the item
    resp = server.request(
        method='PUT', path=f'/large_image/folder/{folderId}/tiles', user=admin)
    assert utilities.respStatus(resp) == 200
    assert resp.json['itemsSkipped'] == 1

    # If the item's source isn't working, it should be recreated.
    item['largeImage']['sourceName'] = 'unknown'
    Item().updateItem(item)
    resp = server.request(
        method='PUT', path=f'/large_image/folder/{folderId}/tiles', user=admin)
    assert utilities.respStatus(resp) == 200
    assert resp.json['largeImagesRemovedAndRecreated'] == 1

[gabrielle6249]

Function modified here.

[manthey]

Great, There are two minor tasks in #1139 that can be in future PRs (the ability to delete pending creation jobs and the ability to force un/redoing existing large images based on a filter).