Lazy resampling: MetaTensor.applied_operations ends up out of order

[atbenmurray]

Describe the bug
When executing pipelines with lazy resampling, the applied_operations list on a MetaTensor instance can end up out of order.

To Reproduce
Steps to reproduce the behavior:

1. Run tests/test_integration_lazy_samples.py
2. Add a breakpoint at

   MONAI/tests/test_integration_lazy_samples.py

   Line 155 in 4f29172

   inverted = [inverter(b_data) for b_data in decollate_batch(batch_data)]

3. Examine the state of batch_data and you'll see that the applied operations are out of order. Lambdad is the last transform in the pipeline, but appears as the first element in the pipeline

Correct order:

[SpatialResample, Orientation, RandRotate90d, RandCropByPosNegLabel, RandRotated, RandZoomd, ResizeWithPadOrCrop, Rotate, Lambdad]

Observed order:

[Lambdad, SpatialResample, Orientation, RandRotate90d, RandCropByPosNegLabel, RandRotated, RandZoomd, ResizeWithPadOrCrop, Rotate]

Note that this issue is fixed in #6257

[wyli]

thanks, for this issue I had discussions with Andriy, the current solution is based on my analysis here (quoting my email response):

I’m able to reproduce the issue, in general it’s not a well-defined case that we apply invertible but not lazy transforms (such as LambaD)
during combining a set of lazy transforms…

e.g.:
Compose([Rotate, LambdaD, Resize], lazy_evaluation=True)

Fixing this with #6372 by enhancing the warning message.

other ways to fix it:

• execute all the pending operations when the next transform is LambdaD
• raise an error when applying such kind of transform sequences instead of a warning

I think these alternative ways are too harsh for people who don’t use Invert, but I’m open to discussions.

(and after this discussion, we had follow-up PRs to delay the warning #6426, #6428)

[atbenmurray]

This works as intended in #6257

[wyli]

Addressed by #6428 providing a warning message.

The source of the issue is that the user specified transform sequence may not be invertible by definition. The user will receive this message when inverting such cases, but the system doesn't force resampling.

Other solutions such as forcing resampling will cause performance regression for example in auto3dseg https://github.com/Project-MONAI/research-contributions/blob/18bb1d859fa65ad2221d2a6b6771e82b8e81ce89/auto3dseg/algorithm_templates/segresnet/scripts/segmenter.py#L294

Cc @myron

[atbenmurray]

@wyli I disagree that this should be closed. It is an error that hasn't been addressed on dev and it is fixable.

[wyli]

The dev branch's warning message should give the user sufficient information to adjust their code accordingly.

The solution in #6257 will quietly cause auto3dseg performance regression, and with the fact that the logging logic not working properly, it'll cause bugs that are very difficult to detect.

[atbenmurray]

The logging logic is working. You just have to configure the logging mechanism. I have tests in tests/test_compose.py that show the logging is working correctly. Please show me what is not working in the logging, rather than the logging configuration, and I'll look at a fix.

[atbenmurray]

I don't think that performance should come at the cost of correctness. It might be that you get better performance in auto3dseg with lazy turned on if the pipeline isn't working correctly but that is not a way to get performance. We can address that in future versions with compilation flags that specifically modify behaviour in the pipeline

[mjorgecardoso]

This seems related to the other lazy resampling PR, so we can discuss tomorrow.