Add Location removal support and update SourcePartitionedTable

[abaranec]

This fixes #4372 and #3077.

[cpwright]

Don't you need to fix this check in MergedListener to use the recordedVariablesAreValid instead?

            for (ListenerRecorder recorder : recorders) {
                if (recorder.getNotificationStep() == currentStep) {
                    added += recorder.getAdded().size();
                    removed += recorder.getRemoved().size();
                    modified += recorder.getModified().size();
                    shifted += recorder.getShifted().getEffectiveSize();
                }
            }

[abaranec]

No, the recorder will return empty everytthing when recorded values are not valid. That's why I did it that way

[rcaudy]

A table cannot double-notify. Those fields have to be empty. I don't think override to recordedVariablesAreValid is necessary or advisable.

[cpwright]

The source table may not double notify. In the case that the source table of the ListenerRecorder notifies us of an error, we should correctly set the notification step and record the error - the listener did have a notification on that step. Given that there is an error there will never again be a valid notification from that source table.

The update in this case is going to be null; and if we ask the question "did this get updated on the current step" and then fetch and dereference the null update as is done on line 319-326 of MergedListener we have an error. The change to the recordedVariablesAreValid (which is just asking about the step) means that we get the appropriate empty update in this section of the MergedListener.

[cpwright]

I am satisfied with this code. @rcaudy can you please assign a Community team member to review it?

[rcaudy]

I think this is fundamentally flawed. You will not be able to properly handle previous data from the removed table location when processing a remove, which makes this entire effort kind of moot.

I think there's utility in adding proper removal propagation, if it's corrected to not be part of the interface.

[rcaudy]

Since the implementations of removal are out of scope for this PR (and for core at this time), I think we can rehabilitate the PR if we ensure that the removeTableLocation JavaDoc clearly states that implementations must provide certain guarantees:
Reads should only succeed against removed locations if they will return complete, correct, consistent data. Otherwise, they should fail with a meaningful error message.

[rcaudy]

• In the buffer:
  ** If you see a remove: if there’s a pending add, delete it, and ignore the remove
• In consumers of the buffer:
  ** Always process removes before adds

[rcaudy]

These are more aggressive than we need to be. We can absolutely read from cached pages. We cannot read new pages.

[abaranec]

I think that we agreed to stay aggressive between @rcaudy @cpwright and @abaranec