EQL: Adds delete async EQL search result action

[imotov]

Adds support for deleting async EQL search results to EQL search API.

Relates to #49638

[elasticmachine]

Pinging @elastic/es-ql (:Query Languages/EQL)

[jimczi]

Nice one. I wonder if we can have a single transport action internally and different rest endpoints ?
The logic is the same for async and eql so they could use the same action ?

[imotov]

The logic is slightly different when it comes to task cancellation at the moment. EQL is relaying on the task manager for cancellation while async is using AsyncSearchTask#cancelTask for that. I also hope that in a long run we will separate async search and eql from the security permissions perspective, and I think at this point transport separation might become handy as well.

[jimczi]

The logic is slightly different when it comes to task cancellation at the moment. EQL is relaying on the task manager for cancellation while async is using AsyncSearchTask#cancelTask for that

We should address this, I don't see why they'd need to call different cancel method.

I also hope that in a long run we will separate async search and eql from the security permissions perspective, and I think at this point transport separation might become handy as well.

Do you have a use case in mind ? I was under the impression that the security model was settled since it relates to the user that submitted the initial action rather than the context (async or eql) ?

[imotov]

We should address this, I don't see why they'd need to call different cancel method.

We use different cancel methods because async search is using its own self-managed cancellation logic and eql search is relying on the task manager's new cascading cancellation functionality for cancellation of children. I think we should switch async search to task management cancellation eventually, but I feel like it might be out of scope for this PR.

I could add a cancelTask method to AsyncTask as a workaround, but I thought having two transports and encapsulating all non-trivial logic into a shared results service would be a cleaner solution that has a nice side effect of simplified unit testing.

Do you have a use case in mind ?

I was thinking "I want to enable only eql functionality, so I enable all eql* transport functionality" without needing to know that deletion of async results is handling by the same shared logic with async search.

I was under the impression that the security model was settled since it relates to the user that submitted the initial action rather than the context (async or eql) ?

I am still hopping we will have separate indices when system indices land. I was under impression that using the same index was temporary workaround to remove dependency for system indices.

[jimczi]

We use different cancel methods because async search is using its own self-managed cancellation logic and eql search is relying on the task manager's new cascading cancellation functionality for cancellation of children. I think we should switch async search to task management cancellation eventually, but I feel like it might be out of scope for this PR.

That's not correct, we use the same cancellation, async search checks if there is a cancel in flight before sending the cancellation but that could be removed. The difference in cancellation is for submit task but that's another problem.

I am still hopping we will have separate indices when system indices land. I was under impression that using the same index was temporary workaround to remove dependency for system indices.

I think it's ok to share, the main benefit is that we can use a single maintenance service. In fact I don't see a good reason not to share.

[imotov]

@jimczi any additional thoughts on the changes I added last week?

[jimczi]

LGTM, I wonder if we can apply the same reasoning to the internal get action ?

[imotov]

It is very tricky with get because of readers.

[costin]

Not much from my side - LGTM.