[SIEM][Detection Engine][Lists] Adds the ability for exception lists to be multi-list queried.

[FrankHassanabad]

Summary

• Adds the ability for exception lists to be multi-list queried
• Fixes a bunch of script issues where I did not update everywhere I needed to use ip_list and deletes an old list that now lives within the new/lists folder
• Fixes a few io-ts issues with Encode Decode while I was in there.
• Adds two more types and their tests for supporting converting between comma separated strings and arrays for GET calls.
• Fixes one weird circular dep issue while adding more types.

You now send into the find an optional comma separated list of exception lists their namespace type and any filters like so:

GET /api/exception_lists/items/_find?list_id=simple_list,endpoint_list&namespace_type=single,agnostic&filtering=filter1,filter2"

And this will return the results of both together with each filter applied to each list. If you use a sort field and ordering it will order across the lists together as if they are one list. Filter is optional like before. If you provide less filters than there are lists, the lists will only apply the filters to each list until it runs out of filters and then not filter the other lists.

If at least one list is found this will not return a 404 but it will only query the list(s) it did find. If none of the lists are found, then this will return a 404 not found exception.

Script testing

See these files for more information:

• find_exception_list_items.sh
• find_exception_list_items_by_filter.sh

But basically you can create two lists and an item for each of the lists:

./post_exception_list.sh ./exception_lists/new/exception_list.json
./post_exception_list_item.sh ./exception_lists/new/exception_list_item.json

./post_exception_list.sh ./exception_lists/new/exception_list_agnostic.json
./post_exception_list_item.sh ./exception_lists/new/exception_list_item_agnostic.json

And then you can query these two lists together:

./find_exception_list_items.sh simple_list,endpoint_list single,agnostic

Or for filtering you can query both and add a filter for each one:

./find_exception_list_items_by_filter.sh simple_list,endpoint_list "exception-list.attributes.name:%20Sample%20Endpoint%20Exception%20List,exception-list-agnostic.attributes.name:%20Sample%20Endpoint%20Exception%20List" single,agnostic

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[yctercero]

LGTM! Thanks so much for this - super appreciated. Pulled down and tested with scripts, the tests look awesome too!

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: fcb78db

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
lists	88	+3	85

History

• 💚 Build #61165 succeeded 66a7246
• 💚 Build #61094 succeeded 254de4a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)