Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM][Detections] Value Lists Management Modal #67068

Merged
merged 31 commits into from
Jul 14, 2020

Conversation

rylnd
Copy link
Contributor

@rylnd rylnd commented May 20, 2020

Summary

Addresses #65929.

Highlights:

  • Adds Modal, Form, and Table components
  • Adds a general useCursor hook to optimize pagination of queries

Outstanding tasks:

  • display feedback to user on success/error
  • styling
  • table pagination
    • includes use of new cursor tech
  • move lists functions to lists plugin
  • tests

Outstanding issues:

Potential UI/UX improvements:

Alerts_-_Kibana

Current view:

Alerts_-_Kibana

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

@rylnd rylnd changed the title Value lists ui [SIEM][Detections] Value Lists Management Modal May 20, 2020
@spong
Copy link
Member

spong commented May 20, 2020

For the two UX items:

[UX]: with displaying feedback in a modal
  • I think it might be worthwhile to see if EUI is okay with us increasing the z-index of the toasts so they display above the modal mask. While the EuiCallout would be sufficient for upload errors, it'd be unwieldy to display success/error feedback from the export/delete actions. E.g. if you click to delete two lists one right after the next, both error out, do we just append the errors? Clear previous import errors? Maybe just append and have an X next to each so they can be dismissed individually? This approach also results in UI bounce unless we reserve a min height of empty space for the callout -- which is okay, but I think keeping things simple with our global toast pattern is best so long as EUI is 👍
[UX]: issues with auto-upload
  • Perhaps we can use the EuiFilePicker in its small format, and then have an Upload list button to the right of it? This also leaves room for helper text like in the mocks (i.e. "Upload a .csv file up to 400MB"). On upload the button (and radios?) disable, and the FilePicker switches to its loading state, and on success the FilePicker clears its previous file ref and the button re-enables -- similar to the Rule Import modal (which had the same bug 😅 )

cc @marrasherrier

@rylnd
Copy link
Contributor Author

rylnd commented May 28, 2020

Regarding the above UX issues: We ended up going with toast notifications, and manual uploads. We reset the form (file picker and radio group) on a successful upload/import.

rylnd added 10 commits July 1, 2020 11:45
Imports and uses the hooks provided by the lists plugin. Tests coming
next.
* uses useEffect on a task's state instead of promise chaining
* handles the fact that API calls can be rejected with strings
* uses exportList function instead of hook
For e.g. findLists, we can send along a cursor to optimize our query. On
the backend, this cursor is used as part of a search_after query.
* Does not require args for setCursor as they're already passed to the
hook
* Finds nearest cursor for the same page size

Eventually this logic will also include sortField as part of the
hash/lookup, but we do not currently use that on the frontend.
We were previously storing the cursor on the _current_ page, when it's
only truly valid for the _next_ page (and beyond).

This was causing a few issues, but now that it's fixed everything works
great.
This allows us to search_after a previous page's search, if available.
This is just a blob, so we have nothing to validate.
After uploading a list, the modal was being shown twice. Declaring the
constituent state dependencies separately fixed the issue.
These hooks no longer care about/expose an abort function. In this one
case where we need that functionality, we can do it ourselves relatively
simply.
rylnd added 8 commits July 1, 2020 15:54
Dates were wrapping (and raw), and so were wrapped in a FormattedDate
component. However, since this component didn't wrap, we needed to
shrink/truncate the uploaded_by field as well as allow the fileName to
truncate.
enzymejs/enzyme#2073 seems to be an ongoing
issue, and causes components with useEffect to update after the test is
completed.

waitForUpdates ensures that updates have completed within an act()
before continuing on.
 Conflicts:
	x-pack/plugins/security_solution/public/lists_plugin_deps.ts
	x-pack/plugins/security_solution/public/overview/pages/overview.test.tsx
Each of these logs a console.error without them.
@rylnd rylnd marked this pull request as ready for review July 6, 2020 17:08
@rylnd rylnd requested review from a team as code owners July 6, 2020 17:08
@marrasherrier
Copy link
Contributor

after uploading a list, will it show up immediately in the table below? not showing up now
Screen Shot 2020-07-07 at 3 21 35 PM

});
fetchLists();
},
// eslint-disable-next-line react-hooks/exhaustive-deps
Copy link
Contributor

@FrankHassanabad FrankHassanabad Jul 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible or easy to remove the disabling of the linter rule by passing in fetchLists?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The story here is in the commit that added it. I agree that this is a smell and I'll try it again from a different angle. If you have suggestions, though, I'd appreciate them! Ditto on useCursor.

{
field: 'created_at',
name: i18n.COLUMN_UPLOAD_DATE,
/* eslint-disable-next-line react/display-name */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a displayName: createdAt to avoid disabling the linter here or no?

Ref:
https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/display-name.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The complaint is because this inline function returning markup is itself technically a component. I think a naive render: (value) => <NamedComponent value={value} /> would suffer the same issue, so you'd need to write a component that takes the full set of arguments to allow you to do render: NamedComponent.

I'll give it a shot.

@rylnd
Copy link
Contributor Author

rylnd commented Jul 7, 2020

after uploading a list, will it show up immediately in the table below? not showing up now

@marrasherrier correct, this is the second bullet under "Potential UI/UX Improvements." I don't know what the best solution is here... @FrankHassanabad @spong ?

await new Promise((resolve) => setTimeout(resolve, 0));
wrapper.update();
});
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably need this extra utility but I usually use a waitFor() from here:
https://testing-library.com/docs/dom-testing-library/api-async

When I can before adding a workaround and then I do the workaround when I can't use one of those. Might help you out, might not. Ignore this if it doesn't, just mentioning it in case its helpful.

@FrankHassanabad
Copy link
Contributor

after uploading a list, will it show up immediately in the table below? not showing up now

@marrasherrier correct, this is the second bullet under "Potential UI/UX Improvements." I don't know what the best solution is here... @FrankHassanabad @spong ?

I have on my list wait_for tasks for lists to add to most of the endpoints which will fix this before we ship.

Copy link
Contributor

@FrankHassanabad FrankHassanabad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked it over and its really clean code and I always learn interesting things from PR's like this that have good links to other issues from Github across libraries.

A lot of good attention to details, I only had some minor questions but am going to give this a 👍 and LGTM! Our code base is better with it than without it.

 Conflicts:
	x-pack/plugins/security_solution/public/lists_plugin_deps.ts
	x-pack/plugins/security_solution/public/overview/pages/overview.test.tsx
 Conflicts:
	x-pack/plugins/lists/public/lists/api.test.ts
	x-pack/plugins/lists/public/lists/api.ts
	x-pack/plugins/lists/public/shared_exports.ts
	x-pack/plugins/security_solution/public/common/lib/kibana/hooks.ts
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/form.test.tsx
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/form.tsx
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/index.tsx
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/modal.test.tsx
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/modal.tsx
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/table.test.tsx
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/table.tsx
	x-pack/plugins/security_solution/public/detections/components/value_lists_management_modal/translations.ts
	x-pack/plugins/security_solution/public/lists_plugin_deps.ts
It looks like this broke because EuiTable's pagination changed from a
button to an anchor tag.
These have style issues, and anything above 5 rows causes the modal to
scroll, so we're going to disable it for now.
We don't display the nice errors in the case of an ApiError right now,
but this is better than it was.
Our start() no longer resolves in a meaningful way, so we instead need
to perform the refetch in an effect watching the result of our delete.
useAsync generally does not know how what its tasks are going to be
rejected with, hence the unknown.

For these API calls we know that it will be an Error, but I don't
currently have a way to type that generally. For now, we'll cast it
where we use it.
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@rylnd rylnd merged commit 835c13d into elastic:master Jul 14, 2020
@rylnd rylnd deleted the value_lists_ui branch July 14, 2020 02:11
rylnd added a commit that referenced this pull request Jul 14, 2020
* Add Frontend components for Value Lists Management Modal

Imports and uses the hooks provided by the lists plugin. Tests coming
next.

* Update value list components to use newest Lists API

* uses useEffect on a task's state instead of promise chaining
* handles the fact that API calls can be rejected with strings
* uses exportList function instead of hook

* Close modal on outside click

* Add hook for using a cursor with paged API calls.

For e.g. findLists, we can send along a cursor to optimize our query. On
the backend, this cursor is used as part of a search_after query.

* Better implementation of useCursor

* Does not require args for setCursor as they're already passed to the
hook
* Finds nearest cursor for the same page size

Eventually this logic will also include sortField as part of the
hash/lookup, but we do not currently use that on the frontend.

* Fixes useCursor hook functionality

We were previously storing the cursor on the _current_ page, when it's
only truly valid for the _next_ page (and beyond).

This was causing a few issues, but now that it's fixed everything works
great.

* Add cursor to lists query

This allows us to search_after a previous page's search, if available.

* Do not validate response of export

This is just a blob, so we have nothing to validate.

* Fix double callback post-import

After uploading a list, the modal was being shown twice. Declaring the
constituent state dependencies separately fixed the issue.

* Update ValueListsForm to manually abort import request

These hooks no longer care about/expose an abort function. In this one
case where we need that functionality, we can do it ourselves relatively
simply.

* Default modal table to five rows

* Update translation keys following plugin rename

* Try to fit table contents on a single row

Dates were wrapping (and raw), and so were wrapped in a FormattedDate
component. However, since this component didn't wrap, we needed to
shrink/truncate the uploaded_by field as well as allow the fileName to
truncate.

* Add helper function to prevent tests from logging errors

enzymejs/enzyme#2073 seems to be an ongoing
issue, and causes components with useEffect to update after the test is
completed.

waitForUpdates ensures that updates have completed within an act()
before continuing on.

* Add jest tests for our form, table, and modal components

* Fix translation conflict

* Add more waitForUpdates to new overview page tests

Each of these logs a console.error without them.

* Fix bad merge resolution

That resulted in duplicate exports.

* Make cursor an optional parameter to findLists

This param is an optimization and not required for basic functionality.

* Tweaking Table column sizes

Makes actions column smaller, leaving more room for everything else.

* Fix bug where onSuccess is called upon pagination change

Because fetchLists changes when pagination does, and handleUploadSuccess
changes with fetchLists, our useEffect in Form was being fired on every
pagination change due to its onSuccess changing.

The solution in this instance is to remove fetchLists from
handleUploadSuccess's dependencies, as we merely want to invoke
fetchLists from it, not change our reference.

* Fix failing test

It looks like this broke because EuiTable's pagination changed from a
button to an anchor tag.

* Hide page size options on ValueLists modal table

These have style issues, and anything above 5 rows causes the modal to
scroll, so we're going to disable it for now.

* Update error callbacks now that we have Errors

We don't display the nice errors in the case of an ApiError right now,
but this is better than it was.

* Synchronize delete with the subsequent fetch

Our start() no longer resolves in a meaningful way, so we instead need
to perform the refetch in an effect watching the result of our delete.

* Cast our unknown error to an Error

useAsync generally does not know how what its tasks are going to be
rejected with, hence the unknown.

For these API calls we know that it will be an Error, but I don't
currently have a way to type that generally. For now, we'll cast it
where we use it.

* Import lists code from our new, standardized modules

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 14, 2020
* master: (314 commits)
  [APM] Use status_code field to calculate error rate (elastic#71109)
  [Observability] Change appLink passing the date range (elastic#71259)
  [Security] Add Timeline improvements (elastic#71506)
  adjust vislib bar opacity (elastic#71421)
  Fix ScopedHistory mock and adapt usages (elastic#71404)
  [Security Solution] Add hook for reading/writing resolver query params (elastic#70809)
  [APM] Bug fixes from ML integration testing (elastic#71564)
  [Discover] Add caused_by.type and caused_by.reason to error toast modal (elastic#70404)
  [Security Solution] Add 3rd level breadcrumb to admin page (elastic#71275)
  [Security Solution][Exceptions] Exception modal bulk close alerts that match exception attributes (elastic#71321)
  Change signal.rule.risk score mapping from keyword to float (elastic#71126)
  Added help text where needed on connectors and alert actions UI (elastic#69601)
  [SIEM][Detections] Value Lists Management Modal (elastic#67068)
  [test] Skips test preventing promotion of ES snapshot elastic#71582
  [test] Skips test preventing promotion of ES snapshot elastic#71555
  [ILM] Fix alignment of the timing field (elastic#71273)
  [SIEM][Detection Engine][Lists] Adds the ability for exception lists to be multi-list queried. (elastic#71540)
  initial telemetry setup (elastic#69330)
  [Reporting] Formatting fixes for CSV export in Discover, CSV download from Dashboard panel (elastic#67027)
  Search across spaces (elastic#67644)
  ...
jloleysens added a commit to jloleysens/kibana that referenced this pull request Jul 14, 2020
…t-apps-page-titles

* 'master' of github.com:elastic/kibana: (88 commits)
  [ML] Functional tests - disable DFA creation and cloning tests
  [APM] Use status_code field to calculate error rate (elastic#71109)
  [Observability] Change appLink passing the date range (elastic#71259)
  [Security] Add Timeline improvements (elastic#71506)
  adjust vislib bar opacity (elastic#71421)
  Fix ScopedHistory mock and adapt usages (elastic#71404)
  [Security Solution] Add hook for reading/writing resolver query params (elastic#70809)
  [APM] Bug fixes from ML integration testing (elastic#71564)
  [Discover] Add caused_by.type and caused_by.reason to error toast modal (elastic#70404)
  [Security Solution] Add 3rd level breadcrumb to admin page (elastic#71275)
  [Security Solution][Exceptions] Exception modal bulk close alerts that match exception attributes (elastic#71321)
  Change signal.rule.risk score mapping from keyword to float (elastic#71126)
  Added help text where needed on connectors and alert actions UI (elastic#69601)
  [SIEM][Detections] Value Lists Management Modal (elastic#67068)
  [test] Skips test preventing promotion of ES snapshot elastic#71582
  [test] Skips test preventing promotion of ES snapshot elastic#71555
  [ILM] Fix alignment of the timing field (elastic#71273)
  [SIEM][Detection Engine][Lists] Adds the ability for exception lists to be multi-list queried. (elastic#71540)
  initial telemetry setup (elastic#69330)
  [Reporting] Formatting fixes for CSV export in Discover, CSV download from Dashboard panel (elastic#67027)
  ...

# Conflicts:
#	x-pack/plugins/index_management/public/application/index.tsx
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 14, 2020
* master: (72 commits)
  [test] Skips test preventing promotion of ES snapshot elastic#71612
  [Logs UI] Remove UUID from Alert Instances (elastic#71340)
  [Metrics UI] Remove UUID from Alert Instance IDs (elastic#71335)
  [ML] Functional tests - disable DFA creation and cloning tests
  [APM] Use status_code field to calculate error rate (elastic#71109)
  [Observability] Change appLink passing the date range (elastic#71259)
  [Security] Add Timeline improvements (elastic#71506)
  adjust vislib bar opacity (elastic#71421)
  Fix ScopedHistory mock and adapt usages (elastic#71404)
  [Security Solution] Add hook for reading/writing resolver query params (elastic#70809)
  [APM] Bug fixes from ML integration testing (elastic#71564)
  [Discover] Add caused_by.type and caused_by.reason to error toast modal (elastic#70404)
  [Security Solution] Add 3rd level breadcrumb to admin page (elastic#71275)
  [Security Solution][Exceptions] Exception modal bulk close alerts that match exception attributes (elastic#71321)
  Change signal.rule.risk score mapping from keyword to float (elastic#71126)
  Added help text where needed on connectors and alert actions UI (elastic#69601)
  [SIEM][Detections] Value Lists Management Modal (elastic#67068)
  [test] Skips test preventing promotion of ES snapshot elastic#71582
  [test] Skips test preventing promotion of ES snapshot elastic#71555
  [ILM] Fix alignment of the timing field (elastic#71273)
  ...
@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Sep 23, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:enhancement Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants