[ML] Adding initial file analysis overrides #74376

jgowdyelastic · 2020-08-05T15:44:05Z

Adds the ability to edit the override settings if the initial analysis has failed.
This allows the user to add some domain knowledge to help the find_file_structure endpoint work out the rest of the file structure.

to test, this data can be added to a file and uploaded:

2344:20200424:175400.779 response code: 200
2344:20200424:175500.519 response code: 200
2344:20200424:175710.752 response code: 200
2344:20200424:175710.777 response code: 503
2344:20200424:175800.001 response code: 200
2344:20200424:175800.006 response code: 404
2344:20200424:175800.315 response code: 200
2344:20200424:175800.535 response code: 200
2344:20200424:175800.769 response code: 201
2344:20200424:175900.411 response code: 200
2344:20200424:175900.648 response code: 200
2344:20200424:180000.444 response code: 200
2344:20200424:180000.777 response code: 200
2344:20200424:180100.537 response code: 200
2344:20200424:180100.890 response code: 504

Fixes #38868

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support

elasticmachine · 2020-08-06T10:04:09Z

Pinging @elastic/ml-ui (:ml)

jgowdyelastic · 2020-08-06T10:04:22Z

cc @droberts195

walterra

LGTM, also did a local test with the provided sample.

walterra · 2020-08-06T13:03:12Z

...on/datavisualizer/file_based/components/file_datavisualizer_view/file_datavisualizer_view.js

-            {!loading && !loaded && <AboutPanel onFilePickerChange={this.onFilePickerChange} />}
+          <>
+            {!loading && !loaded && (
+              <>


I wonder if this fragment wrapper can be removed since it's only one element on the inside, maybe a leftover from refactoring?

alvarezmelissa87 · 2020-08-06T13:46:47Z

...on/datavisualizer/file_based/components/file_datavisualizer_view/file_datavisualizer_view.js

+                <FileCouldNotBeRead
+                  error={serverError}
+                  loaded={loaded}
+                  showEditFlyout={() => this.showEditFlyout()}


Does this.showEditFlyout need to be wrapped in an arrow function here?

alvarezmelissa87

Small question but otherwise LGTM ⚡

jgowdyelastic · 2020-08-06T15:29:12Z

@elasticmachine merge upstream

jgowdyelastic · 2020-08-10T16:23:38Z

@elasticmachine merge upstream

jgowdyelastic · 2020-08-10T19:09:05Z

retest

kibanamachine · 2020-08-10T21:03:35Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 6421627

Build metrics

async chunks size

id	value	diff	baseline
ml	7.9MB	+2.8KB	7.9MB

History

💚 Build #67312 succeeded 6421627
💔 Build #66948 failed bde656c
💔 Build #66851 failed 0a6c91b
💔 Build #66836 failed ff05982
💔 Build #66670 failed 851fffb

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* [ML] Adding initial file analysis overrides * using common default sample lines * correcting bottom padding * adding missed padding * code clean up * fixing translations Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* master: (339 commits) [Ingest Node Pipelines] Sentence-case processor names (elastic#74645) Bump angular dependency from 1.7.9 to 1.8.0 (elastic#74482) [ML] Fixing schema for custom rule conditions (elastic#74676) [ML] Refactor in preparation for new es client (elastic#74552) [ML] Adding initial file analysis overrides (elastic#74376) Allow any hostname for chromium proxy bypass (elastic#74693) [ML] ML on Kibana Management: Add ability to pass a group ID filter to job management page (elastic#74533) [Metrics UI] Fix No Data preview pluralization (elastic#74399) [Bug][Security_Solution][Telemetry] Capitalize S in macOS (elastic#74688) Remove karma tests from legacy maps (elastic#74668) [Ingest Manager] stop creating events-* index pattern and placeholder index (elastic#74683) [Enterprise Search] Update the browser/document title on plugin navigation (elastic#74392) [visualizations] Add i18n translation for 'No results found' (elastic#74619) [maps] convert vector style properties to TS (elastic#74553) bump geckodriver binary to 0.27 (elastic#74638) fix: update apm agents to catch abort requests (elastic#74658) [Security Solution] Resolver children pagination (elastic#74603) add memoryStatus to df analytics page and analytics table in management (elastic#74570) [Ingest Manager] Allow prerelease in package version (elastic#74452) [App Arch]: remove legacy karma tests (elastic#74599) ...

fbaligand · 2020-08-11T09:04:19Z

Great new feature!

…nes/processor-forms-a-d * 'master' of github.com:elastic/kibana: (26 commits) [Telemetry][API Integration] size_in_bytes to be a number (elastic#74664) [ILM] Convert node details flyout to TS (elastic#73707) [Ingest Node Pipelines] Sentence-case processor names (elastic#74645) Bump angular dependency from 1.7.9 to 1.8.0 (elastic#74482) [ML] Fixing schema for custom rule conditions (elastic#74676) [ML] Refactor in preparation for new es client (elastic#74552) [ML] Adding initial file analysis overrides (elastic#74376) Allow any hostname for chromium proxy bypass (elastic#74693) [ML] ML on Kibana Management: Add ability to pass a group ID filter to job management page (elastic#74533) [Metrics UI] Fix No Data preview pluralization (elastic#74399) [Bug][Security_Solution][Telemetry] Capitalize S in macOS (elastic#74688) Remove karma tests from legacy maps (elastic#74668) [Ingest Manager] stop creating events-* index pattern and placeholder index (elastic#74683) [Enterprise Search] Update the browser/document title on plugin navigation (elastic#74392) [visualizations] Add i18n translation for 'No results found' (elastic#74619) [maps] convert vector style properties to TS (elastic#74553) bump geckodriver binary to 0.27 (elastic#74638) fix: update apm agents to catch abort requests (elastic#74658) [Security Solution] Resolver children pagination (elastic#74603) add memoryStatus to df analytics page and analytics table in management (elastic#74570) ...

* [ML] Adding initial file analysis overrides * using common default sample lines * correcting bottom padding * adding missed padding * code clean up * fixing translations Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

jgowdyelastic added 4 commits August 5, 2020 16:43

[ML] Adding initial file analysis overrides

851fffb

using common default sample lines

ff05982

correcting bottom padding

205d370

adding missed padding

0a6c91b

jgowdyelastic requested review from peteharverson, walterra, darnautov, qn895 and alvarezmelissa87 August 6, 2020 10:03

jgowdyelastic self-assigned this Aug 6, 2020

jgowdyelastic added :ml Feature:File and Index Data Viz ML file and index data visualizer release_note:enhancement review v7.10.0 v8.0.0 labels Aug 6, 2020

jgowdyelastic marked this pull request as ready for review August 6, 2020 10:04

jgowdyelastic requested a review from a team as a code owner August 6, 2020 10:04

walterra approved these changes Aug 6, 2020

View reviewed changes

alvarezmelissa87 reviewed Aug 6, 2020

View reviewed changes

alvarezmelissa87 approved these changes Aug 6, 2020

View reviewed changes

code clean up

1ea0ff8

Merge branch 'master' into adding-initial-file-analysis-overrides

bde656c

elasticmachine and others added 2 commits August 10, 2020 10:23

Merge branch 'master' into adding-initial-file-analysis-overrides

afb02fb

fixing translations

6421627

jgowdyelastic merged commit 6bea373 into elastic:master Aug 11, 2020

jgowdyelastic deleted the adding-initial-file-analysis-overrides branch August 11, 2020 07:47

jgowdyelastic mentioned this pull request Aug 11, 2020

[7.x] [ML] Adding initial file analysis overrides (#74376) #74727

Merged

droberts195 mentioned this pull request Dec 3, 2020

Data Visualizer fails to import data without a timestamp #63526

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Adding initial file analysis overrides #74376

[ML] Adding initial file analysis overrides #74376

jgowdyelastic commented Aug 5, 2020 •

edited

Loading

elasticmachine commented Aug 6, 2020

jgowdyelastic commented Aug 6, 2020

walterra left a comment

walterra Aug 6, 2020

alvarezmelissa87 Aug 6, 2020

alvarezmelissa87 left a comment

jgowdyelastic commented Aug 6, 2020

jgowdyelastic commented Aug 10, 2020

jgowdyelastic commented Aug 10, 2020

kibanamachine commented Aug 10, 2020

fbaligand commented Aug 11, 2020

[ML] Adding initial file analysis overrides #74376

[ML] Adding initial file analysis overrides #74376

Conversation

jgowdyelastic commented Aug 5, 2020 • edited Loading

elasticmachine commented Aug 6, 2020

jgowdyelastic commented Aug 6, 2020

walterra left a comment

Choose a reason for hiding this comment

walterra Aug 6, 2020

Choose a reason for hiding this comment

alvarezmelissa87 Aug 6, 2020

Choose a reason for hiding this comment

alvarezmelissa87 left a comment

Choose a reason for hiding this comment

jgowdyelastic commented Aug 6, 2020

jgowdyelastic commented Aug 10, 2020

jgowdyelastic commented Aug 10, 2020

kibanamachine commented Aug 10, 2020

💚 Build Succeeded

Build metrics

async chunks size

History

fbaligand commented Aug 11, 2020

jgowdyelastic commented Aug 5, 2020 •

edited

Loading