Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure S3 datasource flow #1049

Merged
merged 24 commits into from
Sep 27, 2023
Merged

Conversation

derek-ho
Copy link
Collaborator

@derek-ho derek-ho commented Sep 25, 2023

Description

  • Adds configure s3 datasource flow
  • Changes some references of data connections to datasource
  • Adds directory structure to separate out new and manage files/flows

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

derek-ho and others added 15 commits September 7, 2023 10:36
* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <dxho@amazon.com>

* get rid of lint

Signed-off-by: Derek Ho <dxho@amazon.com>

* test for flyout

Signed-off-by: Derek Ho <dxho@amazon.com>

* flyout to medium size

Signed-off-by: Derek Ho <dxho@amazon.com>

* make accelerate extensible

Signed-off-by: Derek Ho <dxho@amazon.com>

* get datasources and hook up to pplservice

Signed-off-by: Derek Ho <dxho@amazon.com>

* get flint working

Signed-off-by: Derek Ho <dxho@amazon.com>

* add datasource page with steps and buttons on bottom bar

Signed-off-by: Derek Ho <dxho@amazon.com>

* datasources as a new plugin and mostly working

Signed-off-by: Derek Ho <dxho@amazon.com>

* hook up manage to show datasources call

Signed-off-by: Derek Ho <dxho@amazon.com>

* update two tables with descriptions

Signed-off-by: Derek Ho <dxho@amazon.com>

* make some updates to the page

Signed-off-by: Derek Ho <dxho@amazon.com>

* cleanup unused files for data connections

Signed-off-by: Derek Ho <dxho@amazon.com>

* cleanup and add overview panel columns

Signed-off-by: Derek Ho <dxho@amazon.com>

* render tabs

Signed-off-by: Derek Ho <dxho@amazon.com>

* add unit tests

Signed-off-by: Derek Ho <dxho@amazon.com>

* update data test subj and snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add datasources to management overview

Signed-off-by: Derek Ho <dxho@amazon.com>

* remove spark logo and update snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

* refactor routes out

Signed-off-by: Derek Ho <dxho@amazon.com>

* separate out the roles

Signed-off-by: Derek Ho <dxho@amazon.com>

* bump version back to 3.0

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
* add acceleration management UI skeleton

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* Create new documentation link for acc

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* fix typos and minor bugs

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update snapshot

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update window location to hash

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove unused headers

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

---------

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
* rename data sources to data connections

Signed-off-by: Derek Ho <dxho@amazon.com>

* final cleanup

Signed-off-by: Derek Ho <dxho@amazon.com>

* update acceleration breadcrumb

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix API call for data connection page

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix integ test and data test subj and snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
…opensearch-project#1008)

* add fallback ui for manage and view datasources

Signed-off-by: Derek Ho <dxho@amazon.com>

* always show datasources via pplservice

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
* basic rendering for the access control tab

Signed-off-by: Derek Ho <dxho@amazon.com>

* hook up basic radio groups and euicombo boxes for query and acceleration permissions

Signed-off-by: Derek Ho <dxho@amazon.com>

* refactor and clean up unuseed inports

Signed-off-by: Derek Ho <dxho@amazon.com>

* remove unused import

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix import and snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix test

Signed-off-by: Derek Ho <dxho@amazon.com>

* Address PR comments

Signed-off-by: Derek Ho <dxho@amazon.com>

* Address PR comments

Signed-off-by: Derek Ho <dxho@amazon.com>

* Remove unused files and variables

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
…t#1024)

* Address previous PR comments and implement rudimentary delete

Signed-off-by: Derek Ho <dxho@amazon.com>

* Implement modal and instant delete showing up in list

Signed-off-by: Derek Ho <dxho@amazon.com>

* Refactor save or cancel to a shared component, implement hard coded datasource configurations tab

Signed-off-by: Derek Ho <dxho@amazon.com>

* Update test with mock role data

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add most functionality of edit connectiondetails

Signed-off-by: Derek Ho <dxho@amazon.com>

* Address PR comments

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
… tests

Signed-off-by: Derek Ho <dxho@amazon.com>
…boards-observability into configure

Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
…vability into configure

Signed-off-by: Derek Ho <dxho@amazon.com>
@derek-ho derek-ho changed the title Configure Configure S3 datasource flow Sep 25, 2023
@codecov
Copy link

codecov bot commented Sep 25, 2023

Codecov Report

Merging #1049 (8251678) into main (6efa950) will increase coverage by 0.00%.
Report is 2 commits behind head on main.
The diff coverage is n/a.

❗ Current head 8251678 differs from pull request most recent head 4d19eb4. Consider uploading reports for the commit 4d19eb4 to get more accurate results

@@           Coverage Diff           @@
##             main    #1049   +/-   ##
=======================================
  Coverage   44.42%   44.43%           
=======================================
  Files         325      325           
  Lines       19055    19072   +17     
  Branches     4631     4634    +3     
=======================================
+ Hits         8465     8474    +9     
- Misses      10046    10054    +8     
  Partials      544      544           
Flag Coverage Δ
dashboards-observability 44.43% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 22 files with indirect coverage changes

Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Derek Ho <dxho@amazon.com>
@derek-ho derek-ho merged commit 79d18df into opensearch-project:main Sep 27, 2023
7 checks passed
@derek-ho derek-ho deleted the configure branch September 27, 2023 15:44
derek-ho added a commit to derek-ho/dashboards-observability that referenced this pull request Oct 4, 2023
* Manage datasources (opensearch-project#967)

* fix name change bug and modify test to test behavior

Signed-off-by: Derek Ho <dxho@amazon.com>

* get rid of lint

Signed-off-by: Derek Ho <dxho@amazon.com>

* test for flyout

Signed-off-by: Derek Ho <dxho@amazon.com>

* flyout to medium size

Signed-off-by: Derek Ho <dxho@amazon.com>

* make accelerate extensible

Signed-off-by: Derek Ho <dxho@amazon.com>

* get datasources and hook up to pplservice

Signed-off-by: Derek Ho <dxho@amazon.com>

* get flint working

Signed-off-by: Derek Ho <dxho@amazon.com>

* add datasource page with steps and buttons on bottom bar

Signed-off-by: Derek Ho <dxho@amazon.com>

* datasources as a new plugin and mostly working

Signed-off-by: Derek Ho <dxho@amazon.com>

* hook up manage to show datasources call

Signed-off-by: Derek Ho <dxho@amazon.com>

* update two tables with descriptions

Signed-off-by: Derek Ho <dxho@amazon.com>

* make some updates to the page

Signed-off-by: Derek Ho <dxho@amazon.com>

* cleanup unused files for data connections

Signed-off-by: Derek Ho <dxho@amazon.com>

* cleanup and add overview panel columns

Signed-off-by: Derek Ho <dxho@amazon.com>

* render tabs

Signed-off-by: Derek Ho <dxho@amazon.com>

* add unit tests

Signed-off-by: Derek Ho <dxho@amazon.com>

* update data test subj and snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add datasources to management overview

Signed-off-by: Derek Ho <dxho@amazon.com>

* remove spark logo and update snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

* refactor routes out

Signed-off-by: Derek Ho <dxho@amazon.com>

* separate out the roles

Signed-off-by: Derek Ho <dxho@amazon.com>

* bump version back to 3.0

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add acceleration management UI  (opensearch-project#989)

* add acceleration management UI skeleton

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* Create new documentation link for acc

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* fix typos and minor bugs

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update snapshot

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* update window location to hash

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* remove unused headers

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

---------

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* Rename data sources to data connections (opensearch-project#1004)

* rename data sources to data connections

Signed-off-by: Derek Ho <dxho@amazon.com>

* final cleanup

Signed-off-by: Derek Ho <dxho@amazon.com>

* update acceleration breadcrumb

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix API call for data connection page

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix integ test and data test subj and snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add fallback to show if user does not have datasource API permissions (opensearch-project#1008)

* add fallback ui for manage and view datasources

Signed-off-by: Derek Ho <dxho@amazon.com>

* always show datasources via pplservice

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add access control tab content (opensearch-project#992)

* basic rendering for the access control tab

Signed-off-by: Derek Ho <dxho@amazon.com>

* hook up basic radio groups and euicombo boxes for query and acceleration permissions

Signed-off-by: Derek Ho <dxho@amazon.com>

* refactor and clean up unuseed inports

Signed-off-by: Derek Ho <dxho@amazon.com>

* remove unused import

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix import and snapshot

Signed-off-by: Derek Ho <dxho@amazon.com>

* fix test

Signed-off-by: Derek Ho <dxho@amazon.com>

* Address PR comments

Signed-off-by: Derek Ho <dxho@amazon.com>

* Address PR comments

Signed-off-by: Derek Ho <dxho@amazon.com>

* Remove unused files and variables

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>

* Delete datasource and Connection Configuration Tab (opensearch-project#1024)

* Address previous PR comments and implement rudimentary delete

Signed-off-by: Derek Ho <dxho@amazon.com>

* Implement modal and instant delete showing up in list

Signed-off-by: Derek Ho <dxho@amazon.com>

* Refactor save or cancel to a shared component, implement hard coded datasource configurations tab

Signed-off-by: Derek Ho <dxho@amazon.com>

* Update test with mock role data

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add most functionality of edit connectiondetails

Signed-off-by: Derek Ho <dxho@amazon.com>

* Address PR comments

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>

* Get most of the workflow working for configure datasource

Signed-off-by: Derek Ho <dxho@amazon.com>

* remove acceleration components

Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>

* Update data connection to datasources in a few more places and update tests

Signed-off-by: Derek Ho <dxho@amazon.com>

* Rename and re-organize in folders

Signed-off-by: Derek Ho <dxho@amazon.com>

* Fix import path in testing file

Signed-off-by: Derek Ho <dxho@amazon.com>

* Get Initial S3 configuration working

Signed-off-by: Derek Ho <dxho@amazon.com>

* Configure S3 Data Source Working

Signed-off-by: Derek Ho <dxho@amazon.com>

* Fix query permissions display

Signed-off-by: Derek Ho <dxho@amazon.com>

* Fix merge conflict

Signed-off-by: Derek Ho <dxho@amazon.com>

* Clean up PR

Signed-off-by: Derek Ho <dxho@amazon.com>

* Add s3 logo in manage table

Signed-off-by: Derek Ho <dxho@amazon.com>

* Fix up PR according to UX feedback

Signed-off-by: Derek Ho <dxho@amazon.com>

* Remove successfully

Signed-off-by: Derek Ho <dxho@amazon.com>

* Update test

Signed-off-by: Derek Ho <dxho@amazon.com>

* Address PR comments

Signed-off-by: Derek Ho <dxho@amazon.com>

* Update test in accordance to useLocation

Signed-off-by: Derek Ho <dxho@amazon.com>

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Co-authored-by: Shenoy Pratik <sgguruda@amazon.com>
pjfitzgibbons added a commit that referenced this pull request Oct 5, 2023
* Update mocha (#960)
* added code for the change of design of the sidebar, fixed overriding … (#928)
* Refactor add_integration_flyout to break out nested methods (#956)
* [Log Explorer] Remove top level tabs (#970)
* Explorer's timeheader & color change (#959)
* Update caching (#965)
* Move query_utils to /public (#983)
* Remove Zeppelin Parser from Notebook (#985)
* Explorer empty bucket (#990)
* refactored classnames for sidebar (#933)
* Refactor Integrations Backend to abstract IO (#947)
* Create React component for Integrations Setup UI (#1009)
* Create Data sources plugin with Manage Datasources Flow  (#1035)
* Manage datasources (#967)
* Add acceleration management UI  (#989)
* Rename data sources to data connections (#1004)
* Add fallback to show if user does not have datasource API permissions (#1008)
* Add access control tab content (#992)
* Delete datasource and Connection Configuration Tab (#1024)
* adjust explorer chart color and spacing (#1051)
* Configure S3 datasource flow (#1049)
* Manage datasources (#967)
* Add acceleration management UI  (#989)
* Rename data sources to data connections (#1004)
* Add fallback to show if user does not have datasource API permissions (#1008)
* Add access control tab content (#992)
* Delete datasource and Connection Configuration Tab (#1024)
* modified explorer data grid to follow discover look and feel (#1041)
* Create prometheus datasource flow (#1054)
* Manage datasources (#967)
* Add acceleration management UI  (#989)
* Rename data sources to data connections (#1004)
* Add fallback to show if user does not have datasource API permissions (#1008)
* Add access control tab content (#992)
* Delete datasource and Connection Configuration Tab (#1024)
* added design changes for sidebar (#1061)
* updated snapshots (#1062)
* Update UI for integrations setup (#1052)
* Data sources UI improvements (#1059)
* adding redirection to datasources (#1063)
* Move convertDateTime, refactor query_utils (#1064)
* Setup S3 connection with integrations (#1057)
* Use approved svg from UX (#1066)
* Fix missing import 'moment' on query_utils. (#1067)

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: sumukhswamy <sumukhhs@amazon.com>
Signed-off-by: Eric Wei <menwe@amazon.com>
Signed-off-by: Paul Sebastian <paulstn@amazon.com>
Signed-off-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@gmail.com>
Co-authored-by: sumukhswamy <sumukhhs@amazon.com>
Co-authored-by: Eric Wei <menwe@amazon.com>
Co-authored-by: Paul Sebastian <paul_sebastian@live.com>
Co-authored-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
Co-authored-by: Paul Sebastian <paulstn@amazon.com>
Co-authored-by: Shenoy Pratik <sgguruda@amazon.com>
Co-authored-by: Simeon Widdis <sawiddis@amazon.com>
amsiglan pushed a commit to amsiglan/dashboards-observability that referenced this pull request Jun 7, 2024
* Update mocha (opensearch-project#960)
* added code for the change of design of the sidebar, fixed overriding … (opensearch-project#928)
* Refactor add_integration_flyout to break out nested methods (opensearch-project#956)
* [Log Explorer] Remove top level tabs (opensearch-project#970)
* Explorer's timeheader & color change (opensearch-project#959)
* Update caching (opensearch-project#965)
* Move query_utils to /public (opensearch-project#983)
* Remove Zeppelin Parser from Notebook (opensearch-project#985)
* Explorer empty bucket (opensearch-project#990)
* refactored classnames for sidebar (opensearch-project#933)
* Refactor Integrations Backend to abstract IO (opensearch-project#947)
* Create React component for Integrations Setup UI (opensearch-project#1009)
* Create Data sources plugin with Manage Datasources Flow  (opensearch-project#1035)
* Manage datasources (opensearch-project#967)
* Add acceleration management UI  (opensearch-project#989)
* Rename data sources to data connections (opensearch-project#1004)
* Add fallback to show if user does not have datasource API permissions (opensearch-project#1008)
* Add access control tab content (opensearch-project#992)
* Delete datasource and Connection Configuration Tab (opensearch-project#1024)
* adjust explorer chart color and spacing (opensearch-project#1051)
* Configure S3 datasource flow (opensearch-project#1049)
* Manage datasources (opensearch-project#967)
* Add acceleration management UI  (opensearch-project#989)
* Rename data sources to data connections (opensearch-project#1004)
* Add fallback to show if user does not have datasource API permissions (opensearch-project#1008)
* Add access control tab content (opensearch-project#992)
* Delete datasource and Connection Configuration Tab (opensearch-project#1024)
* modified explorer data grid to follow discover look and feel (opensearch-project#1041)
* Create prometheus datasource flow (opensearch-project#1054)
* Manage datasources (opensearch-project#967)
* Add acceleration management UI  (opensearch-project#989)
* Rename data sources to data connections (opensearch-project#1004)
* Add fallback to show if user does not have datasource API permissions (opensearch-project#1008)
* Add access control tab content (opensearch-project#992)
* Delete datasource and Connection Configuration Tab (opensearch-project#1024)
* added design changes for sidebar (opensearch-project#1061)
* updated snapshots (opensearch-project#1062)
* Update UI for integrations setup (opensearch-project#1052)
* Data sources UI improvements (opensearch-project#1059)
* adding redirection to datasources (opensearch-project#1063)
* Move convertDateTime, refactor query_utils (opensearch-project#1064)
* Setup S3 connection with integrations (opensearch-project#1057)
* Use approved svg from UX (opensearch-project#1066)
* Fix missing import 'moment' on query_utils. (opensearch-project#1067)

---------

Signed-off-by: Derek Ho <dxho@amazon.com>
Signed-off-by: sumukhswamy <sumukhhs@amazon.com>
Signed-off-by: Eric Wei <menwe@amazon.com>
Signed-off-by: Paul Sebastian <paulstn@amazon.com>
Signed-off-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
Signed-off-by: Shenoy Pratik <sgguruda@amazon.com>
Signed-off-by: Simeon Widdis <sawiddis@gmail.com>
Co-authored-by: sumukhswamy <sumukhhs@amazon.com>
Co-authored-by: Eric Wei <menwe@amazon.com>
Co-authored-by: Paul Sebastian <paul_sebastian@live.com>
Co-authored-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
Co-authored-by: Paul Sebastian <paulstn@amazon.com>
Co-authored-by: Shenoy Pratik <sgguruda@amazon.com>
Co-authored-by: Simeon Widdis <sawiddis@amazon.com>
(cherry picked from commit 171acde)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants