Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate legacy maps service to NP & update refs #60942

Merged
merged 36 commits into from
Apr 13, 2020

Conversation

kindsun
Copy link
Contributor

@kindsun kindsun commented Mar 23, 2020

Migrates the legacy maps service from an angular service in ui/vis to the maps_legacy plugin. This service is a dependency of the Coordinate, Region and Vega maps plugins. This PR accomplishes the following

  • Moves the legacy code to the NP, it can be leveraged by plugins as mapsLegacy
  • Deangularizes service_settings
  • Updates references to the code in Region, Coordinate, Vega Maps
  • Updates dependencies, initialized in a local kibana_services file similar to how new Maps is done

A few things to note:

  • Where needed, TypeScript definitions were used/updated but it wasn't a primary focus of this PR since this is blocking other work on the other legacy maps plugins
  • kibana_map is very nearly a static file that can be provided without any external kibana services, with the exception the notification.toast service leveraged in map_messages. I attempted to supply new instances of kibana_map via a factory function in the plugin > setup method but ran into issues with the Leaflet Draw control that appear to be unrelated to the code changes, and more related to providing Leaflet via an Angular-based pipeline. There were no easy workarounds to make this work as we'd like. For this reason, kibana_map is served as a static file and leveraging plugins can optionally inject the toast service. Since we're only creating KibanaMap instances in 2 places (vega_map_view, base_maps_visualization) and there are no plans to leverage our legacy maps elsewhere, I believe this is the right solution for now.
  • getZoomPrecision and getPrecision were broken out into the separate file precision.ts since they were some of the few functions requiring kibana services

Since the legacy maps visualizations are hidden by default in Kibana versions Basic and above (although the sample data includes Region Maps) the best way to test this out is to use Kibana OSS:

  • Run ES in oss: yarn es snapshot --license oss
  • Run Kibana in oss (there are a few different ways to write this): node scripts/kibana --dev --oss

@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-gis (Team:Geo)

@kindsun kindsun force-pushed the np-update-and-migrate-legacy-maps branch from c7c88cf to a0307fc Compare March 26, 2020 18:28
…caldwell/kibana into np-update-and-migrate-legacy-maps
@kindsun
Copy link
Contributor Author

kindsun commented Apr 6, 2020

retest

Copy link
Contributor

@thomasneirynck thomasneirynck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

I think there's two things to look at:

  1. as discussed in person, there seems to be a regression in the display of attribution

Note that this is a limitation of Leaflet-attribution control and requires the raw HTML to display as-is. This seems to escape it first.

image

  1. the x-pack tilemap plugin (kibana/x-pack/legacy/plugins/tilemap) no longer extends the URL-params of the service-settings. This plugin's only purpose is to add a license-key to the URL-params using the ServiceSettings#addQueryParams method (
    serviceSettings.addQueryParams({ license: tileMapPluginInfo.license.uid });
    serviceSettings.disableZoomMessage();
    ). This plugin likely will need to be migrated as well in this PR to correctly pull the dependency.

@kindsun
Copy link
Contributor Author

kindsun commented Apr 9, 2020

  1. as discussed in person, there seems to be a regression in the display of attribution

As we later discussed, I removed sanitation. Since we're in control of the ems-client lib and the calling code, there appear to be no significant associated risks.

  1. the x-pack tilemap plugin (kibana/x-pack/legacy/plugins/tilemap) no longer extends the URL-params of the service-settings. This plugin's only purpose is to add a license-key to the URL-params using the ServiceSettings#addQueryParams method (
    serviceSettings.addQueryParams({ license: tileMapPluginInfo.license.uid });
    serviceSettings.disableZoomMessage();

    ). This plugin likely will need to be migrated as well in this PR to correctly pull the dependency.

This plugin should be migrated but we should do that in a separate PR. I've updated #60325 to account for this. I experimented with handling licensing logic in MapsLegacy directly. This seemed to work in Basic but failed in OSS since there's no way to specify a conditional dependency like "only depend on licensing if x-pack is available" in an NP plugin's kibana.json. For now I was able to fix it by obtaining a reference to MapsLegacy via an npSetup import, this works as expected in both OSS and Basic+ and is a good fix until we migrate it.

Copy link
Contributor

@miukimiu miukimiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few SCSS files were moved to a different folder. The changes LGMT! 🎉

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@thomasneirynck thomasneirynck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested both region and coordinate maps and works!

wrt sanitize removal, is there a possible equivalent already available? (otherwise consider using a regex to parse out href and contents and sanitize separately).

wrt

This plugin should be migrated but we should do that in a separate PR. I've updated #60325 to account for this. I experimented with handling licensing logic in MapsLegacy directly. This seemed to work in Basic but failed in OSS since there's no way to specify a conditional dependency like "only depend on licensing if x-pack is available" in an NP plugin's kibana.json. For now I was able to fix it by obtaining a reference to MapsLegacy via an npSetup import, this works as expected in both OSS and Basic+ and is a good fix until we migrate it.

+1 on doing this in separate PR.

I think we discussed doing this in the Maps plugin. Maps is loaded in basic+ anyway. It'd bloat the purpose of the Maps-plugin a little bit, but a fair trade-off imho if it can remove an entire plugin.

@@ -319,8 +319,6 @@
"common.ui.stateManagement.unableToStoreHistoryInSessionErrorMessage": "セッションがいっぱいで安全に削除できるアイテムが見つからないため、Kibana は履歴アイテムを保存できません。\n\nこれは大抵新規タブに移動することで解決されますが、より大きな問題が原因である可能性もあります。このメッセージが定期的に表示される場合は、{gitHubIssuesUrl} で問題を報告してください。",
"common.ui.url.replacementFailedErrorMessage": "置換に失敗、未解決の表現式: {expr}",
"common.ui.url.savedObjectIsMissingNotificationMessage": "保存されたオブジェクトがありません",
"common.ui.vis.kibanaMap.leaflet.fitDataBoundsAriaLabel": "データバウンドを合わせる",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are these gone?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran both an i18n check w/ fix flag and an extract. Neither of these indicated anything missing or out of place wrt maps_legacy. i18n did complain a few commits back about adding the new plugin directory, which I did here. It's likely that some translations require manual translation and are flagged in a separate documentation process. Unfortunately it's not as easy as just updating the dot path to its new location since this would just be wiped out by the next node scripts/i18n_extract command someone runs. I did notice when running node scripts/i18n_extract that both of these paths were correctly grabbed for the english (untracked) file here:

image

So they are tracked correctly and should be translated in a later process

@kindsun
Copy link
Contributor Author

kindsun commented Apr 13, 2020

I think we discussed doing this in the Maps plugin. Maps is loaded in basic+ anyway. It'd bloat the purpose of the Maps-plugin a little bit, but a fair trade-off imho if it can remove an entire plugin.

We did indeed. I keep going back and forth on this but I'm (at least currently) in favor of keeping it as a separate plugin providing data to this new maps_legacy plugin. It feels cleaner to me. That idea needs more fleshing out though since it could have the same issue as trying to handle licensing directly in this plugin (i.e.- works in basic, doesn't work sans x-pack in oss). Definitely happy to chat further separately on this in the context of a future PR!

@kindsun
Copy link
Contributor Author

kindsun commented Apr 13, 2020

wrt sanitize removal, is there a possible equivalent already available? (otherwise consider using a regex to parse out href and contents and sanitize separately).

As discussed offline, tracking this here: #63399

@kindsun kindsun merged commit 6f46e6b into elastic:master Apr 13, 2020
@kindsun kindsun deleted the np-update-and-migrate-legacy-maps branch April 13, 2020 20:32
kindsun pushed a commit to kindsun/kibana that referenced this pull request Apr 13, 2020
* Get legacy maps into basic NP plugin form. Swap out legacy services for NP

* Pull service settings from injected vars. Return new instance of service settings from setup

* Use NP service settings in vega maps. Clean up legacy shim

* Use NP maps service in region maps. Clean up shim. Add exports to NP service

* Pull zoom precision out to separate module since it's dependent on config

* Update paths to point to NP resources

* More path updates, clean up, use IServiceSettings

* Remove dependency on legacy service settings. Add tile_map dependency ng-sanitize

* More path cleanup. Use zoomPrecision provided through plugin inteface

* Move getPrecision into contract-provided function since it depends on config

* Move tests to new service location, remove vis dir

* Update test paths. Move origin const declaration into public

* Clean up, fixes

* Fix type errors. General cleaning

* Inject toast service into map when needed

* Fix typo in regionmap config

* i18n fixes

* Update jest test path

* Fix karma tests

* i18n fixes

* Type updates. Add mapsLegacy to np karma mock

* Remove html sanitizer

* Fix vega test that depends on serviceSettings

* Revise xpack license handling to use NP serviceSettings. Remove angular bindings

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
gmmorris added a commit to gmmorris/kibana that referenced this pull request Apr 14, 2020
* master: (132 commits)
  document code splitting for client code (elastic#62593)
  Escape single quotes surrounded by double quotes (elastic#63229)
  [Endpoint] Update cli mapping to match endpoint package (elastic#63372)
  update in-app links to metricbeat configuration docs (elastic#63295)
  investigation notes field (documentation / metadata) (elastic#63386)
  [Maps] fix bug where toggling Scaling type does not re-fetch data (elastic#63326)
  [Alerting] set correct parameter for unauthented email action (elastic#63086)
  [Telemetry] force staging urls in tests (elastic#63356)
  Migrate legacy maps service to NP & update refs (elastic#60942)
  Fix task manager query to return tasks to retry (elastic#63360)
  [Endpoint] Policy list support for URL pagination state (elastic#63291)
  [Canvas] Migrate saved object mappings and migrations to Kibana Platform (elastic#58891)
  [DOCS] Add ILM tutorial (elastic#59502)
  [Maps] Add SOURCE_TYPES enumeration (elastic#62975)
  [Maps] update geospatial filters to use geo_shape query for geo_point fields (elastic#62966)
  Move away from npStart for embeddables in canvas (elastic#62680)
  Use MapInput type from Maps plugin (elastic#61539)
  Update to pagination for workpad and templates (elastic#62050)
  [SIEM] Fix AlertsTable id (elastic#63368)
  Consistent terminology around cypress test data (elastic#63279)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Apr 14, 2020
* master:
  document code splitting for client code (elastic#62593)
  Escape single quotes surrounded by double quotes (elastic#63229)
  [Endpoint] Update cli mapping to match endpoint package (elastic#63372)
  update in-app links to metricbeat configuration docs (elastic#63295)
  investigation notes field (documentation / metadata) (elastic#63386)
  [Maps] fix bug where toggling Scaling type does not re-fetch data (elastic#63326)
  [Alerting] set correct parameter for unauthented email action (elastic#63086)
  [Telemetry] force staging urls in tests (elastic#63356)
  Migrate legacy maps service to NP & update refs (elastic#60942)
  Fix task manager query to return tasks to retry (elastic#63360)
  [Endpoint] Policy list support for URL pagination state (elastic#63291)
  [Canvas] Migrate saved object mappings and migrations to Kibana Platform (elastic#58891)
  [DOCS] Add ILM tutorial (elastic#59502)
  [Maps] Add SOURCE_TYPES enumeration (elastic#62975)
  [Maps] update geospatial filters to use geo_shape query for geo_point fields (elastic#62966)
  Move away from npStart for embeddables in canvas (elastic#62680)
kindsun pushed a commit that referenced this pull request Apr 14, 2020
* Get legacy maps into basic NP plugin form. Swap out legacy services for NP

* Pull service settings from injected vars. Return new instance of service settings from setup

* Use NP service settings in vega maps. Clean up legacy shim

* Use NP maps service in region maps. Clean up shim. Add exports to NP service

* Pull zoom precision out to separate module since it's dependent on config

* Update paths to point to NP resources

* More path updates, clean up, use IServiceSettings

* Remove dependency on legacy service settings. Add tile_map dependency ng-sanitize

* More path cleanup. Use zoomPrecision provided through plugin inteface

* Move getPrecision into contract-provided function since it depends on config

* Move tests to new service location, remove vis dir

* Update test paths. Move origin const declaration into public

* Clean up, fixes

* Fix type errors. General cleaning

* Inject toast service into map when needed

* Fix typo in regionmap config

* i18n fixes

* Update jest test path

* Fix karma tests

* i18n fixes

* Type updates. Add mapsLegacy to np karma mock

* Remove html sanitizer

* Fix vega test that depends on serviceSettings

* Revise xpack license handling to use NP serviceSettings. Remove angular bindings

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@flash1293 flash1293 mentioned this pull request Apr 14, 2020
69 tasks
wayneseymour pushed a commit that referenced this pull request Apr 15, 2020
* Get legacy maps into basic NP plugin form. Swap out legacy services for NP

* Pull service settings from injected vars. Return new instance of service settings from setup

* Use NP service settings in vega maps. Clean up legacy shim

* Use NP maps service in region maps. Clean up shim. Add exports to NP service

* Pull zoom precision out to separate module since it's dependent on config

* Update paths to point to NP resources

* More path updates, clean up, use IServiceSettings

* Remove dependency on legacy service settings. Add tile_map dependency ng-sanitize

* More path cleanup. Use zoomPrecision provided through plugin inteface

* Move getPrecision into contract-provided function since it depends on config

* Move tests to new service location, remove vis dir

* Update test paths. Move origin const declaration into public

* Clean up, fixes

* Fix type errors. General cleaning

* Inject toast service into map when needed

* Fix typo in regionmap config

* i18n fixes

* Update jest test path

* Fix karma tests

* i18n fixes

* Type updates. Add mapsLegacy to np karma mock

* Remove html sanitizer

* Fix vega test that depends on serviceSettings

* Revise xpack license handling to use NP serviceSettings. Remove angular bindings

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
tsullivan pushed a commit to tsullivan/kibana that referenced this pull request Apr 15, 2020
…elastic#63407)

* Get legacy maps into basic NP plugin form. Swap out legacy services for NP

* Pull service settings from injected vars. Return new instance of service settings from setup

* Use NP service settings in vega maps. Clean up legacy shim

* Use NP maps service in region maps. Clean up shim. Add exports to NP service

* Pull zoom precision out to separate module since it's dependent on config

* Update paths to point to NP resources

* More path updates, clean up, use IServiceSettings

* Remove dependency on legacy service settings. Add tile_map dependency ng-sanitize

* More path cleanup. Use zoomPrecision provided through plugin inteface

* Move getPrecision into contract-provided function since it depends on config

* Move tests to new service location, remove vis dir

* Update test paths. Move origin const declaration into public

* Clean up, fixes

* Fix type errors. General cleaning

* Inject toast service into map when needed

* Fix typo in regionmap config

* i18n fixes

* Update jest test path

* Fix karma tests

* i18n fixes

* Type updates. Add mapsLegacy to np karma mock

* Remove html sanitizer

* Fix vega test that depends on serviceSettings

* Revise xpack license handling to use NP serviceSettings. Remove angular bindings

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation Feature:Coordinate Map Feature:NP Migration Feature:Region Map Feature:Vega Vega visualizations release_note:skip Skip the PR/issue when compiling release notes v7.8.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants