[Fleet, Enterprise Search] Integration links lead to "Unable to connect" page if Enterprise Search backend is not running

[yakhinvadim]

Deployments might have Enterprise Search backend not running. In that case, clicking on Enterprise Search cards in the unified integrations view will lead to a generic "Unable to connect" page.

Screen.Cast.2021-10-15.at.12.25.47.PM.mp4

What is the expected behaviour in these cases?

cc: @alexfrancoeur @mostlyjason @bhuvanaurora @jbynum

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[elasticmachine]

Pinging @elastic/workplace-search-frontend (Team:WorkplaceSearch)

[yakhinvadim]

@zumwalt Letting you know about this issue as well, because it also affects all App Search integrations.

[alexfrancoeur]

Thanks for opening @yakhinvadim. Adding @snide @dborodyansky @zumwalt @bhuvanaurora @clintandrewhall and @ryankeairns to participate in the discussion as well. My suggestion below is based off of a few assumptions. If they are wrong, please correct me.

1. We want to provide hard coded, promoted integrations on this view with web crawler being the most appropriate for the Enterprise Search solution.
2. In Cloud, this is much less of a problem for trials or Enterprise Search users. Enterprise Search is enabled by default.
3. In Self Managed use cases for folks interested in custom search, this will probably be more common of an experience. We know the web crawler is a quick and easy way to get data in. With the introduction of these cards (promoted or not), this can impact our "first impression" for net new self-managed clusters.
4. The permission model is as follows. For folks that do not have ingest permissions, these cards will be discoverable but not clickable. If "Fleet & Integrations" is disabled, users will not have this experience. For mature clusters set up with the appropriate RBAC, this is likely a non-issue.

Given that we'd like to promote the web crawler at the top of the integrations page and it is hard coded, it would be nice to provide a better actionable empty state that is on part with the "no data" empty states we are adding to solutions and analytics apps. Realizing that we only have a day between now and feature freeze, this feels like an impossible task. But I'll ask, are there any quick ways we could clean up this empty state? And if so, is this backportable post-FF? I think the details provided are actionable and good enough for an MVP, but any improvements we can make here would be great if possible.

[jonasll]

A few thoughts:

1. This screen will appear on 2 main occasions: (i) on Elastic Cloud, after a cluster has made the decision to autoscale down (i.e. shut down) an Enterprise Search node, and (ii) for self-managed deployments when a new user is exploring and has no context as to what Enterprise Search is, and why it should be running independently.
2. For item (i), this is being solved with better error messaging and a Cloud-specific guide as implemented here.
3. For item (ii), I believe our improved messaging (as stated in the previous point) will give clear actionability to the end-user when reviewing the Setup Guide, as the main exit point.

In other words, in the short term, I believe that we are in a reasonable position. That said, in the longer term, I do believe that the Integrations view could be more aware of the status of the active processes that may be dependencies by still showing the integration, but clicking them may trigger a modal or flyover prompting you to first set up Enterprise Search. I do not believe this to be within the scope of this particular issue, as I assume all Solutions teams will want to participate in this conversation.

I believe this to be a no-op for the time being, but am open to discussing more with @alexfrancoeur.

PS. It is worth noting that the upcoming re-architecture of the Enterprise Search binary may render this conversation irrelevant in a few/several months down the line, as it may not require a separate process on a separate box to operate.

[alexfrancoeur]

Thanks for the reply @jonasll! Adding thoughts and comments inline below.

In other words, in the short term, I believe that we are in a reasonable position.

If you're comfortable with the way this is presented for an onboarding use case, then I think we can pause on this and de-prioritize. Brainstorming, I believe there is room for some small improvements to this UX. Today we show a big red exclamation point and the CTA is go to the docs. Visually, we can probably make this more user friendly. Functionally, we have capabilities we can leverage in Kibana. Kibana is Cloud aware, so we if we wanted to link back to the deployment details for an admin on Cloud - we could. We may also be able to provide a more step-by-step guidance in-product vs. pointing to docs for self managed environments. This all requires work and time so if you and the team feel that this is experience is sufficient enough for onboarding (i) and (ii), I won't push on it. It may be worth revisiting in the next few months as we learn more through telemetry.

That said, in the longer term, I do believe that the Integrations view could be more aware of the status

Big 👍 , I'll make sure we have an open issue to track this.

PS. It is worth noting that the upcoming re-architecture of the Enterprise Search binary may render this conversation irrelevant in a few/several months down the line, as it may not require a separate process on a separate box to operate.

If there's some additional context available, I'd love to learn more.

cc: @LL6688

[jonasll]

@alexfrancoeur That context is being laid out as we speak, please stay posted early in CY22.