Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[APM] Handle read-after-write #155330

Closed
1 of 2 tasks
sorenlouv opened this issue Apr 19, 2023 · 6 comments · Fixed by #159674
Closed
1 of 2 tasks

[APM] Handle read-after-write #155330

sorenlouv opened this issue Apr 19, 2023 · 6 comments · Fixed by #159674
Assignees
@sorenlouv @MiriamAparicio
Labels
8.10 candidate apm:release-feature APM UI - Release Feature Goal apm:serverless discuss Team:APM All issues that need APM UI Team support

Comments

@sorenlouv
Copy link
Member

sorenlouv commented Apr 19, 2023 •
edited by MiriamAparicio
Loading

Background:

To reduce infrastructure costs in Serverless, the Elasticsearch team proposed increasing the stateless refresh interval to 5 seconds compared to “classic” Elasticsearch’s 1 second refreshes.
The recommendation in Kibana is to use refresh=wait_for for all writes to ensure search consistency. E.g. if a user deletes a dashboard and then goes to the dashboard listing page, their deleted dashboard shouldn’t show up in the search results. But this means that writes on stateless would take up to 5 seconds to complete – a user experience we deemed unacceptable.

Action items:

  • Apply index.fast_refresh: true to .apm-custom-link (#159674)
  • Apply refresh: true when creating API keys

Areas affected by read-after-write:

  • Service groups
  • (APM) Rules
  • Settings:
    • General settings
    • Custom links (should be hidden)
    • Agent keys
    • ML Jobs

Related:
@sorenlouv sorenlouv added Team:APM All issues that need APM UI Team support 8.9 candidate apm:serverless labels Apr 19, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/apm-ui (Team:APM)

@achyutjhunjhunwala
Copy link
Contributor

Should we also document places where we are writing data to ES ?

@sorenlouv
Copy link
Member Author

@achyutjhunjhunwala Good point! I've added a list of the areas affected by this. Please add to the list if you can think of other areas affected.

@gbamparop gbamparop added the apm:release-feature APM UI - Release Feature Goal label May 12, 2023
@gbamparop
Copy link
Contributor

Service groups and general settings are saved objects and will be using index.fast_refresh as an interim solution for the MVP. Shall we add this to custom links index as well?

@sorenlouv
Copy link
Member Author

Shall we add this to custom links index as well?

Yes. Alternatively we can disable custom links on serverless. I prefer whichever is easiest (I think that is applying index.fast_refresh)

@sorenlouv sorenlouv mentioned this issue Jun 14, 2023
Merged
@MiriamAparicio MiriamAparicio self-assigned this Jul 12, 2023
MiriamAparicio added a commit that referenced this issue Jul 14, 2023
@sorenlouv @MiriamAparicio
[APM] Add index.fast_refresh to .apm-custom-link (#159674)
efdc760 
Closes: #155330

Adds `index.fast_refresh` to `.apm-custom-link` in order to ensure fast
index refreshes on serverless (1 second periodic refreshes instead of 5
second which is the new default on serverless).

This is pending on Elasticsearch adding support for `index.fast_refresh`
(elastic/elasticsearch#96660)

---------

Co-authored-by: miriam.aparicio <miriam.aparicio@gmail.com>
Co-authored-by: Miriam <31922082+MiriamAparicio@users.noreply.github.com>
@MiriamAparicio
Copy link
Contributor

Apply refresh: true when creating API keys

We leave it for now, we don’t control how many times it will be called (it could be many, thus expensive).
OTOH without refresh=true it could be slow.
We’ll have to manually assess how bad could be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sorenlouv sorenlouv

@MiriamAparicio MiriamAparicio

Labels
8.10 candidate apm:release-feature APM UI - Release Feature Goal apm:serverless discuss Team:APM All issues that need APM UI Team support
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[APM] Add index.fast_refresh to .apm-custom-link sorenlouv/kibana
5 participants
@sorenlouv @gbamparop @achyutjhunjhunwala @elasticmachine @MiriamAparicio