Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Polluting log entry regarding destination migration #1183

Closed
spapadop opened this issue Sep 18, 2023 · 8 comments
Closed

[BUG] Polluting log entry regarding destination migration #1183

spapadop opened this issue Sep 18, 2023 · 8 comments
Labels
bug Something isn't working

Comments

@spapadop
Copy link

spapadop commented Sep 18, 2023

What is the bug?
There is a specific log that keeps on appearing quite frequently on all OpenSearch clusters (even fresh installations). While it does not somehow harm the cluster, its consistent presence is quite annoying:

[INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [my-node-name] Detected cluster change event for destination migration

I tracked this message on DestinationMigrationCoordinator.kt. I am not sure what is exactly wrong there, but something is.

There are couple of related threads already on the forums, like here and here.

And if you google "Detected cluster change event for destination migration", you'll see that it appears on many logs, for unrelated issues, so my guess is that it's just polluting logs.

How can one reproduce the bug?
Steps to reproduce the behavior:
Just bootstrap a new cluster and check the output logs.

What is the expected behavior?
Not to produce such a polluting log.

What is your host/environment?

  • OpenSearch: v2.9.0
  • OS: AlmaLinux 9.2
  • Plugins: all that come by default
@spapadop spapadop added bug Something isn't working untriaged labels Sep 18, 2023
@IAmFQQ
Copy link

IAmFQQ commented Oct 9, 2023

Yeah, I met the same issue, such logs are very annoying.
image

[2023-10-09T04:13:20,219][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [s-masterlvs-lvs-1] Detected cluster change event for destination migration
[2023-10-09T04:13:21,500][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [s-masterlvs-lvs-1] Detected cluster change event for destination migration
[2023-10-09T04:13:24,036][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [s-masterlvs-lvs-1] Detected cluster change event for destination migration

There are logs like this almost every few seconds, confused why alerting logs appeared in Opensearch...

@spapadop
Copy link
Author

spapadop commented Nov 1, 2023

still there in v2.11

@riysaxen-amzn
Copy link
Collaborator

closing in favor of #1356

@uranru
Copy link

uranru commented Apr 16, 2024

version 2.13
the problem persists

изображение

@bagipriyank
Copy link

i am seeing this is version 2.16.0

@Its-Ankush
Copy link

Can confirm i see this all the time [in OS_2.16]

[2024-09-20T09:23:18,647][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.
[2024-09-20T09:23:42,440][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.
[2024-09-20T09:23:43,819][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.
[2024-09-20T09:23:43,920][INFO ][o.o.a.u.d.DestinationMigrationCoordinator] [data_node_1] Cancelling the migration process.

@mousemaxx
Copy link

OS 2.17, messages still in log

@spapadop
Copy link
Author

@riysaxen-amzn could you please reopen this issue? Or shall we create a new one?
The original reported problem is not fixed by #1356

btw, there is a suggestion there: #1356 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

7 participants