[FEATURE] migration guide from spring-data-elasticsearch to spring-data-opensearch

[rursprung]

Is your feature request related to a problem?

all users were so far stuck on spring-data-elasticsearch (incompatible with OpenSearch 2.x) and will now want to switch to spring-data-opensearch (once available).

What solution would you like?

please provide a clear step-by-step migration guide on how to migrate an existing project from spring-data-elasticsearch to spring-data-opensearch.
if it is supported (cf. also #12) also mention how a project can support both Elasticsearch and OpenSearch in parallel (only ES 7.10.2 & older; maybe even all 7.x and maybe there's even a way for a project to support OpenSearch & ES 8.x in parallel, e.g. by picking the library at runtime?)

What alternatives have you considered?

everyone can figure it out on their own, leading to greater variation in how it's implemented and lots of questions around it.

Do you have any additional context?

n/a

[dblock]

@rursprung It's a good idea. Any interest in contributing? I think we should have USER_GUIDE ala https://github.com/opensearch-project/opensearch-js/blob/main/USER_GUIDE.md, and either make migration a section of it, or a separate MIGRATION_GUIDE.md or UPGRADING.md.

[Neuw84]

Hi @dblock,

I could contribute if needed :).

Moreover, I have a project https://github.com/aws-samples/opensearch-bootful with Elastic Libraries that I will upgrade with OpenSearch libraries ( will document the process).

Let me know!

[dblock]

@Neuw84 Amazing! Please do. Also note that we have made a lot of progress on https://github.com/opensearch-project/opensearch-java, so depending on what your sample does use that if you need a client.

[wbeckler]

@Neuw84 The client has made big strides, now includes sigv4 and is on track for Maven soon. What's your thinking on the documentation?

[Neuw84]

Hi!

As soon as we have it on Maven will work on it. I tried to upgrade everything like three weeks ago but I got stuck on the manually install the packages ( I know the procedure but I though that if was better to wait).

When do you think that is going to be available on Maven? :)

Bests!

[rursprung]

When do you think that is going to be available on Maven? :)

in -2 weeks 😆: https://repo1.maven.org/maven2/org/opensearch/client/
(there's also a tag for it in git, but it hasn't been made an official GitHub release yet, so it's not that clearly visible - @reta maybe you'd want to turn that tag into a GitHub Release so it's more prominent?)

[Neuw84]

Great! Will work on it when I have some bandwidth! El jue, 19 ene 2023 a las 18:38, Ralph Ursprung ***@***.***>) escribió:

…

When do you think that is going to be available on Maven? :) in -2 weeks 😆: https://repo1.maven.org/maven2/org/opensearch/client/ (there's also a tag for it in git, but it hasn't been made an official GitHub release yet, so it's not that clearly visible - @reta <https://github.com/reta> maybe you'd want to turn that tag into a GitHub Release so it's more prominent?) — Reply to this email directly, view it on GitHub <#13 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNFQATQ7MXJD2IZFIO66NLWTF33BANCNFSM6AAAAAARBP3Z6Q> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[reta]

When do you think that is going to be available on Maven? :)

in -2 weeks 😆: https://repo1.maven.org/maven2/org/opensearch/client/ (there's also a tag for it in git, but it hasn't been made an official GitHub release yet, so it's not that clearly visible - @reta maybe you'd want to turn that tag into a GitHub Release so it's more prominent?)

Thanks a lot for cheerful comments @rursprung ! The 0.1.0 is a pre-release, there is a one minor issue to address with test artifact name but .... things turned out to be more complicated [1], [2] so I don't know when we could have an official release :)

[1] #75
[2] #87

[Neuw84]

Well, will wait to that security review and the proceed.

My idea is to rebase my repo on top on SpringBoot 3.0 and migrate all the libraries to OpenSearch ones ( and use that process for the guide).

[Neuw84]

Hi @reta,

Seems that both issues and security review has finished.

Will work on this issue as soon as I can 👍

[reta]

Hey @Neuw84 , thanks a lot!

[Neuw84]

Hi,

I have been upgrading the code to SpringBoot 3.0.

Some comments ( maybe a little guidance will be great). If I use password based auth I got it working. The only problem that I have is that I get node sniffer problems

2023-03-22T13:30:31.894+01:00 ERROR 38719 --- [nt_sniffer[T#1]] org.opensearch.client.sniff.Sniffer      : error while sniffing nodes
java.net.ConnectException: Connection refused

My client is configured this way

        final ClientConfiguration clientConfiguration = ClientConfiguration.builder()
                .connectedTo(endpoint)
                .usingSsl()
                .withBasicAuth("user","pass")
                .withConnectTimeout(Duration.ofSeconds(10))
                .withSocketTimeout(Duration.ofSeconds(5))
                .build();

I suppose that is related to this issue?

opensearch-project/OpenSearch#3487

[reta]

@Neuw84 I am curious how to do run the OpenSearch server? locally / managed / Docker?

[Neuw84]

@reta ,

Managed on AWS. Maybe is because it can´t communicate with the nodes directly? (Amazon OpenSearch Service just exposes an endpoint).

Anyway, I have upgraded the code to use OpenSearch libraries and will start working on the migration guide (I also have working code with Elastic libraries).

https://github.com/aws-samples/opensearch-bootful

Will update this post!

[reta]

@Neuw84

Managed on AWS. Maybe is because it can´t communicate with the nodes directly? (Amazon OpenSearch Service just exposes an endpoint).

Exactly, I believe sniffer gets internal IPs which are not accessible.

Anyway, I have upgraded the code to use OpenSearch libraries and will start working on the migration guide (I also have working code with Elastic libraries).

Thank you very much, super helpful

[navaneeth-spotnana]

@reta I am also facing the same issue with AWS Managed Elasticsearch. Is it possible to disable the sniffer? Is that a good idea?

[reta]

@navaneeth-spotnana

Is it possible to disable the sniffer?

Yes, you just need to remove the pensearch-rest-client-sniffer dependency

Is that a good idea?

I mean in this case it is not useful and only contributes to error logging, so yeah, it is good idea (in this context)

[N4zroth]

When migrating, am I supposed to continue using the (now deprecated) ElasticsearchRestTemplate or should I switch to OpenSearchRestTemplate?

[reta]

When migrating, am I supposed to continue using the (now deprecated) ElasticsearchRestTemplate or should I switch to OpenSearchRestTemplate?

The OpenSearchRestTemplate is the right one