Skip to content

Commit

Permalink
Add support to parse sub-aggregations from filter/nested aggregations (
Browse files Browse the repository at this point in the history
…opensearch-project#234)

* Add support to parse sub-aggregations from filter/nested aggregations

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Add USER_GUIDE.md

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Add DCO

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Add link to USER_GUIDE.md in README.md

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Add changelog and changelog verifier (opensearch-project#239)

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Set javadoc encoding to utf-8 (opensearch-project#50) (opensearch-project#241)

Signed-off-by: Meetesh Kumawat<kmeetesh@gmail.com>
Signed-off-by: meetesh <kmeetesh@gmail.com>

Signed-off-by: Meetesh Kumawat<kmeetesh@gmail.com>
Signed-off-by: meetesh <kmeetesh@gmail.com>
Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Update Jackson Databind to 2.13.4.2 (addressing CVE-2022-42003) (opensearch-project#240)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Updates changelog for dependabot PRs (opensearch-project#247)

* Updates changelog for dependabot PRs

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

* Adding dependabot label for workflow

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Update tests to use JUnit's Assert (opensearch-project#244)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Update literature around changelog (opensearch-project#242)

* Update literature

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

* Removing pr template and updating language

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Bump grgit-gradle from 4.0.1 to 5.0.0 (opensearch-project#243)

* Bump grgit-gradle from 4.0.1 to 5.0.0

Bumps [grgit-gradle](https://github.com/ajoberstar/grgit) from 4.0.1 to 5.0.0.
- [Release notes](https://github.com/ajoberstar/grgit/releases)
- [Commits](ajoberstar/grgit@4.0.1...5.0.0)

---
updated-dependencies:
- dependency-name: org.ajoberstar.grgit:grgit-gradle
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Add DCO

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Update CHANGELOG and README

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Add DCO

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Add DCO

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

* Fix CHANGLOG and README TOC

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>

Signed-off-by: Abhinav Nath <abhinavnath@ymail.com>
Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: Meetesh Kumawat<kmeetesh@gmail.com>
Signed-off-by: meetesh <kmeetesh@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Co-authored-by: Meetesh Kumawat <kmeetesh@gmail.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
  • Loading branch information
7 people authored Oct 21, 2022
1 parent 4a3be6f commit 1ac790a
Show file tree
Hide file tree
Showing 5 changed files with 214 additions and 5 deletions.
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Added
- Github workflow for changelog verification ([#239](https://github.com/opensearch-project/opensearch-java/pull/239))
- Github workflow for dependabot PRs ([#247](https://github.com/opensearch-project/opensearch-java/pull/247))

### Dependencies
- Bumps `grgit-gradle` from 4.0.1 to 5.0.0

### Changed
- Update literature around changelog contributions in CONTRIBUTING.md ([#242](https://github.com/opensearch-project/opensearch-java/pull/242))
- Update tests to use JUnit's Assert ([#244]https://github.com/opensearch-project/opensearch-java/pull/244)
- Update tests to use JUnit's Assert ([#244](https://github.com/opensearch-project/opensearch-java/pull/244))
- Add support to parse sub-aggregations from filter/nested aggregations ([#234](https://github.com/opensearch-project/opensearch-java/pull/234))

### Deprecated

Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
OpenSearch Java Client

- [Welcome!](#welcome)
- [Sample Code](#sample-code)
- [Project Resources](#project-resources)
- [Code of Conduct](#code-of-conduct)
- [Compatibility with OpenSearch](#compatibility-with-opensearch)
Expand All @@ -23,6 +24,9 @@ OpenSearch Java Client
For more information, see [opensearch.org](https://opensearch.org/).
This client is meant to replace the existing [OpenSearch Java High Level REST Client](https://opensearch.org/docs/latest/clients/java-rest-high-level/).

## Sample Code

Please see the [USER_GUIDE](USER_GUIDE.md) for code snippets.

## Project Resources

Expand Down
122 changes: 122 additions & 0 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# User Guide

- [User Guide](#user-guide)
- [Sample data](#sample-data)
- [Create an index](#create-an-index)
- [Index data](#index-data)
- [Search for the document](#search-for-the-document)
- [Search documents using a match query](#search-documents-using-a-match-query)
- [Aggregations](#aggregations)
- [Delete the document](#delete-the-document)
- [Delete the index](#delete-the-index)
- [Aggregations](#aggregations)

## Sample data

### IndexData class

```java
static class IndexData {
private String firstName;
private String lastName;

public IndexData(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}

public String getFirstName() {
return firstName;
}

public void setFirstName(String firstName) {
this.firstName = firstName;
}

public String getLastName() {
return lastName;
}

public void setLastName(String lastName) {
this.lastName = lastName;
}

@Override
public String toString() {
return String.format("IndexData{first name='%s', last name='%s'}", firstName, lastName);
}
}
```

## Create an index

```java
String index = "sample-index";
CreateIndexRequest createIndexRequest = new CreateIndexRequest.Builder().index(index).build();
client.indices().create(createIndexRequest);
```

## Index data

```java
IndexData indexData = new IndexData("John", "Doe");
IndexRequest<IndexData> indexRequest = new IndexRequest.Builder<IndexData>().index(index).id("1").document(indexData).build();
client.index(indexRequest);

indexData = new IndexData("John", "Joe");
indexRequest = new IndexRequest.Builder<IndexData>().index(index).id("2").document(indexData).build();
client.index(indexRequest);
```

## Search for the documents

```java
SearchResponse<IndexData> searchResponse = client.search(s -> s.index(index), IndexData.class);
for (int i = 0; i < searchResponse.hits().hits().size(); i++) {
System.out.println(searchResponse.hits().hits().get(i).source());
}
```

## Search documents using a match query

```java
SearchRequest searchRequest = new SearchRequest.Builder().query(q -> q.match(m -> m.field("firstName")
.query(FieldValue.of("John"))))
.build();

SearchResponse<IndexData> searchResponse = client.search(searchRequest, IndexData.class);
for (int i = 0; i < searchResponse.hits().hits().size(); i++) {
System.out.println(searchResponse.hits().hits().get(i).source());
}
```

## Aggregations

```java
SearchRequest searchRequest = new SearchRequest.Builder().query(q -> q.match(m -> m.field("firstName")
.query(FieldValue.of("John"))))
.aggregations("firstNames", new Aggregation.Builder().terms(t -> t.field("firstName.keyword"))
.build())
.build();

SearchResponse<IndexData> searchResponse = client.search(searchRequest, IndexData.class);
for (Map.Entry<String, Aggregate> entry : searchResponse.aggregations().entrySet()) {
System.out.println("Agg - " + entry.getKey());
entry.getValue().sterms().buckets().array().forEach(b -> System.out.printf("%s : %d%n", b.key(), b.docCount()));
}
```

## Delete the document

The following sample code deletes a document whose ID is 1.

```java
client.delete(d -> d.index(index).id("1"));
```

## Delete the index

```java
DeleteIndexRequest deleteIndexRequest = new DeleteRequest.Builder().index(index).build();
DeleteIndexResponse deleteIndexResponse = client.indices().delete(deleteIndexRequest);
```
Original file line number Diff line number Diff line change
Expand Up @@ -41,23 +41,35 @@
import org.opensearch.client.json.ObjectDeserializer;
import org.opensearch.client.util.ApiTypeHelper;
import jakarta.json.stream.JsonGenerator;
import org.opensearch.client.util.ObjectBuilder;

import javax.annotation.Nullable;
import java.util.HashMap;
import java.util.Map;
import java.util.function.Function;

// typedef: _types.aggregations.SingleBucketAggregateBase



public abstract class SingleBucketAggregateBase extends AggregateBase {
private final Map<String, Aggregate> aggregations;
private final long docCount;

// ---------------------------------------------------------------------------------------------

protected SingleBucketAggregateBase(AbstractBuilder<?> builder) {
super(builder);
this.aggregations = ApiTypeHelper.unmodifiable(builder.aggregations);

this.docCount = ApiTypeHelper.requireNonNull(builder.docCount, this, "docCount");

}

public final Map<String, Aggregate> aggregations() {
return this.aggregations;
}

/**
* Required - API name: {@code doc_count}
*/
Expand All @@ -76,11 +88,24 @@ protected void serializeInternal(JsonGenerator generator, JsonpMapper mapper) {
protected abstract static class AbstractBuilder<BuilderT extends AbstractBuilder<BuilderT>>
extends
AggregateBase.AbstractBuilder<BuilderT> {
@Nullable
protected Map<String, Aggregate> aggregations = new HashMap<>();
private Long docCount;

/**
* Required - API name: {@code doc_count}
*/
public final BuilderT aggregations(Map<String, Aggregate> aggregateMap) {
this.aggregations = _mapPutAll(this.aggregations, aggregateMap);
return self();
}

public final BuilderT aggregations(String key, Aggregate value) {
this.aggregations = _mapPut(this.aggregations, key, value);
return self();
}

public final BuilderT aggregations(String key, Function<Aggregate.Builder, ObjectBuilder<Aggregate>> function) {
return aggregations(key, function.apply(new Aggregate.Builder()).build());
}

public final BuilderT docCount(long value) {
this.docCount = value;
return self();
Expand All @@ -94,6 +119,12 @@ protected static <BuilderT extends AbstractBuilder<BuilderT>> void setupSingleBu
AggregateBase.setupAggregateBaseDeserializer(op);
op.add(AbstractBuilder::docCount, JsonpDeserializer.longDeserializer(), "doc_count");

op.setUnknownFieldHandler((builder, name, parser, mapper) -> {
if (builder.aggregations == null) {
builder.aggregations = new HashMap<>();
}
Aggregate._TYPED_KEYS_DESERIALIZER.deserializeEntry(name, parser, mapper, builder.aggregations);
});
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,15 @@
import org.junit.Test;
import org.opensearch.Version;
import org.opensearch.client.opensearch.OpenSearchAsyncClient;
import org.opensearch.client.opensearch._types.FieldValue;
import org.opensearch.client.opensearch._types.OpenSearchException;
import org.opensearch.client.opensearch._types.Refresh;
import org.opensearch.client.opensearch._types.aggregations.Aggregate;
import org.opensearch.client.opensearch._types.aggregations.HistogramAggregate;
import org.opensearch.client.opensearch._types.aggregations.TermsAggregation;
import org.opensearch.client.opensearch._types.mapping.Property;
import org.opensearch.client.opensearch._types.query_dsl.BoolQuery;
import org.opensearch.client.opensearch._types.query_dsl.TermsQuery;
import org.opensearch.client.opensearch.cat.NodesResponse;
import org.opensearch.client.opensearch.core.BulkResponse;
import org.opensearch.client.opensearch.core.ClearScrollResponse;
Expand All @@ -48,6 +53,7 @@
import org.opensearch.client.opensearch.core.InfoResponse;
import org.opensearch.client.opensearch.core.MsearchResponse;
import org.opensearch.client.opensearch.core.SearchResponse;
import org.opensearch.client.opensearch.core.SearchRequest;
import org.opensearch.client.opensearch.core.bulk.OperationType;
import org.opensearch.client.opensearch.core.msearch.RequestItem;
import org.opensearch.client.opensearch.indices.CreateIndexResponse;
Expand All @@ -58,9 +64,9 @@
import org.opensearch.client.opensearch.model.ModelTestCase;
import org.opensearch.client.transport.endpoints.BooleanResponse;


import java.io.IOException;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
Expand Down Expand Up @@ -330,6 +336,44 @@ public void testSearchAggregation() throws IOException {

}

@Test
public void testSubAggregation() throws IOException {

highLevelClient().create(_1 -> _1.index("products").id("A").document(new Product(5, "Blue")).refresh(Refresh.True));
highLevelClient().create(_1 -> _1.index("products").id("B").document(new Product(10, "Blue")).refresh(Refresh.True));
highLevelClient().create(_1 -> _1.index("products").id("C").document(new Product(15, "Black")).refresh(Refresh.True));

List<FieldValue> fieldValues = List.of(FieldValue.of("Blue"));

SearchRequest searchRequest = SearchRequest.of(_1 -> _1
.index("products")
.size(0)
.aggregations(
"price", _3 -> _3
.aggregations(Map.of("price", TermsAggregation.of(_4 -> _4
.field("price"))
._toAggregation()))
.filter(BoolQuery.of(_5 -> _5
.filter(List.of(TermsQuery.of(_6 -> _6
.field("color.keyword")
.terms(_7 -> _7
.value(fieldValues)))
._toQuery())))
._toQuery()
)
));
SearchResponse<Product> searchResponse = highLevelClient().search(searchRequest, Product.class);

Aggregate prices = searchResponse.aggregations().get("price")._get()._toAggregate();
assertEquals(2, searchResponse.aggregations().get("price").filter().docCount());
assertEquals(1, prices.filter().aggregations().get("price").dterms().buckets().array().get(0).docCount());
assertEquals(1, prices.filter().aggregations().get("price").dterms().buckets().array().get(1).docCount());

// We've set "size" to zero
assertEquals(0, searchResponse.hits().hits().size());

}

@Test
public void testGetMapping() throws Exception {
// See also VariantsTest.testNestedTaggedUnionWithDefaultTag()
Expand Down Expand Up @@ -405,12 +449,18 @@ public void setMsg(String msg) {

public static class Product {
public double price;
public String color;

public Product() {}
public Product(double price) {
this.price = price;
}

public Product(double price, String color) {
this.price = price;
this.color = color;
}

public double getPrice() {
return this.price;
}
Expand Down

0 comments on commit 1ac790a

Please sign in to comment.