Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add get field mappings to High Level REST API Client #31423

Merged
merged 17 commits into from
Jun 23, 2018
Merged

Add get field mappings to High Level REST API Client #31423

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1730875
Add get field mappings to High Level REST API Client
Jun 19, 2018
b350c33
don't introduce ParseField in XContentBuilder so far
Jun 19, 2018
4cb66ae
don't introduce complex endpoint method; improve fields randomization…
Jun 19, 2018
207a8d5
rename getFieldMappings to getFieldMapping for consistency with REST …
Jun 20, 2018
f095c57
simplification of GetFieldMappingsResponse.fromXContent + allow rando…
Jun 20, 2018
ce482f1
add missed doc section for get-field-mapping-request-local
Jun 20, 2018
2205664
clean up
Jun 21, 2018
88192e8
use regexp in getRandomFieldsExcludeFilter
Jun 21, 2018
e38c145
add and use setRandomLocal(Consumer)
Jun 21, 2018
a7403ed
rebuilt fromXContent
Jun 21, 2018
b6265f7
Merge remote-tracking branch 'remotes/origin/master' into get_field_m…
Jun 21, 2018
a12c32e
polish a bit randomFieldsExcludeFilter regexp
Jun 22, 2018
3424790
drop unnecessary assertThat-s
Jun 22, 2018
03b0396
drop intermediate builder from fromXContent process
Jun 22, 2018
c991dbc
unused import...
Jun 22, 2018
fa77332
another round of fromXContent simplification
Jun 22, 2018
7d61d23
improve junk protection on parsing
Jun 22, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions client/rest-high-level/src/main/java/org/elasticsearch/client/IndicesClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeResponse;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsResponse;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsResponse;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
Expand Down Expand Up @@ -188,6 +190,35 @@ public void getMappingsAsync(GetMappingsRequest getMappingsRequest, RequestOptio
GetMappingsResponse::fromXContent, listener, emptySet());
}

/**
* Retrieves the field mappings on an index or indices using the Get Field Mapping API.
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-field-mapping.html">
* Get Field Mapping API on elastic.co</a>
* @param getFieldMappingsRequest the request
* @param options the request options (e.g. headers), use {@link RequestOptions#DEFAULT} if nothing needs to be customized
* @return the response
* @throws IOException in case there is a problem sending the request or parsing back the response
*/
public GetFieldMappingsResponse getFieldMapping(GetFieldMappingsRequest getFieldMappingsRequest,
RequestOptions options) throws IOException {
return restHighLevelClient.performRequestAndParseEntity(getFieldMappingsRequest, RequestConverters::getFieldMapping, options,
GetFieldMappingsResponse::fromXContent, emptySet());
}

/**
* Asynchronously retrieves the field mappings on an index on indices using the Get Field Mapping API.
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-field-mapping.html">
* Get Field Mapping API on elastic.co</a>
* @param getFieldMappingsRequest the request
* @param options the request options (e.g. headers), use {@link RequestOptions#DEFAULT} if nothing needs to be customized
* @param listener the listener to be notified upon request completion
*/
public void getFieldMappingAsync(GetFieldMappingsRequest getFieldMappingsRequest, RequestOptions options,
ActionListener<GetFieldMappingsResponse> listener) {
restHighLevelClient.performRequestAsyncAndParseEntity(getFieldMappingsRequest, RequestConverters::getFieldMapping, options,
GetFieldMappingsResponse::fromXContent, listener, emptySet());
}

/**
* Updates aliases using the Index Aliases API.
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html">
Expand Down
20 changes: 20 additions & 0 deletions client/rest-high-level/src/main/java/org/elasticsearch/client/RequestConverters.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
import org.elasticsearch.action.admin.indices.flush.SyncedFlushRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.open.OpenIndexRequest;
Expand Down Expand Up @@ -229,6 +230,25 @@ static Request getMappings(GetMappingsRequest getMappingsRequest) throws IOExcep
return request;
}

static Request getFieldMapping(GetFieldMappingsRequest getFieldMappingsRequest) throws IOException {
String[] indices = getFieldMappingsRequest.indices() == null ? Strings.EMPTY_ARRAY : getFieldMappingsRequest.indices();
String[] types = getFieldMappingsRequest.types() == null ? Strings.EMPTY_ARRAY : getFieldMappingsRequest.types();
String[] fields = getFieldMappingsRequest.fields() == null ? Strings.EMPTY_ARRAY : getFieldMappingsRequest.fields();

String endpoint = new EndpointBuilder().addCommaSeparatedPathParts(indices)
.addPathPartAsIs("_mapping").addCommaSeparatedPathParts(types)
.addPathPartAsIs("field").addCommaSeparatedPathParts(fields)
.build();

Request request = new Request(HttpGet.METHOD_NAME, endpoint);

Params parameters = new Params(request);
parameters.withIndicesOptions(getFieldMappingsRequest.indicesOptions());
parameters.withIncludeDefaults(getFieldMappingsRequest.includeDefaults());
parameters.withLocal(getFieldMappingsRequest.local());
return request;
}

static Request refresh(RefreshRequest refreshRequest) {
String[] indices = refreshRequest.indices() == null ? Strings.EMPTY_ARRAY : refreshRequest.indices();
Request request = new Request(HttpPost.METHOD_NAME, endpoint(indices, "_refresh"));
Expand Down
38 changes: 38 additions & 0 deletions client/rest-high-level/src/test/java/org/elasticsearch/client/IndicesClientIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeResponse;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsResponse;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsResponse;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
Expand Down Expand Up @@ -74,6 +76,7 @@
import org.elasticsearch.cluster.metadata.IndexMetaData;
import org.elasticsearch.cluster.metadata.IndexTemplateMetaData;
import org.elasticsearch.common.ValidationException;
import org.elasticsearch.common.bytes.BytesArray;
import org.elasticsearch.common.settings.Setting;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.unit.ByteSizeUnit;
Expand Down Expand Up @@ -378,6 +381,41 @@ public void testGetMapping() throws IOException {
assertThat(mappings, equalTo(expected));
}

public void testGetFieldMapping() throws IOException {
String indexName = "test";
createIndex(indexName, Settings.EMPTY);

PutMappingRequest putMappingRequest = new PutMappingRequest(indexName);
putMappingRequest.type("_doc");
XContentBuilder mappingBuilder = JsonXContent.contentBuilder();
mappingBuilder.startObject().startObject("properties").startObject("field");
mappingBuilder.field("type", "text");
mappingBuilder.endObject().endObject().endObject();
putMappingRequest.source(mappingBuilder);

PutMappingResponse putMappingResponse =
execute(putMappingRequest, highLevelClient().indices()::putMapping, highLevelClient().indices()::putMappingAsync);
assertTrue(putMappingResponse.isAcknowledged());

GetFieldMappingsRequest getFieldMappingsRequest = new GetFieldMappingsRequest()
.indices(indexName)
.types("_doc")
.fields("field");

GetFieldMappingsResponse getFieldMappingsResponse =
execute(getFieldMappingsRequest,
highLevelClient().indices()::getFieldMapping,
highLevelClient().indices()::getFieldMappingAsync);

final Map<String, GetFieldMappingsResponse.FieldMappingMetaData> fieldMappingMap =
getFieldMappingsResponse.mappings().get(indexName).get("_doc");

final GetFieldMappingsResponse.FieldMappingMetaData metaData =
new GetFieldMappingsResponse.FieldMappingMetaData("field",
new BytesArray("{\"field\":{\"type\":\"text\"}}"));
assertThat(fieldMappingMap, equalTo(Collections.singletonMap("field", metaData)));
}

public void testDeleteIndex() throws IOException {
{
// Delete index if exists
Expand Down
64 changes: 62 additions & 2 deletions client/rest-high-level/src/test/java/org/elasticsearch/client/RequestConvertersTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
import org.elasticsearch.action.admin.indices.flush.SyncedFlushRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.open.OpenIndexRequest;
Expand Down Expand Up @@ -456,6 +457,61 @@ public void testGetMapping() throws IOException {
assertThat(HttpGet.METHOD_NAME, equalTo(request.getMethod()));
}

public void testGetFieldMapping() throws IOException {
GetFieldMappingsRequest getFieldMappingsRequest = new GetFieldMappingsRequest();

String[] indices = Strings.EMPTY_ARRAY;
if (randomBoolean()) {
indices = randomIndicesNames(0, 5);
getFieldMappingsRequest.indices(indices);
} else if (randomBoolean()) {
getFieldMappingsRequest.indices((String[]) null);
}

String type = null;
if (randomBoolean()) {
type = randomAlphaOfLengthBetween(3, 10);
getFieldMappingsRequest.types(type);
} else if (randomBoolean()) {
getFieldMappingsRequest.types((String[]) null);
}

String[] fields = null;
if (randomBoolean()) {
fields = new String[randomIntBetween(1, 5)];
for (int i = 0; i < fields.length; i++) {
fields[i] = randomAlphaOfLengthBetween(3, 10);
}
getFieldMappingsRequest.fields(fields);
} else if (randomBoolean()) {
getFieldMappingsRequest.fields((String[]) null);
}

Map<String, String> expectedParams = new HashMap<>();

setRandomIndicesOptions(getFieldMappingsRequest::indicesOptions, getFieldMappingsRequest::indicesOptions, expectedParams);
setRandomLocal(getFieldMappingsRequest::local, expectedParams);

Request request = RequestConverters.getFieldMapping(getFieldMappingsRequest);
StringJoiner endpoint = new StringJoiner("/", "/", "");
String index = String.join(",", indices);
if (Strings.hasLength(index)) {
endpoint.add(index);
}
endpoint.add("_mapping");
if (type != null) {
endpoint.add(type);
}
endpoint.add("field");
if (fields != null) {
endpoint.add(String.join(",", fields));
}
assertThat(endpoint.toString(), equalTo(request.getEndpoint()));

assertThat(expectedParams, equalTo(request.getParameters()));
assertThat(HttpGet.METHOD_NAME, equalTo(request.getMethod()));
}

public void testDeleteIndex() {
String[] indices = randomIndicesNames(0, 5);
DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest(indices);
Expand Down Expand Up @@ -2239,16 +2295,20 @@ private static void setRandomHumanReadable(GetIndexRequest request, Map<String,
}
}

private static void setRandomLocal(MasterNodeReadRequest<?> request, Map<String, String> expectedParams) {
private static void setRandomLocal(Consumer<Boolean> setter, Map<String, String> expectedParams) {
if (randomBoolean()) {
boolean local = randomBoolean();
request.local(local);
setter.accept(local);
if (local) {
expectedParams.put("local", String.valueOf(local));
}
}
}

private static void setRandomLocal(MasterNodeReadRequest<?> request, Map<String, String> expectedParams) {
setRandomLocal(request::local, expectedParams);
}

private static void setRandomTimeout(Consumer<String> setter, TimeValue defaultTimeout, Map<String, String> expectedParams) {
if (randomBoolean()) {
String timeout = randomTimeValue();
Expand Down
110 changes: 110 additions & 0 deletions ...el/src/test/java/org/elasticsearch/client/documentation/IndicesClientDocumentationIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeResponse;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetFieldMappingsResponse;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest;
import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsResponse;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
Expand Down Expand Up @@ -90,6 +92,7 @@

import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
Expand Down Expand Up @@ -703,6 +706,113 @@ public void onFailure(Exception e) {
}
}

public void testGetFieldMapping() throws IOException, InterruptedException {
RestHighLevelClient client = highLevelClient();

{
CreateIndexResponse createIndexResponse = client.indices().create(new CreateIndexRequest("twitter"), RequestOptions.DEFAULT);
assertTrue(createIndexResponse.isAcknowledged());
PutMappingRequest request = new PutMappingRequest("twitter");
request.type("tweet");
request.source(
"{\n" +
" \"properties\": {\n" +
" \"message\": {\n" +
" \"type\": \"text\"\n" +
" },\n" +
" \"timestamp\": {\n" +
" \"type\": \"date\"\n" +
" }\n" +
" }\n" +
"}", // <1>
XContentType.JSON);
PutMappingResponse putMappingResponse = client.indices().putMapping(request, RequestOptions.DEFAULT);
assertTrue(putMappingResponse.isAcknowledged());
}

// tag::get-field-mapping-request
GetFieldMappingsRequest request = new GetFieldMappingsRequest(); // <1>
request.indices("twitter"); // <2>
request.types("tweet"); // <3>
request.fields("message", "timestamp"); // <4>
// end::get-field-mapping-request

// tag::get-field-mapping-request-indicesOptions
request.indicesOptions(IndicesOptions.lenientExpandOpen()); // <1>
// end::get-field-mapping-request-indicesOptions

// tag::get-field-mapping-request-local
request.local(true); // <1>
// end::get-field-mapping-request-local

{

// tag::get-field-mapping-execute
GetFieldMappingsResponse response =
client.indices().getFieldMapping(request, RequestOptions.DEFAULT);
// end::get-field-mapping-execute

// tag::get-field-mapping-response
final Map<String, Map<String, Map<String, GetFieldMappingsResponse.FieldMappingMetaData>>> mappings =
response.mappings();// <1>
final Map<String, GetFieldMappingsResponse.FieldMappingMetaData> typeMappings =
mappings.get("twitter").get("tweet"); // <2>
final GetFieldMappingsResponse.FieldMappingMetaData metaData =
typeMappings.get("message");// <3>

final String fullName = metaData.fullName();// <4>
final Map<String, Object> source = metaData.sourceAsMap(); // <5>
// end::get-field-mapping-response
}

{
// tag::get-field-mapping-execute-listener
ActionListener<GetFieldMappingsResponse> listener =
new ActionListener<GetFieldMappingsResponse>() {
@Override
public void onResponse(GetFieldMappingsResponse putMappingResponse) {
// <1>
}

@Override
public void onFailure(Exception e) {
// <2>
}
};
// end::get-field-mapping-execute-listener

// Replace the empty listener by a blocking listener in test
final CountDownLatch latch = new CountDownLatch(1);
final ActionListener<GetFieldMappingsResponse> latchListener = new LatchedActionListener<>(listener, latch);
listener = ActionListener.wrap(r -> {
final Map<String, Map<String, Map<String, GetFieldMappingsResponse.FieldMappingMetaData>>> mappings =
r.mappings();
final Map<String, GetFieldMappingsResponse.FieldMappingMetaData> typeMappings =
mappings.get("twitter").get("tweet");
final GetFieldMappingsResponse.FieldMappingMetaData metaData1 = typeMappings.get("message");

final String fullName = metaData1.fullName();
final Map<String, Object> source = metaData1.sourceAsMap();

assertThat(fullName, equalTo("message"));
assertThat(source, equalTo(Collections.singletonMap("message", Collections.singletonMap("type", "text"))));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need these assertions here? given that this test is added to test the docs snippets, I would avoid testing the functionality itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is still here?

latchListener.onResponse(r);
}, e -> {
latchListener.onFailure(e);
fail("should not fail");
});

// tag::get-field-mapping-execute-async
client.indices().getFieldMappingAsync(request, RequestOptions.DEFAULT, listener); // <1>
// end::get-field-mapping-execute-async

assertTrue(latch.await(30L, TimeUnit.SECONDS));
}


}


public void testOpenIndex() throws Exception {
RestHighLevelClient client = highLevelClient();

Expand Down
Loading