Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REST high-level client: add force merge API #28896

Merged
merged 7 commits into from
Mar 22, 2018
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,17 @@
import org.elasticsearch.action.admin.indices.delete.DeleteIndexResponse;
import org.elasticsearch.action.admin.indices.flush.FlushRequest;
import org.elasticsearch.action.admin.indices.flush.FlushResponse;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeResponse;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingResponse;
import org.elasticsearch.action.admin.indices.open.OpenIndexRequest;
import org.elasticsearch.action.admin.indices.open.OpenIndexResponse;
import org.elasticsearch.action.admin.indices.rollover.RolloverRequest;
import org.elasticsearch.action.admin.indices.rollover.RolloverResponse;
import org.elasticsearch.action.admin.indices.refresh.RefreshRequest;
import org.elasticsearch.action.admin.indices.refresh.RefreshResponse;
import org.elasticsearch.action.admin.indices.rollover.RolloverRequest;
import org.elasticsearch.action.admin.indices.rollover.RolloverResponse;
import org.elasticsearch.action.admin.indices.shrink.ResizeRequest;
import org.elasticsearch.action.admin.indices.shrink.ResizeResponse;

Expand Down Expand Up @@ -259,6 +261,28 @@ public void flushAsync(FlushRequest flushRequest, ActionListener<FlushResponse>
listener, emptySet(), headers);
}

/**
* Force merge one or more indices using the Force Merge API
* <p>
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html">
* Force Merge API on elastic.co</a>
*/
public ForceMergeResponse forceMerge(ForceMergeRequest forceMergeRequest, Header... headers) throws IOException {
return restHighLevelClient.performRequestAndParseEntity(forceMergeRequest, Request::forceMerge, ForceMergeResponse::fromXContent,
emptySet(), headers);
}

/**
* Asynchronously force merge one or more indices using the Force Merge API
* <p>
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html">
* Force Merge API on elastic.co</a>
*/
public void forceMergeAsync(ForceMergeRequest forceMergeRequest, ActionListener<ForceMergeResponse> listener, Header... headers) {
restHighLevelClient.performRequestAsyncAndParseEntity(forceMergeRequest, Request::forceMerge, ForceMergeResponse::fromXContent,
listener, emptySet(), headers);
}

/**
* Checks if the index (indices) exists or not.
* <p>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.admin.indices.flush.FlushRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.open.OpenIndexRequest;
Expand Down Expand Up @@ -234,6 +235,16 @@ static Request flush(FlushRequest flushRequest) {
return new Request(HttpPost.METHOD_NAME, endpoint, parameters.getParams(), null);
}

static Request forceMerge(ForceMergeRequest forceMergeRequest) {
String endpoint = endpoint(forceMergeRequest.indices(), "_forcemerge");
Params parameters = Params.builder();
parameters.withIndicesOptions(forceMergeRequest.indicesOptions());
parameters.putParam("max_num_segments", Integer.toString(forceMergeRequest.maxNumSegments()));
parameters.putParam("only_expunge_deletes", Boolean.toString(forceMergeRequest.onlyExpungeDeletes()));
parameters.putParam("flush", Boolean.toString(forceMergeRequest.flush()));
return new Request(HttpPost.METHOD_NAME, endpoint, parameters.getParams(), null);
}

static Request info() {
return new Request(HttpGet.METHOD_NAME, "/", Collections.emptyMap(), null);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@
import org.elasticsearch.action.admin.indices.delete.DeleteIndexResponse;
import org.elasticsearch.action.admin.indices.flush.FlushRequest;
import org.elasticsearch.action.admin.indices.flush.FlushResponse;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeResponse;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingResponse;
Expand Down Expand Up @@ -438,6 +440,32 @@ public void testFlush() throws IOException {
}
}

public void testForceMerge() throws IOException {
{
String index = "index";
Settings settings = Settings.builder()
.put("number_of_shards", 1)
.put("number_of_replicas", 0)
.build();
createIndex(index, settings);
ForceMergeRequest forceMergeRequest = new ForceMergeRequest(index);
ForceMergeResponse forceMergeResponse =
execute(forceMergeRequest, highLevelClient().indices()::forceMerge, highLevelClient().indices()::forceMergeAsync);
assertThat(forceMergeResponse.getTotalShards(), equalTo(1));
assertThat(forceMergeResponse.getSuccessfulShards(), equalTo(1));
assertThat(forceMergeResponse.getFailedShards(), equalTo(0));
assertThat(forceMergeResponse.getShardFailures(), equalTo(BroadcastResponse.EMPTY));
}
{
String nonExistentIndex = "non_existent_index";
assertFalse(indexExists(nonExistentIndex));
ForceMergeRequest forceMergeRequest = new ForceMergeRequest(nonExistentIndex);
ElasticsearchException exception = expectThrows(ElasticsearchException.class,
() -> execute(forceMergeRequest, highLevelClient().indices()::forceMerge, highLevelClient().indices()::forceMergeAsync));
assertEquals(RestStatus.NOT_FOUND, exception.status());
}
}

public void testExistsAlias() throws IOException {
GetAliasesRequest getAliasesRequest = new GetAliasesRequest("alias");
assertFalse(execute(getAliasesRequest, highLevelClient().indices()::existsAlias, highLevelClient().indices()::existsAliasAsync));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.admin.indices.flush.FlushRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.open.OpenIndexRequest;
Expand Down Expand Up @@ -580,6 +581,36 @@ public void testFlush() {
assertThat(request.getMethod(), equalTo(HttpPost.METHOD_NAME));
}

public void testForceMerge() {
String[] indices = randomIndicesNames(0, 5);
ForceMergeRequest forceMergeRequest = new ForceMergeRequest(indices);
Map<String, String> expectedParams = new HashMap<>();
setRandomIndicesOptions(forceMergeRequest::indicesOptions, forceMergeRequest::indicesOptions, expectedParams);
if (randomBoolean()) {
forceMergeRequest.maxNumSegments(randomInt());
}
expectedParams.put("max_num_segments", Integer.toString(forceMergeRequest.maxNumSegments()));
if (randomBoolean()) {
forceMergeRequest.onlyExpungeDeletes(randomBoolean());
}
expectedParams.put("only_expunge_deletes", Boolean.toString(forceMergeRequest.onlyExpungeDeletes()));
if (randomBoolean()) {
forceMergeRequest.flush(randomBoolean());
}
expectedParams.put("flush", Boolean.toString(forceMergeRequest.flush()));

Request request = Request.forceMerge(forceMergeRequest);
StringJoiner endpoint = new StringJoiner("/", "/", "");
if (indices.length > 0) {
endpoint.add(String.join(",", indices));
}
endpoint.add("_forcemerge");
assertThat(request.getEndpoint(), equalTo(endpoint.toString()));
assertThat(request.getParameters(), equalTo(expectedParams));
assertThat(request.getEntity(), nullValue());
assertThat(request.getMethod(), equalTo(HttpPost.METHOD_NAME));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a test for the case when indices are not set or set to null? I found also in the clear cache PR that things break :) (also with refresh and flush)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @javanna, when indices are not set or set to null, we need to do operations on all the indices ? or throw an exception ? According to our docs it should be the former ? But I see currently if we didn't set indices (null), we will get an exception when we build the endpoint here, shall we need to modify it ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should use the endpoint that requires no indices (e.g. /_forcemerge), the server will expand that to all indices.

}

public void testUpdate() throws IOException {
XContentType xContentType = randomFrom(XContentType.values());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@
import org.elasticsearch.action.admin.indices.delete.DeleteIndexResponse;
import org.elasticsearch.action.admin.indices.flush.FlushRequest;
import org.elasticsearch.action.admin.indices.flush.FlushResponse;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest;
import org.elasticsearch.action.admin.indices.forcemerge.ForceMergeResponse;
import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingResponse;
Expand Down Expand Up @@ -769,6 +771,86 @@ public void onFailure(Exception e) {
}
}

public void testForceMergeIndex() throws Exception {
RestHighLevelClient client = highLevelClient();

{
createIndex("index", Settings.EMPTY);
}

{
// tag::force-merge-request
ForceMergeRequest request = new ForceMergeRequest("index1"); // <1>
ForceMergeRequest requestMultiple = new ForceMergeRequest("index1", "index2"); // <2>
ForceMergeRequest requestAll = new ForceMergeRequest(); // <3>
// end::force-merge-request

// tag::force-merge-request-indicesOptions
request.indicesOptions(IndicesOptions.lenientExpandOpen()); // <1>
// end::force-merge-request-indicesOptions

// tag::force-merge-request-segments-num
request.maxNumSegments(1); // <1>
// end::force-merge-request-segments-num

// tag::force-merge-request-only-expunge-deletes
request.onlyExpungeDeletes(true); // <1>
// end::force-merge-request-only-expunge-deletes

// tag::force-merge-request-flush
request.flush(true); // <1>
// end::force-merge-request-flush

// tag::force-merge-execute
ForceMergeResponse forceMergeResponse = client.indices().forceMerge(request);
// end::force-merge-execute

// tag::force-merge-response
int totalShards = forceMergeResponse.getTotalShards(); // <1>
int successfulShards = forceMergeResponse.getSuccessfulShards(); // <2>
int failedShards = forceMergeResponse.getFailedShards(); // <3>
DefaultShardOperationFailedException[] failures = forceMergeResponse.getShardFailures(); // <4>
// end::force-merge-response

// tag::force-merge-execute-listener
ActionListener<ForceMergeResponse> listener = new ActionListener<ForceMergeResponse>() {
@Override
public void onResponse(ForceMergeResponse forceMergeResponse) {
// <1>
}

@Override
public void onFailure(Exception e) {
// <2>
}
};
// end::force-merge-execute-listener

// Replace the empty listener by a blocking listener in test
final CountDownLatch latch = new CountDownLatch(1);
listener = new LatchedActionListener<>(listener, latch);

// tag::force-merge-execute-async
client.indices().forceMergeAsync(request, listener); // <1>
// end::force-merge-execute-async

assertTrue(latch.await(30L, TimeUnit.SECONDS));
}

{
// tag::force-merge-notfound
try {
ForceMergeRequest request = new ForceMergeRequest("does_not_exist");
client.indices().forceMerge(request);
} catch (ElasticsearchException exception) {
if (exception.status() == RestStatus.NOT_FOUND) {
// <1>
}
}
// end::force-merge-notfound
}
}

public void testCloseIndex() throws Exception {
RestHighLevelClient client = highLevelClient();

Expand Down
102 changes: 102 additions & 0 deletions docs/java-rest/high-level/indices/force_merge.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
[[java-rest-high-force-merge]]
=== Force Merge API

[[java-rest-high-force-merge-request]]
==== Force merge Request

A `ForceMergeRequest` can be applied to one or more indices, or even on `_all` the indices:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-request]
--------------------------------------------------
<1> Force merge one index
<2> Force merge multiple indices
<3> Force merge all the indices

==== Optional arguments

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-request-indicesOptions]
--------------------------------------------------
<1> Setting `IndicesOptions` controls how unavailable indices are resolved and
how wildcard expressions are expanded

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-request-segments-num]
--------------------------------------------------
<1> Set `max_num_segments` to control the number of segments to merge down to.

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-request-only-expunge-deletes]
--------------------------------------------------
<1> Set the `only_expunge_deletes` flag to `true`

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-request-flush]
--------------------------------------------------
<1> Set the `flush` flag to `true`

[[java-rest-high-force-merge-sync]]
==== Synchronous Execution

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-execute]
--------------------------------------------------

[[java-rest-high-force-merge-async]]
==== Asynchronous Execution

The asynchronous execution of a force merge request requires both the `ForceMergeRequest`
instance and an `ActionListener` instance to be passed to the asynchronous
method:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-execute-async]
--------------------------------------------------
<1> The `ForceMergeRequest` to execute and the `ActionListener` to use when
the execution completes

The asynchronous method does not block and returns immediately. Once it is
completed the `ActionListener` is called back using the `onResponse` method
if the execution successfully completed or using the `onFailure` method if
it failed.

A typical listener for `ForceMergeResponse` looks like:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-execute-listener]
--------------------------------------------------
<1> Called when the execution is successfully completed. The response is
provided as an argument
<2> Called in case of failure. The raised exception is provided as an argument

[[java-rest-high-force-merge-response]]
==== Force merge Response

The returned `ForceMergeResponse` allows to retrieve information about the
executed operation as follows:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-response]
--------------------------------------------------
<1> Total number of shards hit by the force merge request
<2> Number of shards where the force merge has succeeded
<3> Number of shards where the force merge has failed
<4> A list of failures if the operation failed on one or more shards

By default, if the indices were not found, an `ElasticsearchException` will be thrown:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[force-merge-notfound]
--------------------------------------------------
<1> Do something if the indices to be force merged were not found
2 changes: 2 additions & 0 deletions docs/java-rest/high-level/supported-apis.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Index Management::
* <<java-rest-high-split-index>>
* <<java-rest-high-refresh>>
* <<java-rest-high-flush>>
* <<java-rest-high-force-merge>>
* <<java-rest-high-rollover-index>>

Mapping Management::
Expand All @@ -72,6 +73,7 @@ include::indices/shrink_index.asciidoc[]
include::indices/split_index.asciidoc[]
include::indices/refresh.asciidoc[]
include::indices/flush.asciidoc[]
include::indices/force_merge.asciidoc[]
include::indices/rollover.asciidoc[]
include::indices/put_mapping.asciidoc[]
include::indices/update_aliases.asciidoc[]
Expand Down
Loading