Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mapping update for “date_range” field type is not idempotent #2094

Merged
merged 1 commit into from
Feb 14, 2022
Merged

Mapping update for “date_range” field type is not idempotent #2094

merged 1 commit into from
Feb 14, 2022

Conversation

reta
Copy link
Collaborator

@reta reta commented Feb 11, 2022

Signed-off-by: Andriy Redko andriy.redko@aiven.io

Description

Update the same field using the same mapping with the same settings causes


  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_state_exception",
        "reason" : "[index.version.created] is not present in the index settings for index with UUID [null]"
      }
    ],
    "type" : "illegal_state_exception",
    "reason" : "[index.version.created] is not present in the index settings for index with UUID [null]"
  },
  "status" : 500
}

Issues Resolved

Closes #1921

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@reta reta requested a review from a team as a code owner February 11, 2022 22:49
@opensearch-ci-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 8205142772544c6a8667384cf88242a22bad7f42
Log 2347

Reports 2347

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure a9cdbdee3d2e1f79d44b2b2be2ab60693888476f
Log 2348

Reports 2348

@andrross
Copy link
Member

Unrelated failure:

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.gateway.QuorumGatewayIT.testQuorumRecovery" -Dtests.seed=4529A6E9B61BB600 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-DO -Dtests.timezone=America/Inuvik -Druntime.java=17

org.opensearch.gateway.QuorumGatewayIT > testQuorumRecovery FAILED
    java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([4529A6E9B61BB600:A0169A3A1766611B]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:985)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:924)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:913)
        at org.opensearch.gateway.QuorumGatewayIT.testQuorumRecovery(QuorumGatewayIT.java:111)

@andrross
Copy link
Member

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure a9cdbdee3d2e1f79d44b2b2be2ab60693888476f
Log 2349

Reports 2349

andrross
andrross reviewed Feb 12, 2022
View reviewed changes
modules/percolator/src/main/java/org/opensearch/percolator/PercolatorFieldMapper.java
@@ -195,7 +195,7 @@ static BinaryFieldMapper createQueryBuilderFieldBuilder(BuilderContext context)
}

static RangeFieldMapper createExtractedRangeFieldBuilder(String name, RangeType rangeType, BuilderContext context) {
RangeFieldMapper.Builder builder = new RangeFieldMapper.Builder(name, rangeType, true);
RangeFieldMapper.Builder builder = new RangeFieldMapper.Builder(name, rangeType, true, context.indexCreatedVersion());
// For now no doc values, because in processQuery(...) only the Lucene range fields get added:
builder.docValues(false);
return builder.build(context);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused about how this change works. This builder.build(context) is where the failure was coming from previously, right? (this build() method would call the setupFieldType() method which would call context.indexCreatedVersion() which would trigger the "[index.version.created] is not present in the index settings..." exception). Won't this change trigger the same exception on line 198 when it calls context.indexCreatedVersion()?

Copy link
Collaborator Author

@reta reta Feb 12, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is correct but the important part is what context is used when method is called. The exception was raised because BuilderContext was created out of empty settings, but I will double check where this method is called to make sure no exceptions are going to be raised. Thanks for spotting it!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrross made a change to fetch index created only when it is available, should be safer, thanks a lot!

@reta
Copy link
Collaborator Author

reta commented Feb 12, 2022

❌ Gradle Check failure a9cdbde Log 2349

Reports 2349

 2> REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.cluster.routing.MovePrimaryFirstTests.testClusterGreenAfterPartialRelocation" -Dtests.seed=89B359A6A3383DA -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en-AU -Dtests.timezone=America/Barbados -Druntime.java=17
  2> java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([89B359A6A3383DA:5B5151FF4177FCCE]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:985)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:924)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:913)
        at org.opensearch.cluster.routing.MovePrimaryFirstTests.testClusterGreenAfterPartialRelocation(MovePrimaryFirstTests.java:71)
  2> NOTE: leaving temporary files on disk at: /var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/search/server/build/testrun/test/temp/org.opensearch.cluster.routing.MovePrimaryFirstTests_89B359A6A3383DA-001
  2> NOTE: test params are: codec=Asserting(Lucene87): {index_uuid=Lucene84, type=Lucene84}, docValues:{}, maxPointsInLeafNode=27, maxMBSortInHeap=6.418928721965157, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=en-AU, timezone=America/Barbados
  2> NOTE: Linux 5.4.0-1045-aws amd64/Eclipse Adoptium 17.0.2 (64-bit)/cpus=72,threads=1,free=198504208,total=536870912
  2> NOTE: All tests run in this JVM: [BinaryDocValuesRangeQueryTests, AddVotingConfigExclusionsResponseTests, SnapshotStatusTests, GetDataStreamsRequestTests, GetComposableIndexTemplateRequestTests, MultiGetResponseTests, SearchPhaseControllerTests, TransportReplicationAllPermitsAcquisitionTests, ShardStartedClusterStateTaskExecutorTests, DataStreamMetadataTests, MovePrimaryFirstTests]

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 14d440d2229515cf97d13b221d358092b5f8ea19
Log 2374

Reports 2374

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 5c0e50f0a6d496987daa1f368d3f7e338a5caed2
Log 2375

Reports 2375

@reta
Copy link
Collaborator Author

reta commented Feb 14, 2022

x Gradle Check failure 5c0e50f Log 2375

Reports 2375

#1957 :(

@reta
Mapping update for “date_range” field type is not idempotent
6143aaa 
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 6143aaa
Log 2377

Reports 2377

@andrross andrross merged commit 6b6f033 into opensearch-project:main Feb 14, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Feb 14, 2022
@reta @github-actions
Mapping update for “date_range” field type is not idempotent (#2094)
0bd3e17 
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
(cherry picked from commit 6b6f033)
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Feb 14, 2022
Merged
andrross pushed a commit that referenced this pull request Feb 15, 2022
@opensearch-trigger-bot @reta
Mapping update for “date_range” field type is not idempotent (#2094) (…
b0d94cb 
…#2106)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
(cherry picked from commit 6b6f033)

Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrross andrross andrross approved these changes

Assignees
No one assigned
Labels
backport 1.x
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Mapping update for “date_range” field type is not idempotent
3 participants
@reta @opensearch-ci-bot @andrross