Investigate CI Failures #3064
Comments
cwperks
added
bug
|
I've broken the windows wget install into a separate issue that I'll track down |
|
I'm seeing the following consistant failures, lets start with these:
|
|
I'm going to tackle
|
|
Note; there are still these tests that are intermittently failing, please address them in # of failure order after addressing the consistent failures
|
|
Issue is not from the REST API change. |
|
I believe these test failures are a result of the Lucene upgrade in core: opensearch-project/OpenSearch#8668 Some audit log events are working like access granted or access denied, but its failing to log document read events. Looking into this further now. Could any of the recent changes in IndexSearcher relate? https://github.com/apache/lucene/commits/main/lucene/core/src/java/org/apache/lucene/search/IndexSearcher.java |
|
Thanks @cwperks , looking into it! |
|
We are seeing issues in DLS/FLS & Audit logging because the way we detect field access is through the DlsFlsFilterLeafReader which looks like no longer works in the same with the most recent version of lucene. This class is used to notify the audit logging systems after a document read has been completed so it can save the document get audit event and it should be of little surprise that the DLS/FLS systems depend on knowing what fields were accessed. This is what is impacting all of the different classes of tests that are consistently failing [comment] - same root cause. Intermittent tests still need consideration. Pretty sure the underlying reason is coming from changes to IndexReader [link] /**
* Expert: visits the fields of a stored document, for custom processing/loading of each field. If
* you simply want to load all fields, use {@link #document(int)}. If you want to load a subset,
* use {@link DocumentStoredFieldVisitor}.
*
* @deprecated use {@link #storedFields()} to retrieve one or more documents
*/
@Deprecated
public abstract void document(int docID, StoredFieldVisitor visitor) throws IOException; |
|
Haha- well @cwperks you beat me to it. I say that is definitely what is going on. |
|
@cwperks thanks a lot for the investigation, are you looking into the fix or you need a hand there? (I started to look but haven't yet reached the cause and/or solution), thank you |
|
Hey @reta , I just opened up a draft PR a second ago: #3069 It was a Lucene-related change that broke the build, but it wasn't the upgrade to 9.8 that broke the security plugin's build. It was this PR: opensearch-project/OpenSearch#7792 |
…3069) There are multiple PRs in core affecting the security plugin that the security plugin needs to adapt to. - opensearch-project/OpenSearch#7792 - opensearch-project/OpenSearch#8826 - opensearch-project/OpenSearch#8668 I am opening a Draft PR that includes a fix for the Lucene-related test failures which was caused by opensearch-project/OpenSearch#7792 Resolves: #3064 Signed-off-by: Craig Perkins <cwperx@amazon.com>
…pensearch-project#3069) There are multiple PRs in core affecting the security plugin that the security plugin needs to adapt to. - opensearch-project/OpenSearch#7792 - opensearch-project/OpenSearch#8826 - opensearch-project/OpenSearch#8668 I am opening a Draft PR that includes a fix for the Lucene-related test failures which was caused by opensearch-project/OpenSearch#7792 Resolves: opensearch-project#3064 Signed-off-by: Craig Perkins <cwperx@amazon.com> (cherry picked from commit 08d1734)
The PR queue is piling up with various issues seen on pull requests.
There is one known error causing flakiness in primarily the
ComplianceAuditlogTestdue to an issue described #3021 [1] which is being actively worked on by @DarshitChanpura.As of writing, the top 3 PRs in the queue are failing with various different issues:
Getting to the root of CI issues and getting stable CI will help clear the PR queue from piling up
Related
The text was updated successfully, but these errors were encountered: