Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to repair PartialReparseTest #7823

Merged
merged 3 commits into from
Oct 9, 2024

Conversation

mbien
Copy link
Member

@mbien mbien commented Oct 2, 2024

PartialReparseTest has a fairly high failure rate (a recent merge required 7 restarts).

Investigation showed that it had multiple issues. Fixing those does seem
to improve stability, however, likely does not fix the underlying race condition.
With the changes applied, 1 out of of 50 local runs failed.

But what is more important: the test is hopefully working correctly now

issues fixed:

code.replaceFirst("^", "") was used by accident. This method expects
regexps and won't have the desired effect. This essentially means
that the first parsing run had always errors since the test code
contained ^ characters.

The second parsing run, which did replace the injected snippet had
and off-by-one copy/paste mistake in one of the doRun methods.

But since the first run was already wrong, the result of reparse run
didn't really matter since both were compared - that is also why the
missing semicolon wasn't noticed in testAnonymousFullReparse2() or
the double escaped newline character in testAnonymous().

More asserts added to exclude potential document update issues.

Enabled a previously disabled test case.

first commit has the fixes, second switches to multi-line-string blocks

@mbien mbien added Code cleanup Java [ci] enable extra Java tests (java.completion, java.source.base, java.hints, refactoring.java, form) tests labels Oct 2, 2024
@mbien mbien added this to the NB24 milestone Oct 2, 2024
@mbien mbien marked this pull request as draft October 2, 2024 20:06
@mbien
Copy link
Member Author

mbien commented Oct 2, 2024

first CI run failed again (4th attempt did work)

new discoveries:

  • the fixes did either make it worse or they didn't influence it while run in CI (locally it seemed to work fine)
  • so I tried something else: I switched from memory FS to plain old folders, then ran it 100 times in my clone -> all green

MemoryFileSystem hasn't been changed in a while and might need some cleanup but on first glance I don't see anything suspicious. However it could be as mundane as modification timestamp precision issues when run in-memory or something similar.

Might be worth taking a closer look at it.

Copy link
Contributor

@lahodaj lahodaj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable to me. Thanks for stabilizing the test. For the record, I've ran the test locally 200 times this morning, and it never failed for me.

@mbien
Copy link
Member Author

mbien commented Oct 3, 2024

I've ran the test locally 200 times this morning, and it never failed for me.

@lahodaj did you run the original version or with this PR applied? Since to me it did also fail significantly less often locally than on CI. But since switching away from memory FS, there were no failures anymore, not locally and not on CI (100 runs).

@lahodaj
Copy link
Contributor

lahodaj commented Oct 3, 2024

I've ran the test locally 200 times this morning, and it never failed for me.

@lahodaj did you run the original version or with this PR applied? Since to me it did also fail significantly less often locally than on CI. But since switching away from memory FS, there were no failures anymore, not locally and not on CI (100 runs).

I was running the original version.

@mbien
Copy link
Member Author

mbien commented Oct 6, 2024

MemoryFileSystem hasn't been changed in a while and might need some cleanup but on first glance I don't see anything suspicious. However it could be as mundane as modification timestamp precision issues when run in-memory or something similar.

Might be worth taking a closer look at it.

we can likely exclude memory fs specific modification time resolution issues since I changed the impl to:

  • use Instant instead of Date since it uses an internal high res offset for higher precision
  • checked that all timestamps always go forward in time to exclude multi-cpu locality issues (which are rare these days but I wanted to be sure)
  • updated the modification time inside the write method of the memory file, in case something doesn't close streams

this didn't help. So it is probably safe to exclude the memory fs impl itself being the problem - whats left is the MemoryFileSystem identity or timing issues outside of mem fs code. A clue that this might be about identity is the log which is different when mem fs is in use (no matter if green or red) - so the code path certainly is different somewhere.

mbien added 3 commits October 9, 2024 23:20
Test has a fairly high failure rate, investigation showed that
it had multiple issues. Fixing those does seem to improve stability,
however, likely does not fix the underlying race condition. With the
changes applied, 1 out of of 50 local runs failed.

But what is more important: the test is hopefully working correctly now

issues fixed:

'code.replaceFirst("^", "")' was used by accident. This method expects
regexps and won't have the desired effect. This essentially means
that the first parsing run had always javac errors since the test code
contained '^' characters.

The second parsing run, which did replace the injected snippet had
and off-by-one copy/paste mistake in one of the doRun methods.

But since the first run was already wrong, the result of reparse run
didn't really matter since both were compared - that is also why the
missing semicolon wasn't noticed in testAnonymousFullReparse2() or
the double escaped newline character in testAnonymous().

other:

 - more asserts added to exclude potential document update issues
 - enabled a previously disabled test case
 - testing showed that using a real filesystem (or not using mem fs)
   resolves the sporadic failures entirely. Unfortunately it is
   not clear what the cause for the failures is. (see PR discussion)
 - move test source files into folder matching the declared package
@mbien mbien force-pushed the java-partialreparsetest-fix branch from 5b18e7e to b8d43a3 Compare October 9, 2024 21:29
@mbien mbien marked this pull request as ready for review October 9, 2024 21:29
@mbien
Copy link
Member Author

mbien commented Oct 9, 2024

updated the commit message of the last commit. I did also move the test files into their test package as @neilcsmith-net suggested on slack (unfortunately this did also not fix stability while using mem fs).

Thanks for review, planning to merge once green.

@mbien mbien merged commit ed97fb6 into apache:master Oct 9, 2024
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Code cleanup Java [ci] enable extra Java tests (java.completion, java.source.base, java.hints, refactoring.java, form) tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants