Fixing flaky test

[sleepy-owl]

The test test_cbow_hs_training_fromfile is flaky. It failed 21 out of 250 times that I ran. This PR addresses this issue.

In some cases, the assertion fails when overlap_count is less than 2 (it can be 1 or 0). To reduce the flakiness of this test, I suggest marking it as flaky, so that it can be re-run once on failure. Running twice will reduce the failure rate to less than 1% instead of over 8% currently.

Do you guys think this makes sense? Please let me know if this looks good or if you have any other suggestions. Also, here I assume there are no bugs in the code under test.

[piskvorky]

Thanks. Any idea why it's flaky? Unless the cause is understood, I'm worried this may be just covering symptoms of a real problem.

[sleepy-owl]

My guess would be perhaps some parameters of FT_gensim can be better tuned? Maybe use more epochs? However, that would increase the running time of the test very likely. Re-running may be cheaper on average?

Let me know what you think.

[mpenkov]

Could it have something to do with your local setup? I don't recall this test being a frequent offender on CI.

[sleepy-owl]

Seems like this only happens if I run multiple instances of the test in parallel. Do you know why? Are they reading/writing to the same file?

[mpenkov]

No, I don't know OTOH. I'd have to investigate, but it wouldn't happen anytime soon, as there are other, more pressing issues in the project. From what you've mentioned, it seems like a potential race condition - things working well in series but not working in parallel is definitely a red flag.

Are you able to investigate yourself? If yes, then please go ahead and let us know when you find the cause. If no, please let me know and I will add it to the issue list.

I'm closing the PR for now, as merging this change would mask an underlying issue instead of solving it.

[sleepy-owl]

I will try to investigate, but it will take me some time to understand what the test is doing.