Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
bug: fix complete literal optimization issue
This bug results in some regexes reporting matches at every position even when it should't. The bug happens because the internal literal optimizer winds up using an "empty" searcher that reports a match at every position. This is technically correct whenever the literal searcher is used as a prefilter (although slower than necessary), but an optimization added later enabled the searcher to run on its own and not as a prefilter. i.e., Without the confirm step by the regex engine. In that context, the "empty" searcher is totally incorrect. So this bug corresponds to a code path where the "empty" literal searcher is used, but is also in a case where the literal searcher is used directly to find matches and not as a prefilter. I believe at least the following are required to trigger this path: * The literals extracted need to be "complete." That is, the language described by the regex is small and finite. * There needs to be at least 26 distinct starting bytes among all of the elements in the language described by the regex. * There needs to be fewer than 26 distinct ending bytes among all of the elements in the language described by the regex. * Possibly other criteria... The actual fix is to change the code that selects the literal searcher. Indeed, there was even a comment in the erroneous code saying that the path was impossible, but of course, it isn't. We change that path to return None, as it should have long ago. This in turn results in the case outlined above not using a literal searcher and just the regex engine. Fixes #999
- Loading branch information