EZP-30664: Skipped indexing erroneous Content and logged all exceptions

[SerheyDolgushev]

Jira: https://jira.ez.no/browse/EZP-30664
Target version: 1.5 (eZ Platform 1.7+)

Continue to index other content in the iteration when an exception happens.

[andrerom]

@SerheyDolgushev what kind of exception are you having issues with? aka could you expand on the issue / motivation for the change here? (ideally in JIRA ticket, and link here)

[SerheyDolgushev]

Few strange scenarios:

1. Trying to index a field type without indexable:

[2019-06-10 22:13:29] console.ERROR: Error thrown while running command "ezplatform:reindex --content-ids='35617,35618,35619,35620,35622,35624,35625,35627,35629,35630,35631,35632,35633,35634,35635,35637,35638,35639,35640,35641,35642,35643,35644,35645,35646,35648,35650,35655,35660,35664,35665,35667,35669,35676,35677,35680,35681,35684,35686,35688,35691,35692,35693,35694,35695,35696,35697,35700,35701,35702' --env=dev". Message: "No type registered for ezbotrvideo." {"exception":"[object] (OutOfBoundsException(code: 0): No type registered for ezbotrvideo. at /Users/sd/Projects/XXX/platform/vendor/ezsystems/ezpublish-kernel/eZ/Publish/Core/Search/Common/FieldRegistry.php:58)","command":"ezplatform:reindex --content-ids='35617,35618,35619,35620,35622,35624,35625,35627,35629,35630,35631,35632,35633,35634,35635,35637,35638,35639,35640,35641,35642,35643,35644,35645,35646,35648,35650,35655,35660,35664,35665,35667,35669,35676,35677,35680,35681,35684,35686,35688,35691,35692,35693,35694,35695,35696,35697,35700,35701,35702' --env=dev","message":"No type registered for ezbotrvideo."} []

2. Trying to index the content with invalid RichText field:

[2019-06-10 21:53:51] console.ERROR: Error thrown while running command "ezplatform:reindex --content-ids='4,10,11,12,13,14,41,42,45,49,50,51,52,54,56,57,104,105,506,507,508,509,514,518,519,520,523,524,525,526,527,528,529,530,531,533,534,535,536,539,541,543,545,547,548,551,553,555,557,558' --env=dev". Message: "Warning: DOMDocument::loadXML(): Entity 'nbsp' not defined in Entity, line: 1" {"exception":"[object] (Symfony\\Component\\Debug\\Exception\\ContextErrorException(code: 0): Warning: DOMDocument::loadXML(): Entity 'nbsp' not defined in Entity, line: 1 at /Users/sd/Projects/XXX/platform/vendor/ezsystems/ezplatform-richtext/src/lib/eZ/FieldType/RichText/RichTextStorage.php:131)","command":"ezplatform:reindex --content-ids='4,10,11,12,13,14,41,42,45,49,50,51,52,54,56,57,104,105,506,507,508,509,514,518,519,520,523,524,525,526,527,528,529,530,531,533,534,535,536,539,541,543,545,547,548,551,553,555,557,558' --env=dev","message":"Warning: DOMDocument::loadXML(): Entity 'nbsp' not defined in Entity, line: 1"} []

And there might be more strange cases

[SerheyDolgushev]

Without this PR, if there is a problematic content in the indexing batch (50 items) - indexing for this batch will be stopped. With this PR problematic content will be ignored (an error will be logged) but the rest content will be indexed.

[andrerom]

@alongosz The two cases are:

• OutOfBoundsException : No type registered for ezbotrvideo.
• Warning: DOMDocument::loadXML(): Entity 'nbsp' not defined in Entity, line: 1"

Should we change interface to state exceptions will be skipped but logged, or do we add exceptions for these cases and document those will be handled?

[alongosz]

Should we change interface to state exceptions will be skipped but logged, or do we add exceptions for these cases and document those will be handled?

If I get it right, the possibilities here are actually limitless due to custom Field Type implementations, so I would log all exceptions and skip.

[SerheyDolgushev]

As this PR does :)

[alongosz]

@SerheyDolgushev you'll probably need to rebase it to at least branch 1.6 to have it in 2.5.x LTS. Fixing this for master will get you that fix in eZ Platform 3.0 ;)
If the bug occurs on some older LTS, like 1.7 or 1.13, please let me know.

[SerheyDolgushev]

@alongosz rebased against 1.6

[andrerom]

@SerheyDolgushev Do you have a Jira ticket for this as well? ;)

[SerheyDolgushev]

Done: https://jira.ez.no/browse/EZP-30664
@andrerom please let me know if something else is required.
BTW, I`m going to submit the same PR for a legacy search.

[andrerom]

@alongosz given this is bug fix I'd like this on 1.5 and kernel on 6.7, ok? Otherwise we will surely face this coming in via support in the near feature anyway ;)

[SerheyDolgushev]

@andrerom @alongosz rebased against 1.5

[andrerom]

@alongosz On closer look, isn't this doing exactly the same as PR from @kmadejski that goes further on this? #122

Ignore, this also covers cases where for instance field types are missing. So this should be done regardlessly of that. Sending to QA.

[micszo]

@andrerom @alongosz I suppose this Travis failure is not related to the changes, right? https://travis-ci.org/ezsystems/ezplatform-solr-search-engine/jobs/547806906?utm_medium=notification&utm_source=github_status

[andrerom]

I suppose this Travis failure is not related to the changes, right?

correct, it's something to do with travis, composer or both. Restarted it, but if it still fails it's a clue we need to do some fixes on the branches here to adapt for changes in travis / composer.

[micszo]

QA Approved on eZ Studio v1.13.5 & eZ Platform EE 2.5 with Solr.

[lserwatka]

@alongosz could you merge it up?

[alongosz]

Merged up to 1.6 as 7521c57, 1.7 as 3d360d2, 2.0 as c409744, and master (3.0@dev) as 1d9e300.