Fix tests and edge cases in jump to, diagnostics, highlighting, and rename

[krassowski]

References

Fixes #961

Code changes

• implements a custom Wait Until Page Contains Diagnostic keyword

User-facing changes

• fixes in every single features

Backwards-incompatible changes

None

[github-actions]

👈 Launch a binder notebook on branch krassowski/jupyterlab-lsp/fix-integration-tests

[krassowski]

Problem: CodeMirror no longer tokenizers every variable, instead multiple tokens and even punctuation may be in the same text node. Many variables are also no longer syntax highlighted due to CM choices. The old selectors like:

${def} = Set Variable    xpath:(//span[contains(@class, 'cm-atom')][contains(text(), 'foo')])[last()]

do not work anymore.

Attempt 1:

Select any node within the line containg text using xpath

${def} = Set Variable    xpath:(//div[contains(@class, 'cm-line')]//*[contains(text(), 'foo')])[last()]

This worked... sometimes. The problem is that the cursor may point to a different part of the line and it was terribly flaky.

Attempt 2:

Select the text node using xpath

${def} = Set Variable    xpath:(//div[contains(@class, 'cm-line')]//text()[contains(., 'text')])[last()]

This did not work:

JavascriptException: Message: Cyclic object value
Stacktrace:
RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:188:5
JavaScriptError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:458:5
cloneObject@chrome://remote/content/marionette/json.sys.mjs:41:11
...

Initially I did not know why...

Attempt 3:

Select the node in JS

Locate Last Token With Text
    [Arguments]    ${browser}    ${locator}    ${tag}    ${constraints}
    Execute JavaScript
    ...    window.breadthFirstSearchReverse = function (nodes) {
    ...      for (let n of nodes.toReversed()) {
    ...        if (n.childNodes.length) {
    ...           var result = window.breadthFirstSearchReverse([...n.childNodes]);
    ...           if (result) {return result }
    ...        } else if (n.textContent.match('${locator}')) {
    ...          return n;
    ...        }
    ...      }
    ...    }
    ${element}=    Execute JavaScript
    ...    return window.breadthFirstSearchReverse([...document.querySelectorAll('.cm-line')]);
    [Return]    ${element}

with

Add Location Strategy    lastToken    Locate Last Token With Text    True

Error. Oh, I know this error!

JavascriptException: Message: Cyclic object value
Stacktrace:
RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:187:5
JavaScriptError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:457:5

Attempt 4:

I noticed CodeMirror adds .cmView on nodes which has .dom property - removing it in browser console fixed JSON export for me.

    ...        } else if (n.textContent.match('${locator}')) {
    ...          n.cmView = null;
    ...          return n;
    ...        }

But still getting Cyclic object value in test :(

Attempt 5:

What if we created an overlay which would be event-transparent and used that instead?

    ...          const range = document.createRange();
    ...          range.setStartBefore(n);
    ...          range.setEndAfter(n);
    ...          rect = range.getBoundingClientRect();
    ...          cover.style.position = 'fixed';
    ...          cover.style.background = 'red';
    ...          cover.style.top = rect.top + 'px';
    ...          cover.style.left = rect.left + 'px';
    ...          cover.style.height = rect.height + 'px';
    ...          cover.style.width = rect.width + 'px';
    ...          covert.style.pointerEvents = 'none';
    ...          document.body.appendChild(cover);
    ...          return cover;

Nope:

ElementClickInterceptedException: Message: Element <div> is not clickable at point (482,152) because it does not have pointer events enabled, and element <div class="cm-line"> would receive the click instead

Attempt 6:

What if I kept the pointer events but moved it underneath?

    ...          covert.style.zIndex = -1;

Nope:

ElementClickInterceptedException: Message: Element <div> is not clickable at point (482,152) because another element <div class="cm-line"> obscures it

Attempt 7:

Ok, the text node is problematic, maybe I can just wrap it with a span an return that?

    ...          wrapper = document.createElement('span');
    ...          n.replaceWith(wrapper);
    ...          wrapper.appendChild(n);
    ...          return wrapper;

Nope:

StaleElementReferenceException: Message: The element <span> is no longer attached to the DOM

[krassowski]

It looks this is a general limitation of Selenium (/with Firefox driver?), because even:

...          return document.createTextNode('a');

leads to the Message: Cyclic object value, regardless of the cycle introduced by CodeMirror.

[krassowski]

Some other ideas:

• using on coordinates - requires rewriting almost all keywords (like open context menu) to accept coordinates, and there is zero support in robot selenium library for this
• augmenting CodeMirror tokenizer - would not test the real thing, and is quite hard to do from tests; would require a custom extension activated specifically during tests

[krassowski]

@bollwyvl any suggestions, anything obvious I missed?