Show verified publishers in Rewards panel for new Twitter

[emerick]

Fix brave/brave-browser#5388

Submitter Checklist:

• Submitted a ticket for my issue if one did not already exist.
• Used Github auto-closing keywords in the commit message.
• Added/updated tests for this change (for new code or code which already has tests).
• Verified that these changes build without errors on
  • Android
  • iOS
  • Linux
  • macOS
  • Windows
• Verified that these changes pass automated tests (unit, browser, security tests) on
  • iOS
  • Linux
  • macOS
  • Windows
• Verified that all lint errors/warnings are resolved (npm run lint)
• Ran git rebase master (if needed).
• Ran git rebase -i to squash commits (if needed).
• Tagged reviewers and labelled the pull request as needed.
• Request a security/privacy review as needed.
• Add appropriate QA labels (QA/Yes or QA/No) to include the closed issue in milestone
• Public documentation has been updated as necessary. For instance:
  • https://github.com/brave/brave-browser/wiki/Deviations-from-Chromium-(features-we-disable-or-remove)
  • https://github.com/brave/brave-browser/wiki/Proxy-redirected-URLs
  • https://github.com/brave/brave-browser/wiki/Fingerprinting-Protection-Mode
  • https://github.com/brave/brave-browser/wiki/Brave%E2%80%99s-Use-of-Referral-Codes
  • https://github.com/brave/brave-browser/wiki/Custom-Headers
  • https://github.com/brave/brave-browser/wiki/Web-compatibility-issues-with-tracking-protection
  • https://github.com/brave/brave-browser/wiki/QA-Guide

Test Plan:

1. Open brave
2. Enable rewards
3. Open verified pubs in new tab
4. Open BR panel
5. Verified pubs info is not displayed in BR panel

Must test this for three scenarios:

1. While logged into old Twitter
2. While logged into new Twitter
3. While not logged in at all, and just looking directly at a user page like http://twitter.com/emerick

Also, for the various scenarios, please use a clean profile as the data can be cached and lead you into thinking it's working when it's not.

Reviewer Checklist:

• New files have MPL-2.0 license header.
• Request a security/privacy review as needed.
• Adequate test coverage exists to prevent regressions
• Verify test plan is specified in PR before merging to source

After-merge Checklist:

• The associated issue milestone is set to the smallest version that the
  changes has landed on.
• All relevant documentation has been updated.

[diracdeltas]

other than #3029 (comment), lgtm

[petemill]

I’m a little confused. It does work in my testing, but I’m trying to understand why the change is needed. Primarily, I'm worried about requesting the user details via the twitter API as soon as the page starts loading. Could this happen before the page makes any authenticated API requests, leaving our request with no auth headers yet? I'm not sure I saw that happening in my testing, but wondering if it's possible.

In general, I'm finding it difficult to follow the path of data through the code between content-script / background script and API. When a user visits the page twitter.com/user(/status/tweet), doesn’t the c++ backend interpret the URL and extract the username, regardless of 'new' or 'old' twitter. Can't that be fed to the rewards backend (for attention attribution) and, later, the panel (for display), without the content script having to look it up? Also (or similarly?) why do we provide the url in a different format for 'new' twitter (.../intent/...)?

[emerick]

@petemill The main problem is that the C++ backend scrapes the DOM for the Twitter user ID, but that ID is no longer present in the new Twitter redesign. In order to work around that, we request the user ID from the Twitter API and then send it to the backend. In order to avoid adding a new parameter to our APIs for the user ID (and then threading that through Mojo), @NejcZdovc suggested to just pass a different URL that already contains everything we need for the backend parsing when dealing with a "new Twitter" profile.

As far as the possibility of sending a request with no auth headers, I had the same concern. It doesn't seem to happen in practice, but it doesn't mean it's not possible. I'm definitely open to other suggestions!

[petemill]

@emerick oh ok if it was already decided to do it this way instead of adding an explicit API. Thanks for the explanation @emerick. I think it would help us in the future if we add a code comment to explain exactly the above: that for old twitter, the backend parses the DOM itself, but for new twitter we fetch the userId ourselves and add it to a fake URL so the backend has everything it needs. And @NejcZdovc perhaps create an issue to modify the API to expect a specific data object where one of the properties is the userId instead of putting it in a fake Url?

[emerick]

@petemill Just to be totally clear, that isn't a fake URL. It's just an alternate URL that Twitter provides to a profile-like page (with the happy side effect that it allows us to parse a user ID from the DOM).

[petemill]

Nice solution. Thanks for clearing it up 👍