Publisher list improvements

[NejcZdovc]

Resolves brave/brave-browser#9376

Submitter Checklist:

• Submitted a ticket for my issue if one did not already exist.
• Used Github auto-closing keywords in the commit message.
• Added/updated tests for this change (for new code or code which already has tests).
• Verified that these changes build without errors on
  • Android
  • iOS
  • Linux
  • macOS
  • Windows
• Verified that these changes pass automated tests (unit, browser, security tests) on
  • iOS
  • Linux
  • macOS
  • Windows
• Verified that all lint errors/warnings are resolved (npm run lint)
• Ran git rebase master (if needed).
• Ran git rebase -i to squash commits (if needed).
• Tagged reviewers and labelled the pull request as needed.
• Request a security/privacy review as needed.
• Add appropriate QA labels (QA/Yes or QA/No) to include the closed issue in milestone
• Public documentation has been updated as necessary. For instance:
  • https://github.com/brave/brave-browser/wiki/Deviations-from-Chromium-(features-we-disable-or-remove)
  • https://github.com/brave/brave-browser/wiki/Proxy-redirected-URLs
  • https://github.com/brave/brave-browser/wiki/Fingerprinting-Protections
  • https://github.com/brave/brave-browser/wiki/Brave%E2%80%99s-Use-of-Referral-Codes
  • https://github.com/brave/brave-browser/wiki/Custom-Headers
  • https://github.com/brave/brave-browser/wiki/Web-compatibility-issues-with-tracking-protection
  • https://github.com/brave/brave-browser/wiki/QA-Guide

Test Plan:

• clean profile (do it with production)
• enable rewards
• go to db file and make sure that publisher data from the server is there
• visit https://brave.com/ and make sure that you see custom banner

Reviewer Checklist:

• New files have MPL-2.0 license header.
• Request a security/privacy review as needed.
• Adequate test coverage exists to prevent regressions
• Verify test plan is specified in PR before merging to source

After-merge Checklist:

• The associated issue milestone is set to the smallest version that the
  changes has landed on.
• All relevant documentation has been updated.

[NejcZdovc]

Restarting Windows as it failed on unrelated stuff

[emerick]

LGTM, just one minor comment

[emerick]

Was there a reason for pulling this variable out of the loop?

[kjozwiak]

Verification PASSED on macOS 10.15.3 x64 using the following build:

Brave | 1.10.3 Chromium: 81.0.4044.113 (Official Build) nightly (64-bit)
-- | --
Revision | e3225dafb0475864a1812a374d73a92e391635ac-refs/branch-heads/4044@{#936}
OS | macOS Version 10.15.3 (Build 19D76)

Verified that the Publisher List was being downloaded without any issues once brave://rewards was enabled:

[ RESPONSE - OnDownload ]
> time: 1587586315
> result: Success
> http code: 200
> response: Publisher list
> headers accept-ranges: bytes
> headers access-control-allow-headers: Origin, X-Requested-With, Content-Type, Accept, Authorization
> headers access-control-allow-methods: GET
> headers access-control-allow-origin: https://rewards.bravesoftware.com
> headers access-control-request-method: GET
> headers age: 2207
> headers cache-control: max-age=3600, public
> headers content-encoding: gzip
> headers content-length: 18452120
> headers content-type: application/json; charset=utf-8
> headers date: Wed, 22 Apr 2020 20:11:54 GMT
> headers etag: W/"52145fabb213314e5a845e288a185216"
> headers server: Cowboy
> headers status: 200
> headers strict-transport-security: max-age=31536000; includeSubDomains
> headers vary: Accept-Encoding
> headers via: 1.1 varnish
> headers x-cache: HIT
> headers x-cache-hits: 0
> headers x-frame-options: deny
> headers x-request-id: fd7138c9-a631-46b1-937a-14f5305b7691
> headers x-runtime: 5.987030
> headers x-served-by: cache-yyz4549-YYZ
[ END RESPONSE ]

Ensured that brave.com, chess.com and kjozwiak.github.com appeared as verified and displayed the custom banners without any issues.

brave.com example:

chess.com example:

kjozwiak.github.io example:

Also checked and ensured that the server_publisher_info database was populated as per the following:

Ensured that publisher_info was populated whenever you visited a website: