Classify search engine results page content

[tmancey]

Resolves brave/brave-browser#6000

Submitter Checklist:

• I confirm that no security/privacy review is needed, or that I have requested one
• There is a ticket for my issue
• Used Github auto-closing keywords in the PR description above
• Wrote a good PR/commit description
• Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
• Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
• Checked the PR locally: npm run test -- brave_browser_tests, npm run test -- brave_unit_tests, npm run lint, npm run gn_check, npm run tslint
• Ran git rebase master (if needed)

Reviewer Checklist:

• A security review is not needed, or a link to one is included in the PR description
• New files have MPL-2.0 license header
• Adequate test coverage exists to prevent regressions
• Major classes, functions and non-trivial code blocks are well-commented
• Changes in component dependencies are properly reflected in gn
• Code follows the style guide
• Test plan is specified in PR before merging

After-merge Checklist:

• The associated issue milestone is set to the smallest version that the
  changes has landed on
• All relevant documentation has been updated, for instance:
  • https://github.com/brave/brave-browser/wiki/Deviations-from-Chromium-(features-we-disable-or-remove)
  • https://github.com/brave/brave-browser/wiki/Proxy-redirected-URLs
  • https://github.com/brave/brave-browser/wiki/Fingerprinting-Protections
  • https://github.com/brave/brave-browser/wiki/Brave%E2%80%99s-Use-of-Referral-Codes
  • https://github.com/brave/brave-browser/wiki/Custom-Headers
  • https://github.com/brave/brave-browser/wiki/Web-Compatibility-Exceptions-in-Brave
  • https://github.com/brave/brave-browser/wiki/QA-Guide
  • https://github.com/brave/brave-browser/wiki/P3A

Test Plan:

Confirm visiting a selection of the following sites and then performing a search classifies the page content for supported classifier languages:

Before the change Search Engine Result Pages (SERP) were not classified. After the change SERP should be classified and the following text should appear in the console logs Classified text with the top segment as...

https://developer.mozilla.org/
https://duckduckgo.com/
https://en.wikipedia.org/
https://fireball.de/
https://github.com/
https://infogalactic.com/
https://ja.wikipedia.org/
https://search.brave.com/
https://search.yahoo.com/
https://stackoverflow.com/
https://swisscows.com/
https://twitter.com/explore/
https://uk.search.yahoo.com/
https://www.amazon.co.uk/
https://www.amazon.com/
https://www.baidu.com/
https://www.bing.com/
https://www.dogpile.com/
https://www.ecosia.org/
https://www.excite.com/
https://www.findx.com/
https://www.gigablast.com/
https://www.google.co.uk/
https://www.google.com/
https://www.lycos.com/
https://www.metacrawler.com/
https://www.mojeek.co.uk/
https://www.mojeek.com/
https://www.petalsearch.com/
https://www.qwant.com/
https://www.semanticscholar.org/
https://www.sogou.com/
https://www.startpage.com/
https://www.webcrawler.com/
https://www.wolframalpha.com/
https://www.youtube.com/
https://yandex.com/

There are unit tests to cover all sites including those that are multiilingual.

[andywillis]

Why are you still testing for the racist search-engine infogalactic? I thought we were done with that back in 2019?