Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hidden renderer for fetching backup search results #27285

Merged
merged 5 commits into from
Jan 27, 2025
Merged

Conversation

DJAndries
Copy link
Collaborator

@DJAndries DJAndries commented Jan 21, 2025

Resolves brave/brave-browser#43399

Security review: https://github.com/brave/reviews/issues/1843

Submitter Checklist:

  • I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
  • There is a ticket for my issue
  • Used Github auto-closing keywords in the PR description above
  • Wrote a good PR/commit description
  • Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
  • Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
  • Checked the PR locally:
    • npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
    • npm run presubmit wiki, npm run gn_check, npm run tslint
  • Ran git rebase master (if needed)

Reviewer Checklist:

  • A security review is not needed, or a link to one is included in the PR description
  • New files have MPL-2.0 license header
  • Adequate test coverage exists to prevent regressions
  • Major classes, functions and non-trivial code blocks are well-commented
  • Changes in component dependencies are properly reflected in gn
  • Code follows the style guide
  • Test plan is specified in PR before merging

After-merge Checklist:

Test Plan:

For Brave Search fallback on Desktop:

  1. Launch browser with MITM proxy configured
  2. Access search.brave.software, click on the settings gear, click on "All settings", enable "Google fallback mixing"
  3. Close the Search tab. Open a new tab and go to https://search.brave.software/search?q=javier+likes+google+42
  4. Ensure two requests to google.com are sent. The second request should have a response body that's over 50KB.
  5. Ensure there's no "Not many great matches" notice on the search page
  6. Close the browser and ensure there is no crash

For Brave Search fallback on Android:

  1. Launch browser, no MITM proxy needed.
  2. Access search.brave.software, click on the settings gear, click on "All settings", enable "Google fallback mixing"
  3. Ensure the device is connected via ADB. Access brave://inspect, ensure the connected device is listed.
  4. Close the Brave Search tab and open a new tab, navigate to Search
  5. Click "Inspect" or "Inspect fallback" near the search.brave.software "Service Worker" listed underneath the device
  6. In the DevTools console, enter await brave.fetchBackupResults('test query', 'en', '', '', true, 0, '')
  7. Ensure a string that is larger than 50kb gets returned
  8. Browse a few web pages, ensure there are no crashes

For Web Discovery on Desktop:

  1. Launch browser with MITM proxy configured
  2. Enable Web Discovery Project in the settings
  3. Access brave://inspect, navigate to the Extensions section
  4. Click Inspect under the Brave extension
  5. In the DevTools console, enter WDP.modules['web-discovery-project'].background.webDiscoveryProject.patternsLoader.resourceWatcher.forceUpdate(), wait 10 seconds
  6. Access https://www.google.com/search?q=this+is+a+test+query
  7. Close the tab and clear the MITM proxy log
  8. Enter WDP.modules['web-discovery-project'].background.webDiscoveryProject.strictQueries.map(x=>x.tDiff=0) in the extension DevTools console, wait up to 20 seconds
  9. Ensure there are two Google requests logged in the MITM proxy, with the second one having a response body size that is over 50KB

@diracdeltas diracdeltas requested a review from bridiver January 21, 2025 17:07
@DJAndries DJAndries requested a review from a team as a code owner January 22, 2025 03:15
@DJAndries DJAndries force-pushed the gmix-fix branch 3 times, most recently from 0566421 to f112abc Compare January 22, 2025 18:12
@ShivanKaul
Copy link
Collaborator

not creating extra OTR profiles (just using storage partitions instead?),

That sounds like a great approach to me.

request->timeout_timer.Start(
FROM_HERE, kTimeout,
base::BindOnce(&BackupResultsServiceImpl::CleanupAndDispatchResult,
base::Unretained(this), request, std::nullopt));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reported by reviewdog 🐶
[semgrep] base::Unretained is most of the time unrequited, and a weak reference is better suited for secure coding.
Consider swapping Unretained for a weak reference.
base::Unretained usage may be acceptable when a callback owner is guaranteed
to be destroyed with the object base::Unretained is pointing to, for example:

- PrefChangeRegistrar
- base::*Timer
- mojo::Receiver
- any other class member destroyed when the class is deallocated


Source: https://github.com/brave/security-action/blob/main/assets/semgrep_rules/client/chromium-uaf.yaml


Cc @thypon @goodov @iefremov

pending_request->simple_url_loader->DownloadToString(
pending_request->shared_url_loader_factory.get(),
base::BindOnce(&BackupResultsServiceImpl::HandleURLLoaderResponse,
base::Unretained(this), pending_request),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reported by reviewdog 🐶
[semgrep] base::Unretained is most of the time unrequited, and a weak reference is better suited for secure coding.
Consider swapping Unretained for a weak reference.
base::Unretained usage may be acceptable when a callback owner is guaranteed
to be destroyed with the object base::Unretained is pointing to, for example:

- PrefChangeRegistrar
- base::*Timer
- mojo::Receiver
- any other class member destroyed when the class is deallocated


Source: https://github.com/brave/security-action/blob/main/assets/semgrep_rules/client/chromium-uaf.yaml


Cc @thypon @goodov @iefremov

@DJAndries DJAndries requested a review from a team as a code owner January 24, 2025 21:02
@github-actions github-actions bot added the CI/run-audit-deps Check for known npm/cargo vulnerabilities (audit_deps) label Jan 24, 2025
Copy link
Contributor

[puLL-Merge] - brave/brave-core@27285

Here's my review of the PR:

Description

This PR introduces a new BackupResultsService for handling backup search results in Brave Search. The service provides functionality to fetch search results from backup providers (like Google) in a secure way, supporting both direct fetching and full page rendering modes. This is used for Brave Search fallback mixing (GMix) and Web Discovery Project features.

Security Hotspots

  1. URL Validation: The IsBackupResultURLAllowed function whitelist for Google domains needs careful review since it restricts which domains can be used for backup results. An oversight could allow requests to malicious domains.

  2. Cookie Handling: The service creates temporary OTR (Off The Record) profiles for cookie handling. Care should be taken that these are properly cleaned up to prevent cookie leakage.

  3. Response Size Limit: The 5MB response size limit (kMaxResponseSize) needs to be properly enforced to prevent memory exhaustion attacks.

Changes

Changes

By file:

browser/brave_search/

  • Added BackupResultsService interface and implementation
  • Added BackupResultsNavigationThrottle for request validation
  • Added service factory classes and tests
  • Implemented URL validation for backup providers

components/brave_search/browser/

  • Added allowed URLs validation logic
  • Added service interface definitions
  • Added backup results handling logic

browser/extensions/api/

  • Added new Web Discovery API endpoints for backup results retrieval

Misc Changes

  • Updated DEPS file with new Web Discovery Project dependency
  • Added new build configurations and test targets
  • Updated browser tests to handle new backup results service

The PR adds significant new functionality around search backup handling while maintaining security through proper URL validation and profile isolation.```mermaid
sequenceDiagram
participant C as Client
participant BS as BackupResultsService
participant P as OTR Profile
participant G as Google Search

C->>BS: FetchBackupResults(url)
BS->>P: Create Temp Profile
alt Full Render Mode
    BS->>P: Navigate to URL
    P->>G: Initial Request
    G->>P: Redirect
    P->>G: Final Request
    G->>P: Search Results
    P->>BS: Extract HTML
else Direct Mode
    BS->>G: Direct Request
    G->>BS: Search Results
end
BS->>C: Results Callback
BS->>P: Cleanup Profile
</details>

<!-- Generated by claude-3-5-sonnet-20241022 -->

@iefremov
Copy link
Contributor

@DJAndries can you please file follow-up issues/issues ?

]
}

if (!is_android) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does this test not run on android? In a follow-up please convert it to run on android. Also fyi using !is_android around browser tests is not an alternative to adding tests to android_browser_tests.gni. There is code owner for that file specifically to enforce that new tests run on android where applicable.

Copy link
Collaborator

@bridiver bridiver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving for android-test-reviewers, but this needs follow-up to convert to PlatformBrowserTest

@DJAndries DJAndries merged commit bde574a into master Jan 27, 2025
18 checks passed
@DJAndries DJAndries deleted the gmix-fix branch January 27, 2025 16:13
@github-actions github-actions bot added this to the 1.76.x - Nightly milestone Jan 27, 2025
brave-builds added a commit that referenced this pull request Jan 27, 2025
@brave-builds
Copy link
Collaborator

Released in v1.76.41

@kjozwiak
Copy link
Member

kjozwiak commented Jan 28, 2025

Verification PASSED on Win 11 x64 using the following build(s):

Brave | 1.76.41 Chromium: 133.0.6943.27 (Official Build) nightly (64-bit)
-- | --
Revision | 2fb7a7a08ae862edc23649bc4f0d099cedf25819
OS | Windows 11 Version 24H2 (Build 26100.2605)

Test Case #1 - Brave Search fallback on Desktop

Using the STR/Cases mentioned via #27285 (comment), ensured that:

  • ensured that two requests are sent to google.com and the response body of the second request is over 50kb
  • ensured there's no Not many great matches error when searching https://search.brave.software/search?q=javier+likes+google+42
  • ensured opening/closing Brave doesn't crash the browser while https://search.brave.software/search?q=javier+likes+google+42 is opened/active tab.
Example Example Example Example
Screenshot 2025-01-28 110451 Screenshot 2025-01-28 110555 Screenshot 2025-01-28 110734 Screenshot 2025-01-28 110758

Test Case #2 - Web Discovery on Desktop

Using the STR/Cases mentioned via #27285 (comment), ensured that:

  • enabled WDP via brave://settings/search
  • visited brave://inspect and clicked on Inspect via Brave chrome-extension://mnojpmjdmbbfmejpflffifhffcmidifd/_generated_background_page.html
  • within Dev Tools via the Console, pasted the following and waited ~10s:
WDP.modules['web-discovery-project'].background.webDiscoveryProject.patternsLoader.resourceWatcher.forceUpdate()
WDP.modules['web-discovery-project'].background.webDiscoveryProject.strictQueries.map(x=>x.tDiff=0)
  • waited ~10s and ensured there was two google.com requests logged with the second being over 50kbs
Example Example Example Example
Screenshot 2025-01-28 111454 Screenshot 2025-01-28 111623 Screenshot 2025-01-28 111723 Screenshot 2025-01-28 112704

Verification PASSED on Pixel 6 Pro running Android 16 using the following build(s):

Brave | 1.76.42 Chromium: 133.0.6943.27 (Official Build) canary (64-bit)
--- | ---
Revision | d76c74413094ca56b777b7efbffe99d6d63ceb48
OS | Android 15; Build/BP22.250103.008; 35; Baklava

Test Case #1 - Brave Search fallback on Android

Using the STR/Cases mentioned via #27285 (comment), ensured that:

  • enabled Google Fallback Mixing via search.brave.software
  • connected the Android device to the machine and ensured that it was being displayed under brave://inspect
    • ensured the website and the service worker were both appearing
  • clicked on Inspect on the service worker
  • within the DevTools via console, pasted/dumped the following and ensured the response was larger than 50kbs
await brave.fetchBackupResults('test query', 'en', '', '', true, 0, '')
Example Example Example
Screenshot_20250128-122321 Screenshot 2025-01-28 122526 Screenshot 2025-01-28 123344

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/run-audit-deps Check for known npm/cargo vulnerabilities (audit_deps) needs-security-review puLL-Merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add hidden renderer for fetching backup search results