Skip to content

Latest commit

 

History

History
683 lines (622 loc) · 46.4 KB

2024-09-27-wecg-tpac.md

File metadata and controls

683 lines (622 loc) · 46.4 KB

WECG TPAC Sep 27, 2024

Agenda

Friday: https://github.com/w3c/webextensions/wiki/TPAC-2024-Coordination#friday

  • 09:00 - 10:15
    • [75 min] WebExtensions integration into Web Platform Test
  • 11:00 - 12:30
    • [90 min] Issue triage
  • 14:00 - 17:10
    • [130 min] wpt testing
  • 17:10 - 17:30
    • [20 min] Spec Discussion

Attendees

  • Rob Wu (Mozilla)
  • Christos Bacharakis (Eyeo)
  • Timothy Hatcher (Apple)
  • Kiara Rose (Apple)
  • David Johnson (Apple)
  • Elijah Sawyers (Apple)
  • Simeon Vincent (Mozilla)
  • Mukul Purohit (Microsoft)
  • Devlin Cronin (Google)
  • Carlos Jeurissen (Jeurissen Apps)
  • Oliver Dunk (Google)
  • Jordan Spivack (Capital One)
  • Casey Garland (Capital One)
  • George Henderson (Capital One)
  • Tomislav Jovanovic (Mozilla)
  • Mohamed Elgendi (Malwarebytes)
  • Panos Astithas (Google)
  • Aaron Selya (Google Chrome)

Notes

WebExtensions integration into Web Platform Test

  • [oliver] Patrick has a branch with some web-platform-tests work: https://github.com/web-platform-tests/wpt/compare/master...patrickkettner:web-platform-tests:webextensions
  • [oliver] WIP for the tests in the runtime namespace, and how to load extensions in Chrome to load an unpacked extension via the CDP. Has restrictions to connect to Chrome in a special way that WPT does not use. We should fix the Chrome command so it works in WPT.
  • [simeon] Are the changes ready to review?
  • [oliver] https://chromedevtools.github.io/devtools-protocol/tot/Extensions/#method-loadUnpacked
  • [oliver] You pass a path you want to load.
  • [simeon] This is not the base64 zip?
  • [oliver] No. I'm not sure if we need that. The driver can write to a temp file and pass the path.
  • [simeon] [tries and fails to find source to see]
  • [oliver] Should we go back to loading with the command line flag, then explore DTP / BiDi commands.
  • [simeon] Seems reasonable. We can crawl, walk, run here.
  • [oliver] Would be good to move forward, and have smaller PRs. That is about all I have for the Chrome side.
  • [simeon] Safari folks? Do we have a way to do this?
  • [kiara] We have a way we can hook up here.
  • [simeon] Would it be possible to make this available publicly now?
  • [kiara] What we can do it have some type of detection, and allow it.
  • [timothy] This is something we will likely get in STP soon.
  • [simeon] *A wild Rob appears.*
  • [rob] I'll join in person for the next meeting.
  • [simeon] Where is Firefox for integration with WPT?
  • [rob] Tom worked on it mostly. I will check. From the protocol perspective, it is all doable. We have everything we need to do it.
  • [simeon] So what are next steps?
  • [oliver] If we merge the branch Patrick has, I can maybe do that for the Chrome side. Do the WPT folks know this is coming?
  • [simeon] They are part of the Browser Tools and Testing team?
  • [panos] They are aware, I am here observing.
  • [simeon] We do have a branch in progress from Google.
  • [oliver] I can work with Patrick and update the PR. We should all look at the WebDriver BiDi patch too.
  • [rob] Does Patrick's patch build upon WebDriver-bidi?
  • [oliver] Right now using Chrome Devtools Protocol.
  • [simeon] At least with load unpack at launch we can start and switch over to dynamic loading later.
  • [oliver] What WPT does in general - I think I spoke with Maxim - they are currently not working on WebDriver-bidi but that is a P0 top priority that they are working on. Rob, did you say someone from Moz is willing to implement the BiDi PR?
  • [rob] Yes, a Moz contributor is interested in implementing this.
  • [oliver] Safari folks, you're open to this as well?
  • [timothy] We will with our folks more familiar with the WebDriver stuff to implement the classic approach - expect it to be straightforward.
  • [simeon] Next step?
  • [timothy] Test with runtime.getURL().
  • [simeon] Wanting to flesh out the shape of the tests. Maybe pair programming later today?
  • [timothy] We have not talked yet about how this should look like internally. E.g. browser.test namespace for asserts, or adopting the wpt harness.
  • [simeon] Is browser.test extension-specific? Or wpt?
  • [timothy] Extension-specific namespace that Firefox, Chrome and Safari have, with testing utilities such as assertTrue, .
  • [rob] This namespace was originally from Chrome. There are some minor differences, such as assertDeepEqual vs. checkDeepEq. We should specify more specifically what we expect these utilities to do.
  • [simeon] If we don't have a set of common capabilities, should we lean more on WPT infrastructure?
  • [timothy] The reason for considering this is that our internal tests are already writing these are already using browser.test.
  • [rob] To be clear, there is a lot of overlap between browser.test implementations. If there are differences we can fix them up, run tests and fix up failing tests if any. No external consumers of that namespace.
  • [kiara] I'd also lean towards preferring browser.test.
  • [timothy] Agree that there's enough overlap that we could use it relatively easily.
  • [kiara] How do we handle chrome.test vs. browser.test?
  • [oliver] Maybe we can just use browser.test?
  • [timothy] We have a special test mode that we can run, it adds the browser.test namespace, reduces timer constraints for browser.alarms, etc..
  • [oliver] Thinking we'd use our existing method names from the browser.test namespace and under the hood use WPT implementations.
  • [timothy] Was thinking of that too - a stub that basically does what the browser does internally.
  • [simeon] What are the next steps for browser.test?
  • [timothy] Sit down and look at the differences.
  • [rob] Is safari's browser.test part of WebKit?
  • [timothy] Yes.
  • [rob] That means all test implementations are open source. Should be easy to see what they're doing.
  • [timothy] Regarding browser vs. chrome, I expect that all of our tests will need to do globalThis.browser ??= chrome for now.
  • [rob] Would we prefer to import a shared file that includes the namespace alias? That would allow us to include other common utilities as well.
  • [timothy] I'm sure we would. Makes sense.
  • [simeon] Question about internal timers, time-based operations. How do we fake that? I assume we don't use wall clock time?
  • [timothy] We do. That's why we relax the timer constraints of the alarms API.
  • [rob] I don't think that any browsers fake the time. Running faster/slower can have side effects. In Firefox we increase / decrease the timers as needed.
  • [oliver] Looks like in some chrome tests we adjust test timers in some cases. For example, advance time by one second.
  • [rob] We have something similar in Firefox that provides more precise control in specific tests. But translating that to WPT tests is likely difficult.
  • [simeon] Anything else we need to work through?
  • [rob] Do we expect to run all assertions inside the tests? That's what WPT requires. In Firefox we have asserts that are outside the extension of the test itself. E.g. in cases where there are no extension contexts.
  • [timothy] We run most of our test logic in background scripts. If it does suspend, that's useful as well.
  • [rob] If you want to test that a background suspends after a reasonable amount of time, you cannot wait for that condition and do the check in the background script. You need another context to monitor and react.
  • [timothy] In testing mode we reduce our BG suspension time down to 3 seconds from 30 so we can exercise it more.
  • [rob] Just for that test? Imagine that could cause issues with other tests.
  • [timothy] That length seems to be a good balance between normal expected behavior and faster execution.
  • [rob] Sounds like we may want some utilities for these specific situations.
  • [timothy] Could hang a setter on browser.test.
  • [simeon] Parallelization concerns?
  • [timothy] Could impact.
  • [oliver] Could we defer those sorts of tests until we have more API tests? Could see having a new WebDriver BiDi command to change SW timeout.
  • [rob] Agree with that. We should consider these capabilities in the design phase. If we hang a lot of utilities, the may not be available everywhere.
  • [timothy] We expose the utilities everywhere.
  • [rob] We don't. In Chrome and Firefox, the test namespace is only exposed in extension contexts.
  • [timothy] They are browser utilities for us, but we expose them to the main world. They're not used outside of extensions use cases today.
  • [rob] Open question is whether we should expose them in the main world as well.
  • [oliver] Asking Devlin is probably best. If we're exposing them in WPT utilities, we can decide where we want to expose them in that interface.
  • [rob] What is the preferred direction for WPT specific utilities? Can tackle this when we hack something together.
  • [timothy] depends on how many tests we're doing. It would be ideal if the majority is drag and drop rather than find/replace from assert1 to assert2.
  • [rob] Makes sense to have the test utilities behave identically. Would also be nice to have names of similar length for formatting purposes.
  • [rob] Concrete next steps?
  • [oliver] Have some called out earlier in the notes.
  • [rob] WPT folks, any opinion on how the utility should look?
  • [panos] If things are more consistent with what you already have, would be easier to review and accept what you have. Preference would be as seamless an integration as possible, but if you get a lot of value from the current differences, that's not a deal breaker.
  • [rob] Does “seamless integration” imply following the conventions of wpt or conventions or conventions that browsers are already following?
  • [panos] Preference would be following the conventions of WPT.
  • [rob] Test APIs. Extension's browser.assert vs wpt's assert_equals.
  • [oliver] Example: https://source.chromium.org/chromium/chromium/src/+/main:chrome/test/data/extensions/api_test/omnibox/test.js;l=35;drc=dd9f87319822765342f2e9ba7e32a3034bdb3260
  • [timothy] These would already work in Safari today - I see utility in copy-pasting without rewriting or rethinking. We have 400+ tests that use this syntax, that we could easily move over without much syntax.
  • [panos] Concern is if someone familiar with web and wants to add a test to extensions, that the test APIs are different. That having said, there are already wpt tests that are different. Personally I could be convinced that extension tests can use browser.test, but others on the wpt team may differ.
  • [simeon] What would be the best way to ask?
  • [panos] Yes. This should be the consultation that happens as part of opening the PR. We have at least monthly meetings where we go over these.
  • [simeon] When are these meetings usually held?
  • [panos] Tuesday 8am pacific.
  • [timothy] We usually use 8am on Thursdays; would work for us.
  • [rob] Can you drop a link with information about joining these meetings?
  • [panos] Don't have that information on hand. We can invite any of you if you'd like. Do you want to join the next one soon, or after the PR?
  • [rob] We can work on the PR first (potentially today) and then get to you. Since you are aware of the ongoing discussions, if you have any strong opinions already, please reach out to us internally.
  • [panos] Next Tuesday at 8am we have another session. Also have a channel on Matrix, https://matrix.to/#/#wpt:matrix.org
  • [simeon] I'd suggest we aim for the session after that.
  • [panos] Next session after that is November 5th.
  • [rob] Sounds good.
  • [simeon] Would it be useful to schedule some check-ins before then?
  • [kiara] Yes.
  • [simeon] We can work out the specific times later, tentatively thinking 8AM on a day other than Thursday.
  • [rob] Any questions that might block us from prototyping in the afternoon? Safari folks, are you available to hack on this in the afternoon?
  • [timothy] Kiara and Elijah can work on it.
  • [rob] Oliver do you know if Patrick is available?
  • [oliver] Not sure. We're good to prototype, though.
  • [rob] I'll check with Tom.
  • [rob] Let's do a hackathon today; objective is to have a runtime.getURL() test running across browsers.

Issue Triage

Candidates for Discussion

Criteria:

  • Issues should be things we can discuss and make progress on in 5 - 10 minutes
  • Goal is triage, not necessarily completion or resolution
  • Candidates types
    • Determining general browser supportiveness for small - medium issues
    • Updating labels on existing issues (e.g., marking as implemented)
    • Closing out issues that are fixed or WontFix
  • Avoid
    • Discussions of entire new API namespaces (unless proposing we WontFix them)
    • Fundamental changes to the platform

Discussed issues

  • (no issue) ttsEngine
    • [oliver] Quick FYI: Updates to chrome.ttsEngine. Checking there's no interest from other vendors.
    • [devlin] Chrome has two text to speech APIs. There's chrome.tts and chrome.ttsEngine. tts invokes the speaking, ttsEngine performs the speaking. Chrome needed to provide more features for engine in order to support adding new languages on the fly. Adding this for reading mode and being added to the public API for its general utility. Callers will be able to request new languages via tts API and ttsEngine will be able to download the appropriate resources in the background. AFAIK we are the only implementers of this API. Anyone else interested in this capability?
    • [timothy] No
    • [tomislav] We are working on genAI experiments, one of them might be a ttsEngine. Probably not through this API for now.
    • [rob] Would like to follow up internally about interest in exposing this, and if yes, whether it would be through the ttsEngine extension API.
    • [devlin] Oliver, is there much functional difference between the extension and web APIs?
    • [oliver] I don't know off the top of my head.
    • [devlin] ttsEngine might make sense in the world where other browsers want to enable custom voices.
    • [timothy] May not in a world with Web Speech?
    • [devlin] TTS engine may make sense, but tts may not.
    • [timothy] is this used by many extensions?
    • [devlin] No, there aren't many but the ones that are there are important.
    • [carlos' computer] TEST.
    • [carlos] The web speech API works in a service worker.
  • Issue 683: Feature request. More granular Extension's hotkeys scope
    • [devlin] No. I don't think that scoping to tabs makes as much sense to hotkeys in general. Content scripts would probably make more sense. Do think we should do more with commands: toggling enabled state, allowing commands to be defined outside the manifest, etc. But this capability seems like it would have relatively little utility for a good amount more complexity.
    • [oliver] Concerned about UI complexity?
    • [devlin] Agreed.
    • [timothy] Agree that commands could use more love. Firefox has some capabilities like the ability to change shortcuts at runtime.
    • [rob] … I see use cases such as being able to invoke even when tab content area isn't focused.
    • [oliver] Everyone in favor of closing when we have a comment?
    • [timothy] Yes.
    • [devlin] Yes.
    • [rob] Should we address my last comment?
    • [devlin] Would make sense to expose an API, isCommandEnabled.
    • [rob] User could customize the commands.
    • [devlin] If there's a single extension that uses ctrl+cmd+shift+g users can figure that out.
    • [rob] Not concerned about that.
    • [carlos] Seems more user facing to solve in the browser UI rather than in extensions.
    • [timothy] We've solved more or less in browser UI.
    • [devlin] Letting an extension know if a command is enabled makes sense
    • [rob] Should we expose in an event?
    • [devlin] Would make sense. Should table for now.
    • Resolution: Close
    • devlin to add comment and close out
  • Issue 639: i18n.getMessage() pluralization
    • [devlin] Solved with message format 2, right?
    • [timothy] Agreed.
    • [devlin] Even messages TC39 proposal for Intl.MessageFormat.
    • [oliver] Close in favor of issue 698?
    • [devlin] Yes.
    • Resolution: Close, the work will be tracked by issue 698.
  • Issue 640: Proposal: Remove multiple bookmarks in bookmarks.remove()
    • [devlin] No objections. Not fond of options (e.g. string | strings[]) but makes sense in this context. Performance benefits and race condition benefits to making this a single operation.
    • [rob] What would the race condition be?
    • [devlin] If it's an atomic operation you can be assured that you will remove all of them at once.
    • [rob] Would we specify this to be atomic? Either all or nothing.
    • [devlin] That's not a part of triage ;) Good question. Inclined to say we'll do our best and it's okay to have partial removal. If you ask to remove 50 and we only remove 49 because one was already removed, that's acceptable.
    • [timothy] We don't support bookmarks yet, but this would be preferable for us as we use a DB for bookmarks.
    • Resolution: Everyone supportive of the ability to remove multiple bookmarks at once.
  • Issue 615: userstyles support
    • [rob] TL;DR is request to add support for registering CSS in the scripting/userScripts API.
    • [timothy] duplicating something already in scripting...
    • [devlin] So we have file support for registering stylesheets in scripting.registerContentScripts, but not arbitrary css code. Supportive of adding this, but through the scripting API. We do not need to add it to the userScripts namespace since the scripting namespace already accepts arbitrary CSS code in scripting.insertCSS.
    • [rob] So to recap, we are all supportive of supporting arbitrary stylesheets through scripting.registerContentScripts, but not in the userScripts API.
    • [devlin] Correct.
    • [timothy] Yes.
    • Resolution: Add support for registering user styles to the scripting.registerContentScripts() method.
  • Issue 624: Proposal: getLeafTarget() method
    • [oliver] When discussed, both Firefox and Safari were supportive. I chatted with Devlin, and wonder whether this could be added to the web side, with an extension-only option to opt in to closed shadow roots. I spoke to one of the Dominics on the Chrome side, and there is an expectation that we would encounter strong opposition to adding this to the web platform. Someone from Apple had similar reservations.
    • [devlin] Do we need to add something? Is there a reason where we want event.getComposedPath() to not include shadow roots?
    • [rob] Web compatibility. A web API behaving differently in a web page vs content script could be surprising.
    • [devlin] That seems okay to me.
    • [oliver] Returning more elements than expected seems okay.
    • [devlin] We should discuss this more. Making this work with compose path would be preferable.
    • [rob] Should we require a manifest key for that? This is a strong deviation from the web platform, which assumes that shadow DOM is generally invisible.
    • [devlin] Do you feel strongly about piercing shadow roots?
    • [rob] Don't want to set the precedent of changing behavior of existing web APIs. Documentation story can become very complicated. If we have a manifest key, at least we have something specific we can point to that changes web APIs in a specific way.
    • [devlin] Okay, so we would need some sort of mechanism.
    • [rob] Yes. And I'd prefer to have a flag to change the behavior.
    • [oliver] I'd feel weird about having an extension changing web behavior.
    • [devlin] Yeah, would prefer a property passed into the function call. Agree with Timothy that we collectively need to get over hesitation of extension specific functionality.
    • Action item: Oliver to follow up with Chrome to see the preferred approach.
  • Issue 658: Proposal: API to allow incognito access
    • [devlin] No. I can see the utility and understand the rationale for the request, but this is a capability that we want to keep behind user initiation and to make it harder for extensions to socially engineer access to.
    • [timothy] Agreed.
    • [simeon] Checking if you have permission?
    • [devlin] We already have a way to do that.
    • [devlin] Any objections to closing issue 658 as wontfix?
    • [rob] No objection.
    • Resolution: Close.
    • Tom to close as WontFix with rationale
  • Issue 657: [MV3] Clarify browser inconsistency for temporary host permissions granted on extension click
    • [simeon] Short version: we discussed but have not aligned on click-to-script behavior, the ability to restrict host permissions to extensions and only running scripts in response to interaction with the extension button. Like activeTab but different (forcing extension to be activeTab). What do other browsers think?
    • [timothy] that also reloads?
    • [devlin] Depends. When you withhold host permissions, things that would have injected don't. If you have things that inject at document start or end, we can't retroactively inject at those times. If you only have scripts that inject at document idle then we can inject at any moment.
    • [timothy] We inject everything after the host permissions have been granted, without forcing to reload.
    • [tomislav] We do the same. If a MV3 extension is not granted permissions, and the user enabled access, scripts are injected.
    • [devlin] Main reason for this behavior is that we don't expect extensions to accommodate that behavior. We don't think that most extensions would do that. The only way to not have the extension break is to not do that.
    • [simeon] When this capability was introduced, the concept of a requested host permission not being granted was foreign. Chrome had to take this approach then.
    • [tomislav] And that predates Safari WebExtensions and Firefox's MV3.
    • [timothy] We haven't had complaints about this.
    • [rob] We haven't either.
    • [devlin] I'd push back on this as this issue specifically says they need document start and they'd prefer Chrome's behavior.
    • [rob] Majority of extensions don't care.
    • [devlin] I don't think that's true. The default is idle. If you're specifying document start, there's a strong change there's a reason you have requested that point in the lifecycle.
    • [rob] The extension could trigger a reload if they really wanted to. Every approach has disadvantages. Would be nice if extensions could specify their desired behavior in the manifest: only inject if at the run_at time, allow later injection, reload after permission change.
    • [timothy] Supportive of a manifest key to specify the desired behavior.
    • [devlin] Generally hesitant to introduce that level of behavior control to extensions. Not necessarily fundamentally opposed, but don't like it.
    • [tomislav] If extensions prefer the behavior of Chrome, the extension could show a popup and tell the user to reload. The developer is in the best position to know their intent and prompt.
    • [devlin] Expect that a single digit percentage of developers are considering these edge cases. They have already provided an explicit signal that they want to run early.
    • [tomislav] Maybe they just want to run as early as possible, not explicitly at that moment.
    • [devlin] Not entirely opposed to manifest key, but would not expect much usage either.
    • [devlin] Extension could code to not rely on document_start.
    • [tomislav] Maybe not rely.
    • [devlin] I don't think so, but we don't have a way to prove this one way or the other.
    • [rob] As an extension dev I often want to run as early as possible but it's okay if I don't, running later is better than nothing.
    • [devlin] I think we have both groups: extensions that want to run early and its okay if they don't, and extensions that will break if they don't run before anything else. On the Chrome side, we want to optimize for not breaking the extensions as much as possible. We want users to be able to withhold host permissions.
    • [rob] On the other side this breaks UX where the user installs the extension and the page doesn't behave as expected.
    • [simeon] User could also do what they do and then choose to reload later.
    • [timothy] We communicate to devs that they cannot rely on document_start.
    • [devlin] At some point this is a browser decision and it's okay to not align.
    • [devlin] In Chrome, activeTab applies while you are in the same origin while in the tab.
      • [rob] Does this mean that the whole extension has access to the origin while the tab has not made a cross-origin navigation yet?
      • [devlin] It has access to that origin in that tab.
      • [tomislav] For us that's the concept of active script and active tab permissions.
    • [tomislav] That is the behavior we aim for in Firefox.
    • [timothy] We match the Chrome behavior.
    • Resolution: Stickiness of “activeTab” is aligned. Different viewpoints on the tradeoffs related to script injection at document_start. No consensus on providing a way for an extension to specify the desired behavior in the manifest.
  • Issue 698: MessageFormat 2 support
    • [oliver] Just confirming we are all happy with supportive labels here :)
    • [devlin] Yes.
    • [rob] Firefox also supportive.
    • [timothy] Safari too.
    • Resolution: Everyone in favor of MessageFormat 2 support.
  • Issue 645: Inconsistency: Commands API support for Cmd+Alt on Apple OSs
    • [oliver] Had some discussion in Chromium. Someone at Microsoft is working on this.
    • [devlin] If you specify Cmd+Alt as a mac specific shortcut but not the default, I think it works. Because it's a platform specific entry and no internal mapping of Ctrl to Command is necessary, I think this is handled correctly on Mac.
    • [oliver] Just tested, doesn't work.
    • [devlin] Well, we should fix that. Supportive of allowing mac-specific fields in the mac-specific fields (not in linux, windows, etc).
    • Resolution: Chrome in favor of fixing (Firefox and Safari already implemented).
  • Issue 503: Ability to detect browser variant / fork
    • [rob] Simeon, you reached out to other browsers.
    • [simeon] They did not care.
    • [oliver] Is Chrome supportive of runtime.getBrowserInfo?
    • [devlin] In general returning names is fine. This relies on forks updating the name, not sure if they would do that.
    • [simeon] I reached out to Arc and they didn't have an appetite for it.
    • [devlin] Many forks deliberately mimic Chrome.
    • [carlos] My impression is that this is less of an issue for extensions.
    • [simeon] Could we take the name from the about page and return that?
    • [devlin] No. We …
    • [simeon] I assume there's a well-known string that identifies the browser.
    • [timothy] What Simeon is saying is whether it can return something dynamically that is already defined in a fork.
    • [aaron] There are also malicious reasons why some forks would want to pretend to be Chrome and Chromium as much as possible.
    • [devlin] Would like to not return strings since it can be localized. What we could do is to return Chrome in Google Chrome and something else in Chromium.
    • [timothy] We have a desire to return a browser name as well. Ideally something that forks would automatically expose the correct value.
    • [rob] We cannot do perfect. What we can do is to have Chrome and Chromium, where forks can customize if needed. Firefox and Safari can also do something.
    • [carlos] Would like an easier way to differentiate between browsers than parsing the extension's URL.
    • [timothy] Why can it not be async.
    • [simeon] Every extension API is async by default and only made sync when there's a compelling reason.
    • [rob] On the web, feature detection sometimes requires synchronous execution. Do we want to support those cases?
    • [devlin] If we are just looking at identifying which browser it is, we can make it sync.
    • [timothy] We'd just have to send over that info synchronously on renderer start.
    • [rob] In Firefox also feasible. Question is not whether we can, but whether we want/should.
    • [devlin] I am okay with this being sync. I don't expect adverse impacts on future implementations.
    • [rob] Since the existing getBrowserInfo() API is async, we would then have to introduce a new method.
    • [devlin] Is getBrowserInfo() new, and widely used?
    • [rob] getBrowserInfo() has been around for a long while and is used by at least hundreds of extensions in Firefox.
    • [carlos] Name and vendor seems like they are likely to be different in forks. In other browsers it may make sense to have a hardcoded string for “Firefox” or “Safari”
    • [devlin] Would that not be addressed by the name field?
    • [timothy] No, Firefox Night, Firefox Developer Edition, etc. Vendor might make sense across forks.
    • [devlin] that could change as well.
    • [simeon] In the web equivalent of this API, “brands” can returns multiple fields. For example, “vivaldi” and “chromium”.
    • [rob] Should we consult others that have more experience with “brands” on whether we should return a single or multiple values?
    • [devlin] Could, but what benefit?
    • [rob] For example, something that works in Chrome but not chromium.
    • [devlin] Extensions could use navigate.userAgentData
    • [rob] Right, in that case we can just return what it does not provide yet.
    • [rob] Do we want this to be a method or a constant?
    • [timothy] Method seems fine.
    • [devlin] I can see it either way. Constant matches navigator.userAgentData better.
    • [timothy] True.
    • [devlin] Could also enable no name collision with Firefox's implementation, e.g. runtime.BROWSER_INFO (or runtime.browserInfo).
    • Resolution: Everyone supportive of a constant exposing the info, next step is for someone to author a proposal.
  • Issue 653: API for extensions to exclusion/deny list their content scripts
    • [devlin] This functionality is already provided through excludeMatches and excludeGlobs
    • [oliver] The possible solution in the issue mentions (1) injecting into existing tabs and (2) unloading when a content script becomes excluded. We don't want to do (1) and (2) is not possible.
    • [carlos] When content scripts are injected is different across browsers, at install time (whether to apply to already-injected tabs).
    • [rob] For that topic we already have an issue tracking this (issue 617). How does that relate to the current topic?
    • [devlin] Reason for excludeMatches being insufficient is that Chrome does not inject in existing tabs. We are unlikely to change that.
    • [timothy] We do. Open to a property to control this, but in general would prefer the default to be inject into all tabs.
    • [rob] Could add an option to do this, as suggested by 617.
    • [devlin] Since 617 exists, inclined to close this.
    • Resolution: Close this issue in favor of 617.
  • Issue 617: manifest key to enable automatic injection of content scripts after installation/update
    • [devlin] Not opposed.
    • [rob] Meaning neutral?
    • [devlin] Yes.

wpt testing

Attendees

  • Rob Wu (Mozilla)
  • Simeon Vincent (Mozilla)
  • Tomislav Jovanovic (Mozilla)
  • Kiara Rose (Apple)
  • Elijah Sawyers (Apple)
  • Devlin Cronin (Google)
  • Oliver Dunk (Google)
  • Carlos Jeurissen (Jeurissen Apps)

wpt summary

This is a summary of the agreed-upon direction from the wpt discussion, on the topic of the steps towards testing extension APIs in wpt across browsers (Firefox, Chrome, Safari).

  • We will use browser.test (probably with globalThis.browser??=chrome shim) in the extension contexts for now
  • This will require exposing browser.test in these contexts in Chrome
    • We can do this with a new commandline switch, only respected if the browser is automated
    • (Firefox and Safari already expose the browser.test namespace)
  • We need to introduce a new browser.test API to signal that the context is in a WPT (e.g., browser.test.setIsWebPlatformTest()). This then adjusts the behavior of test.notifyPass(), notifyFail(), succeed(), etc to pass the result to the WPT page rather than signaling in another way.
    • This is per context, because it enables for easier initial implementation. Most tests would likely be one JS file anyway. When multiple JS contexts are involved (e.g. content scripts, then each context needs to call this method again.
  • We will introduce an extension_test style test shim (in a JS file resource served from wpt) which will load the extension, wait for results
  • Future areas of investigation and investment include:
    • Using WPT assertions and success / failure indications directly. We are not doing this in the initial version because there's no good way to pass test bindings to the extension context (e.g. the test_driver object)
    • Generated extensions to reduce boilerplate, require only one file per test (instead of multiple)

wpt discussion

Note: this session lasted 130 minutes and the minutes do not capture the full discussion due to the pace of the conversation and the fact that the scribes were busily participating in the meeting itself. Parts of the discussion are captured below, and a summary is available above in “wpt summary”

  • [rob] At least one from every vendor to work on getting wpt running.
  • [tomislav] wpt may test service worker but always requires a web page to be present.
  • [simeon] IIRC, Patrick's prototype has a web page where.
  • [tomislav] Should we try to plug in the same system that they have - register for test harness events, shuffle it, and post to the main world? Many wpt tests have custom asserts / test functions. They are not strongly enforcing the use of wpt only utilities.
  • [rob] They
  • [tomislav] So do we want to use the existing browser.test namespace?
  • [kiara] Yes.
  • [devlin] Yes.
  • [rob] Complication: browser.test is only available in extension contexts in Chrome, but it's available in the main world in Safari and Firefox.
  • [oliver] Before that, the browser needs a way to communicate test results back to WPT.
  • [rob] I think browser.test comes first.
  • [oliver] Perhaps in parallel.
  • [devlin] Perhaps we should first talk about what we want our WPT tests to look like and work backwards from there.
  • [rob] Do you mean an empty JS or HTML file?
  • [devlin] Let's say we start with a given extension. In Chrome I'd do something like:
constexpr char kBackgroundJs[] =
    R"(const id = 'a'.repeat(32);
       chrome.test.assertEq(id, chrome.runtime.id);
       chrome.test.notifyPass();)";

IN_PROC_BROWSER_TEST_F(Foo, Foo) {
  TestExtensionDir test_dir;
  test_dir.WriteManifest(kManifest); // defined elsewhere
  test_dir.WriteFile(FILE_PATH_LITERAL("background.js"), kBackgoundJs);
  ASSERT_TRUE(RunExtensionTest(test_dir.UnpackedPath());
}
  • [devlin] In Chrome, that would be the entirety of our test file.
  • [rob] I like minimizing the boilerplate.
  • [devlin] WPT won't look like the above block as it uses Chrome- and Google-specific concepts.
// HTML
<!doctype html>
<html>
  <meta extensions="path/to/extension/dir">
</html>

//path/to/extension/dir/manifest.json
{
  "manifest_version": 3,
  "version": "0.1",
  "name": "runtime.id test",
  "background": {"service_worker": "background.js"}
}

//path/to/extension/dir/background.js
const id = 'a'.repeat(32);
chrome.test.assertEq(id, chrome.runtime.id);
chrome.test.notifyPass();

chrome.tabs.onCreated.addListener((tab) => {
  chrome.test.notifyPass();
});
chrome.tabs.create({url: 'http://exaple.com'});
  • [rob] That could work. I was thinking of something different.
const id = 'a'.repeat(32);
chrome.test.assertDeepEq({ ... }, await browser.runtime.getBrowserInfo());
chrome.test.notifyPass();
  • [devlin] Shouldn't require notifyPass() at the end. For flexible code paths, it shouldn't be related to a specific function call or something at the end of the file.
  • [rob] There's no explicit notification.
let done = new Promise(resolve => {
  chrome.tabs.onCreated.addListener((tab) => {
    resolve();
  }
});
chrome.tabs.create({url: 'http://exaple.com'});
await done;
chrome.test.notifyPass();
  • [devlin] Ugly
  • [rob] Yes. Just discussing options.
  • [devlin] That's not a linear flow. Line 3 executes after line 6.
  • [rob] Semantics. Point is that the test file could be JS that defines a set of steps.
  • [devlin] Don't know how many cases we wouldn't care about the extension context.
  • [rob] By default we could assume the background and allow specific overrides. One complication is that in MV3 the background context could terminate, so that may not be an ideal default.
  • [devlin] There may be too much customization that needs to happen for each test.
  • [rob] The easiest way to load a test might be to declare a background key in the manifest. Is that right?
  • [devlin] Depends on the test. Some in a background, some in an extension page, some in a content script.
  • [tomislav] Patrick's test is modeled after basic WPT cases. WPT has other examples where there's a worker or other specialized context. I'm suggesting we start with a boilerplate-full approach that enables us to cover all of the use cases we want. In the future we can optimize. Search for .worker.js for examples.
  • [rob] I don't see many examples of this.
  • [tomislav] Most are .any.js.
  • [rob] Example with async code and assertions: https://searchfox.org/mozilla-central/rev/837f3a1ff2622c8303750c35d84bdc41a5cd079c/testing/web-platform/tests/IndexedDB/blob-valid-after-deletion.any.js
  • [rob] Are Google and Apple onboard on such a syntax? I recall Apple expressing the desire for extension test resources to be put in the source tree.
  • [kiara] The proposed single-file syntax looks good.
  • [devlin] Something like this?

Generated file version: // test/runtime.js

//// Might need:
// META: script-"support.js"
// META: permissions="foo,bar"
// META: script_type="service_worker"
//// Or, if you want:
// META: manifest="manifest.json"

// In the future, we could avoid having extension_test() and build this support
// into the WPT runner, but for now, we essentially polyfill it.
extension_test(
  function runtimeId() {
    assert_eq('a'.repeat(32), chrome.runtime.id);
  },
  function someOtherTest() {
    ...
  },
);

-> This generates an extension with a manifest like:

{
  "name": "test",
  "manifest_version": 3,
  "version": "0.1",
  "permissions": ["foo", "bar"],
  "background": {"service_worker": "service_worker.js"}
}

And service_worker.js contains the test file above

Non-generated file version:

test/runtime.js

// META: script-"support.js"
extension_test('path/to/extension');

test/manifest.json

{
  "name": "test",
  "manifest_version": 3,
  "version": "0.1",
  "permissions": ["foo", "bar"],
  "background": {"service_worker": "runtime_worker.js"}
}

test/runtime_worker.js

chrome.test.setIsWPT(true);

chrome.test.runTests([
  function runtimeId() {
    chrome.test.assertEq('a'.repeat(32), chrome.runtime.id);
  },
  function someOtherTest() {
    ...
  },
]);
  • [oliver] Rather than script_type="service_worker", that would be implied by the filename.
  • [tomislav] Could be, yes. Depending on where you put this example, e.g. if you use .service_worker.js that's one thing, but if you use .any (or appropriate extension-specific variant) then that
  • (... long discussion…)
  • [devlin] So if the wpt test runner loads the script and executes as an extension, then we would not need a separate extension_test function.
  • [rob] Example with plain “promise_test”. Also shows parametrization of tests: https://searchfox.org/mozilla-central/rev/837f3a1ff2622c8303750c35d84bdc41a5cd079c/testing/web-platform/tests/fetch/api/cors/cors-basic.any.js
  • [devlin] In this case we're looking at modifying the wpt core runner to support this. How willing are they willing to modify this?
  • [tomislav] We first need to show that we have all browser vendors participating in getting this up and running with tests before asking for any core WPT changes.

// support.js

// Generation version:
function extension_test(...tests) {
  // 1. Read metatags from script file
  // 2. Generate Manifest content based on metatags
  // 3. Create temp directory                               <-- Is this possible?
  // 4. Write Manifest content to temp_dir/manifest.json    <-- Is this possible?
  // 5. Write service_worker.js (in this case) with         <-- Is this possible?
  //    stringified `tests`.
  // 6. Load extension
  // 7. Wait for signal of done-ness                        <-- What does this look like?
}

// Non-generation version:
function extension_test(path) {
  // 1. Load extension from `path`
  await test_driver.install_unpacked_extension(path);
  window.addEventListener("message", e => { // or CustomEvent, whatever.
    // do something with e.data received from the browser.
    // browser sends data here upon calling
    // browser.test.notifyWPT() in the extension.
  }, { once: true });
}
  • [devlin] Want to drill a little into signaling done-ness.
  • [tomislav] Don't recall offhand.
  • [devlin] Extension would have to signal and pass back to WPT runner, right?
  • [tomislav] (thinking) yes
  • [rob] This would generate a lot of files. Another approach is to have unpacked extensions in a directory; each test can be a separate js file (with a corresponding auto-generated HTML page), and tests are loaded e.g. in browser tabs or iframes.
  • [devlin] If we have the full extension in the repo, we don't need to generate anything. We just load the extension and wait for done-ness.
  • [rob] But something needs to inject the WPT test utilities in the test contexts.
  • [devlin] Initiating load – we have the ability to load the extension in that we (will all) have a WebDriver loader.
  • [oliver] Here's an example of an any.js: https://searchfox.org/mozilla-central/source/testing/web-platform/tests/xhr/formdata/has.any.js
    • view-source:https://wpt.live/xhr/formdata/has.any.worker.html
  • (.... long discussion…)
  • [rob] browser.test.notifyPass() could instruct the browser to look up the wpt document and send the results there.
  • []
  • [devlin] What we want to do is to signal at the start of the extension code that this is an extension test, and that modifies the behavior of the test API so that assertXX/../notifyPass/notifyFail would forward everything to the wpt runner.
  • [devlin] See no issues on the Chrome side to do this.
  • [rob] Same.
  • [kiara] Same.
  • [tomislav] The above browser.test hookup with the browser are not needed for today's prototype - we can implement everything independent of the browser.
  • [devlin] How browser.test bindings expose to release version of Chrome.
    • Tentative proposal:
      • New command line switch (--enable-chrome-test-api)
      • Only works if the browser is automated
  • [rob] Kiara, do you see this all as feasibly implementable in Safari? Specifically collecting assertions etc. in the browser.test namespace.
  • [kiara] Yes.
  • [devlin]
  • [rob] Could expose <extension-origin>/_wpt/ that forwards any requests to a wpt server.
  • [devlin] If needed, we could.
  • [oilver] https://web-platform-tests.org/writing-tests/testharness-api.html#external-api
  • []
  • [oliver] Do we use browser.test or chrome.test?
  • [devlin] Chrome doesn't have browser, don't want to block on that. Could be solved by adding browser shim at the top of each JS file.
  • Summaries
    • (see wpt summary at the top of the wpt testing section)

Spec Discussion

  • (No minutes for this section, only the conclusions are recorded in the notes)
  • Principles
    • We should specify things that are the same between browsers already (not wishful thinking)
  • Agreed upon candidates for specification
  • Concepts
    • Extension Events (including the API)
    • Maybe messaging, but with explicit gaps in e.g. message serialization
    • Match patterns
    • Content scripts
  • APIs
    • Permissions (this is relatively similar between browsers)
    • Runtime (much of this is shared between browsers)
      • runtime.id and runtime.getURL()
      • not runtime.sendMessage / runtime.onMessage due to subtle differences (e.g. structured cloning vs JSON serialization).
    • Commands (commands manifest key, commands.getAll(), commands.onCommand)
  • Considered but rejected
    • alarms (subtle timing differences, see existing WECG issues)
    • dom