feat: MV3 boilerplate #1044

meandavejustice · 2022-01-13T19:24:38Z

This is the starting point for Manifest v3 extension.

What currently works?

simple redirects from ipfs.io and dweb links.
dnslink urls are fetched and added to the DeclarativeNetRequest api ruleset for redirection.
added workflow to start testing out CI
add typescript
add prettier
add eslint
building with esbuild for ⚡ fast builds
adds boilerplate for popup

All of this is in /mv3

- adds mv3/ directory - setup fast builds with esbuild - setup prettier for formatting - setup eslint - setup typescript - adds BASIC handling of ipfs.io & dweb.link URLs to local node via declarativeNetRequest api (manifest v3) - adds BASIC handling of DNSLink requests by parsing, talking to local node via ipfs-http-client, and adding rule to declarativeNetRequest ruleset - adds boilerplate for popup

lidel

Thanks! Small asks before we merge (inline)

.github/workflows/ci.yml

mv3/addon/background.ts

mv3/package.json

mv3/addon/background.ts

meandavejustice · 2022-01-26T16:22:20Z

@lidel Comments addressed 👍

meandavejustice · 2022-02-07T23:14:04Z

@lidel Could I get yours 👀 on this again? Thanks!

- adds mv3/ directory - setup fast builds with esbuild - setup prettier for formatting - setup eslint - setup typescript - adds BASIC handling of ipfs.io & dweb.link URLs to local node via declarativeNetRequest api (manifest v3) - adds BASIC handling of DNSLink requests by parsing, talking to local node via ipfs-http-client, and adding rule to declarativeNetRequest ruleset - adds boilerplate for popup

- add welcome page - add options page - add clipboard functionality - add status updates to popup - add basic UI styling to popup

- on installed checks - on uninstalled checks - runtime checks - options/state cleanup - add firefox request handling for when they support mv3

lidel

Thank you @meandavejustice, and apologies for late review.

Dynamic rules are pretty powerful, and I think we should be able to live with them as long we add proper cache around their creation.

Left some feedback below:

⚠️ important things we should fix
- lmk if you prefer to do them in this PR or merge as-is and track them in separate issues
💭 lower priority thoughts on style and nice-to-haves

mv3/addon/background.ts

lidel · 2022-03-28T12:02:23Z

mv3/addon/background.ts

+  return url.includes("chrome://");
+}
+
+async function resolveDNSAndAddRule(domain) {


🔧 can we make resolveDNSAndAddRule return boolean to indicate if companion added a new rule or not?
That way, we could act on that hint in places like retryOnTabUpdate and reload impacted tabs immediately (find all tabs with URL matching the added rule and reload them fast enough that the user does not even see the original URL).

mv3/addon/background.ts

lidel · 2022-03-28T12:21:16Z

mv3/addon/background.ts

+          priority: 1,
+          action: {
+            type: "redirect",
+            redirect: { url: `https://dweb.link${contentPath}` },


⚠️ This needs additional tweak to stay on /ipns/ namespace (we don't want users to be redirected to status snapshot and lose future updates).

Assuming example.com has a valid dnslink pointing at /ipfs/{cid}

Opening https://example.com/foo/bar?arg=a#buzz should redirect to
https://dweb.link/ipns/example.com/foo/bar?arg=a#buzz.

(We want to preserve all params, simply replace https?:// with https://dweb.link/ipns/ prefix)

lidel · 2022-03-28T12:24:39Z

mv3/addon/background.ts

+}
+
+async function resolveDNSAndAddRule(domain) {
+  const contentPath = await client.resolve(`/ipns/${domain}`);


⚠️ this really needs a cache at some point (we can't send additional HTTP request to RPC api every time user opens new URL from already resolved domain)

I realized this is an out of date file. I will push the updates shortly with the updated branch I've been working in 🤦. Luckily for me there was some things you caught that I wasn't doing in the new branch! so it was NOT a wasted effort :)

lidel · 2022-03-28T12:36:27Z

mv3/addon/background.ts

+  });
+}
+
+async function retryOnTabUpdate(tabId, info) {


⚠️ i think we could make this smarter and redirect every public URL that returns true from IsIpfs.url test

For example, https://cloudflare-ipfs.com/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR does not get redirected to a local node because there is dynamic no rule for https://cloudflare-ipfs.com/ipfs/* – but it could with a few additional lines :)

We should detect when IsIpfs.url(info.url) returns true and then domain is different from the default public gateway (URL.host.endsWith('dweb.link')) and local one (URLhost.endsWith(localhost:8080')). In such case, we would add dynamic rule, just like we do for DNSLink. 👌

lidel · 2022-03-28T13:11:07Z

mv3/addon/rules.json

+    "action": {
+      "type": "redirect",
+      "redirect": {
+        "transform": { "scheme": "http", "host": "localhost", "port": "8080" }


💭 given that port will be something that can be changed by the user, we will most likely end up adding these as dynamic rules during startup.

lidel · 2022-03-28T13:22:34Z

.github/workflows/ci-mv3.yml

+      - name: Restore .cache
+        uses: actions/cache@v3
+        id: cache
+        with:
+          path: ${{ github.workspace }}/.cache
+          key: ${{ runner.os }}-${{ hashFiles('package*json', 'yarn.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-${{ hashFiles('package*json', 'yarn.lock') }}
+            ${{ runner.os }}-
+
+      - name: Restore node_modules
+        id: yarn-cache
+        uses: actions/cache@v3
+        with:
+          path: node_modules
+          key: ${{ runner.os }}-${{ hashFiles('package*json', 'yarn.lock') }}


⚠️ please remove yarn.lock and this cache – npm ci uses package-lock.json and ignores yarn.lock anyway (which is good, we want to try using package-lock.json and remove yarn dependency)

id expects and integer

meandavejustice · 2022-03-29T00:35:55Z

@lidel Thanks again for the review, I apologize for the branching mixup. I've got it working now correctly in this branch and addressed most of your comments. (I have two of them added to my todo list for the week). I'm going to go ahead and close this pr and open up a fresh one later on in the week. I need to add an updated list of features, and screenshots and clean up the git history.

meandavejustice added 2 commits January 10, 2022 20:07

task: add workflow for mv3 testing/build and update CODEOWNERS file

dd60790

meandavejustice requested a review from lidel as a code owner January 13, 2022 19:24

ipfs deleted a comment from welcome bot Jan 25, 2022

lidel requested changes Jan 25, 2022

View reviewed changes

task(mv3): split ci-mv3 file, add dnr types, use npm over yarn

0a9eee4

meandavejustice requested a review from lidel January 26, 2022 16:22

lidel requested review from lidel and removed request for lidel February 19, 2022 02:51

meandavejustice added 12 commits March 10, 2022 17:10

task: add workflow for mv3 testing/build and update CODEOWNERS file

9f88cea

task(mv3): split ci-mv3 file, add dnr types, use npm over yarn

09b2df4

Add functionality to popup

6cdb61c

- add welcome page - add options page - add clipboard functionality - add status updates to popup - add basic UI styling to popup

add declarativeNetRequest rule for ipfs:// urls

7c8cd5e

add missin p-memoize dependency

504299d

fix lint errors and ignore build/ dir from eslint

bbfed90

remove web-ext dep

05e4c16

add locales, audit deps, add notifications

eb927e6

fix github actions to run in mv3 working directory

eb3a59a

clear cache

747853c

restructuring addon

330f7b3

- on installed checks - on uninstalled checks - runtime checks - options/state cleanup - add firefox request handling for when they support mv3

lidel mentioned this pull request Mar 25, 2022

CI windows-tests failing #1071

Closed

lidel added 3 commits March 28, 2022 13:20

chore: cleanup CODEOWNERS

1369955

chore(ci): separate mv2 from mv3

1c15251

docs: README MV3

fdd3eee

lidel changed the title ~~MV3 Extension boilerplate~~ feat: MV3 boilerplate Mar 28, 2022

lidel requested changes Mar 28, 2022

View reviewed changes

fix(ci): ignore mv3 when running mv2

4fa6195

meandavejustice added 8 commits March 28, 2022 16:02

couple nits from pr review

e26b42c

Merge branch 'panel' into mv3

ba5ac27

remove package:addon command from build

8f3a59c

cache break

14964d3

add dynamicRulePrefix to addRuleToDynamicRuleset log

fcc0bba

remove dynamicRulePrefix

81e0939

id expects and integer

remove string casting of id and remove excess logs

299ee50

stop duplicate lookups of converted dnslink/ipfs urls

de8f2cc

meandavejustice closed this Mar 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: MV3 boilerplate #1044

feat: MV3 boilerplate #1044

meandavejustice commented Jan 13, 2022

lidel left a comment

meandavejustice commented Jan 26, 2022

meandavejustice commented Feb 7, 2022

lidel left a comment

lidel Mar 28, 2022

lidel Mar 28, 2022

lidel Mar 28, 2022

meandavejustice Mar 28, 2022

lidel Mar 28, 2022

lidel Mar 28, 2022

lidel Mar 28, 2022

meandavejustice commented Mar 29, 2022

feat: MV3 boilerplate #1044

feat: MV3 boilerplate #1044

Conversation

meandavejustice commented Jan 13, 2022

lidel left a comment

Choose a reason for hiding this comment

meandavejustice commented Jan 26, 2022

meandavejustice commented Feb 7, 2022

lidel left a comment

Choose a reason for hiding this comment

lidel Mar 28, 2022

Choose a reason for hiding this comment

lidel Mar 28, 2022

Choose a reason for hiding this comment

lidel Mar 28, 2022

Choose a reason for hiding this comment

meandavejustice Mar 28, 2022

Choose a reason for hiding this comment

lidel Mar 28, 2022

Choose a reason for hiding this comment

lidel Mar 28, 2022

Choose a reason for hiding this comment

lidel Mar 28, 2022

Choose a reason for hiding this comment

meandavejustice commented Mar 29, 2022