Decide and implement unit testing framework

[melink14]

Probably use Jasmine with karma?

At first should add test harness and a couple of tests as proof of concept.

(maybe add some incremental coverage measurements)

[melink14]

codecov.io free for open source
https://codecov.io/

[melink14]

Browserstack for running on browsers in the cloud and some extension tips:
https://www.browserstack.com/guide/test-chrome-extensions-in-selenium

[melink14]

https://ianwalter.dev/running-selenium-webdriver-tests-using-github-actions/

[melink14]

https://developers.google.com/web/updates/2017/06/headless-karma-mocha-chai

[melink14]

https://github.com/microsoft/tsyringe

Especially for testing with modules, we may want dependency injection for easier stubbing.

[melink14]

https://github.com/acvetkov/sinon-chrome

[melink14]

Started working on this for #366 since it should have a test.

Currently using a karma + jasmine with headless chrome as per #159 (comment) minus mocha

For typescript, I had tried karma-typescript plugin but it didn't work with es6 well so instead I switched to using karma-webpack since I already use that for production.

As part of this, I also ended up making a separate tsconfig for ts-node so that I could have advanced es6 enabled for my code and use legacy modules for node TS files (configs and update-db). I'll probably split that part out since I don't think it's related to getting testing working. It will be useful if it allows me to use top level awaits though!

In my attempts to get a idiomatic setup and everything working I took a circuitous path but the following resources were most helpful:

• https://developerlife.com/2019/07/06/starter-project-typescript-karma-jasmine-webpack/ had good tips on sharing webpack config though it wasn't enough to get line numbers or debugging working.
• https://github.com/Janne252/karma-webpack-typescript-coverage-vscode-debug covers my usecase 100% and the launch.json was especially nice for getting vscode debugging working. There's a recent bug that I had to debug myself but the advice here was great.
• I couldn't get chrome.* APIs to load early enough for the longest time until I found a karma plugin that did it. I ended up using somebody's fork with recent updates but will probably fork it myself for more control given it still has some out of date packages. Maybe I'll publish it for others if I think it's helpful. I still haven't tried to use the stubs so I could run into problems later.
• I like using Typescript configuration files (autocomplete!) and karma allows that but it's default types don't allow plugin configuration (for webpack in this case). I got it working with a combination of a helpful related gist and general Googling on how to use Omit to replace an interface property with a more general version.
• Linking to Window's chrome binary didn't work well, it was slow and unstable. I ended up installing chrome into WSL directly after reading some SO posts or github issues. (Example)

Things seem to work as well as I could expect now for unit testing though I still need to write a real working test.

Other things to come later is full integration test as well as code coverage via something like codecov.io or just native karma coverage plugin (combined with github action etc)

[melink14]

Update:

This style guide for jasmine is pretty good except for the examples moving setup into beforeEach. Though if the describes are short maybe it would be interesting.

I decided to not use the sinon chrome stuff since the jasmine integration didn't work well. Instead, I made my own chrome fake with Jasmine spys for the functions. It can be easily expanded as tests are added.

[melink14]

Updates:

I added a spy reset method to my fake chrome since otherwise it just accumulates method call recordings.

It seems I didn't need any specific sourcemap stuff in my launch.json for some reason. Perhaps due to inline source maps?

I feel like I don't need webpack since I can use ES modules directly but I'd need something to extract the node deps like lit into something that can be used easily! Maybe snowpack can be looked at again now that I'm a bit more knowledable.

[melink14]

After doing all of this I decided to try out snowpack for the build tool and it worked better than expected and then I followed that thread to get a matching test setup as well since it's always better for tests to match production as much as possible.

The recommended test runner is @web/test-runner which is new but also built for modern web. I came across some problems trying to use it:

• Default chrome launcher didn't work. Required --disable-gpu for some reason.
• content-scripts can't be modules easily and so my test only export in rikaicontent broke when importing to chrome. I fixed this with snowpack-plugin-replace deleting that line in prod configuration.
• It took a few attempts to get the test runner to find my tests due to how my snowpack was configured. It turned out I needed a custom snowpack mount, mapping extension directory to the root of my output directory.
• It had no good spec test result reporter so I embedded one from the relevant github issue.

On the other hand after that most things worked directly:

• vscode debugger connected and was able to work with test runner using some launch config after turning on snowpack sourcemap option.
• @web/test-runner works best with mocha/chai so I migrated my tests to that but tests themselves were almost identical.
• chai easily works with sinon so I switched to using sinon-chrome directly and removed my custom chrome fakes.
• In order to install fake chrome earlier, the setup was to use a custom HTML fragment as opposed to karma's file config but its more explcit and worked out of the box.

The final result is a fast test and useful test runner.

[melink14]

Future ideas:

Create medium sized unit tests by using connecting content script to background scripts with custom sendMessage fake.

[melink14]

🎉 This issue has been resolved in version 2.0.0 🎉

The release is available on:

• Chrome Web Store
• GitHub release

Your semantic-release bot 📦🚀