Dart API: Support custom importers

[ranquild]

In Dart ecosystem there are 2 primary build systems:

1. https://pub.dartlang.org/packages/build
2. https://pub.dartlang.org/packages/barback

As in the docs, It should be noted that you should never touch the file system directly. Go through the buildStep#readAsString and buildStep#writeAsString methods in order to read and write assets.. The main thing here is that build system creates graph of assets dependencies for incremental compilation and that's why directly using of file system is prohibited.

What I suggest is to add file system abstraction layer so there will be different implementation for direct file system access and access of assets in build system steps. This can be interface in sass package itself, or, better, you can use https://pub.dartlang.org/packages/file and accept FileSystem instance in compile method and use LocalFileSystem as default. This also eliminates need for compileString function because if someone needs to compile sass from source in memory MemoryFileSystem can be used for this.

If you don't have time for implementing this yourself just guide me in right direction and I will do this. This is very important for me because lack of this abstraction do not allow me to use this package in any of dart projects in my company. If abstraction will be introduced, I will publish packages like sass_build and sass_backback that can be used to directly embed sass in dart build systems.

[nex3]

I'd like to support this by providing an importer interface similar to the one supported by Ruby Sass and node-sass. An importer is effectively just a function that maps URLs to Sass strings, although in practice the existing importer APIs are more complex than that, so we'd need to do some design to figure out what's right for our use-case.

[nex3]

One thing that's important to consider for this API is the issue of canonicalization. Right now, Dart Sass uses a map with resolved file: URL keys to avoid re-parsing stylesheets within a given invocation. This is important for performance, but if we fudge it in edge cases it's not a huge problem. However, one of the goals of 4.0's import system is to avoid evaluating a stylesheet more than once. This makes canonicalization visible in the compiler's behavior, not just its performance, which means we'll need to be careful to do the right thing in all cases.

Ideally, any imports which point to some intuitive notion of "the same place" will share a module. Cases to consider include:

1. Imports with equivalent but textually different URLs (foo/bar.scss versus foo/./bar.scss).
2. Imports with different levels of explicitness (foo/bar versus foo/bar.scss; foo/bar.scss versus foo/_bar.scss).
3. Imports that may or may not be absolute (/home/nweiz/myapp/sass/foo.scss versus foo.scss).
4. Relative imports versus imports from the load path (bar.scss from mylib/foo.scss versus mylib/bar.scss from the load path).
5. Imports from different load paths (bar.scss from a load path that points to /usr/lib/mylib versus mylib/bar.scss from a load path that points to /usr/lib).
6. Imports from different importers (package:mylib/foo.scss versus foo.scss from a relative import or a load path).

One option I considered was to add a method to Importer, Uri canonicalize(Uri url), that would canonicalize a URL before passing it to the importer to load. This covers most of the purely path-based use-cases, since the filesystem importer could return an absolute canonicalized file: URL. However, it's not a great fit for case 6. The package importer could canonicalize to file: URLs, but then that raises the question of which URLs we use for stack traces and error messages. We definitely want to show users package: URLs, since those are what they understand; but we don't necessarily want to show them the literal URLs they imported, because they may not include the full file extensions. We need some intermediate representation of human-friendly but not fully-canonicalized URLs.

We also want to require that a given canonical URL always loads the same stylesheet, even across importers. This is a difficult guarantee to enforce with heterogeneous importers that may be written without knowledge of one another. It's possible we should rethink the structure entirely: rather than following Ruby Sass's example of having a single Importer class that resolves imports as-is, we could have two classes: ImportResolvers that canonicalize imported URLs, and ImportLoaders that load those URLs. We'd probably want to associate each loader one-to-one with a URL scheme so we could easily look the right one up for a given canonical URL.

That might most directly represent the underlying semantics, but it also adds a lot of complexity. What's worse, the complexity won't be contained to the importer implementations; it'll bubble up to the configuration layer as well. I suspect it's not worth the pain. I would like to find a way to enforce the canonicalization requirement, but if possible I'd like to do it within a given importer.

[nex3]

After thinking about this more, I think I've come up with a decent API:

• Add Importer.canonicalize() as described above. We leave it up to the importer authors to guarantee that this is canonical across importers. This isn't ideal, but I don't see a better way that doesn't add a lot of complication.

• Change Importer.find() to take both the canonicalized URL and the original URL. It should use the canonicalized URL to look up the file, but it can use the original URL to generate the display URL (see below).

• Add a Uri displayUrl field to ImporterResult that indicates the URL to show for the import in error messages and stack traces.

[ranquild]

@nex3 thanks, waiting for this new api!

[ranquild]

@nex3 I’ve encountered a problem. BuildStep api is asynchronous only, but Importer api is synchronious. It’s seems impossible to implement this interface with build api. Any ideas and workarounds? https://www.dartdocs.org/documentation/build/0.10.2+1/build/AssetReader/readAsString.html

[nex3]

That's a really tough issue. Dart Sass is written synchronously, and Dart doesn't provide any means of calling async callbacks in a synchronous context, or of writing code that's polymorphic over asynchrony. I think this is ultimately something that will need to be solved as the language or VM level.

[nex3]

I've filed dart-lang/sdk#31102 to track Dart support for some sort of synchronization primitive, if you want to 👍 it.

[ranquild]

@nex3 Thanks, I will track this issue