PostCSS `css-compile-variables` is flakey with `vite@2.9.0`+ #722

MadLittleMods · 2022-04-19T22:44:43Z

With vite@2.9.1 installed, I'm noticing that sometimes yarn build:sdk fails with the following error related to scripts/postcss/css-compile-variables.js. I'm able to reproduce on the latest master with the changes from #721 to get around the first error noted in that linked PR.

~~I'm only able to reproduce on Windows 10. I tried running while yarn build:sdk; do :; done on macOS to repeat until failure but never saw any failure there.~~

Failing run ❌:

$ C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\.bin\vite build -c vite.sdk-assets-config.js
[vite:css] Cannot derive from background-color-secondary because it is neither defined in config nor is it an alias!
file: C:/Users/MLM/Documents/GitHub/element/hydrogen-web/src/platform/web/ui/css/themes/element/theme.css?type=runtime
error during build:
Error: Cannot derive from background-color-secondary because it is neither defined in config nor is it an alias!
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\src\platform\web\ui\css\themes\element\timeline.css:24:5
    at resolveDerivedVariable (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\vite.sdk-assets-config.js:280:19)
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\vite.sdk-assets-config.js:328:36
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:72:18
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:55:18
    at Rule.each (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:41:16)
    at Rule.walk (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:52:17)
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:60:24
    at Root.each (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:41:16)
    at Root.walk (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:52:17)
    at Root.walkDecls (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:70:19)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
$ C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\.bin\vite build -c vite.sdk-lib-config.js

Getting some more debugging info on which variable is failing

Added some extra logging to scripts/postcss/css-compile-variables.js#L78, we can see what declaration it's failing on.

console.log(`resolveDerivedVariable decl.prop=${decl.prop} decl.value=${decl.value} variable=${variable} value=${value} baseVariable=${baseVariable} baseVariables=${JSON.stringify(Array.from(baseVariables.entries()))} aliasMap=${JSON.stringify(Array.from(aliasMap.entries()))} resolvedMap=${JSON.stringify(Array.from(resolvedMap.entries()))}`);

For some reason, background-color-secondary isn't being populated in baseVariables:

$ C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\.bin\vite build -c vite.sdk-assets-config.js
resolveDerivedVariable decl.prop=border decl.value=1px solid var(--background-color-secondary--darker-7) variable=--background-color-secondary--darker-7 value=undefined baseVariable=background-color-secondary
baseVariables=[["size","20px"],["avatar-size","12px"],["usercolor1","#368BD6"],["usercolor2","#AC3BA8"],["usercolor3","#03B381"],["usercolor4","#E64F7A"],["usercolor5","#FF812D"],["usercolor6","#2DC2C5"],["usercolor7","#5C56F5"],["usercolor8","#74D12C"]] 
aliasMap=[["icon-color","background-color-secondary--darker-40"],["light-border","background-color-secondary--darker-5"],["light-text-color","background-color-secondary--darker-55"],["timeline-time-text-color","background-color-secondary--darker-35"],["icon-background","background-color-secondary--darker-7"],["right-panel-text-color","background-color-secondary--darker-35"]] 
resolvedMap=[]

[vite:css] Cannot derive from background-color-secondary because it is neither defined in config nor is it an alias!
file: C:/Users/MLM/Documents/GitHub/element/hydrogen-web/src/platform/web/ui/css/themes/element/theme.css?type=runtime
error during build:
Error: Cannot derive from background-color-secondary because it is neither defined in config nor is it an alias!
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\src\platform\web\ui\css\themes\element\timeline.css:24:5
    at resolveDerivedVariable (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\vite.sdk-assets-config.js:281:19)
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\vite.sdk-assets-config.js:329:36
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:72:18
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:55:18
    at Rule.each (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:41:16)
    at Rule.walk (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:52:17)
    at C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:60:24
    at Root.each (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:41:16)
    at Root.walk (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:52:17)
    at Root.walkDecls (C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\postcss\lib\container.js:70:19)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

For reference, here is how it looks when it works successfully:

Successful run

 C:\Users\MLM\Documents\GitHub\element\hydrogen-web\node_modules\.bin\vite build -c vite.sdk-assets-config.js
resolveDerivedVariable decl.prop=border decl.value=1px solid var(--background-color-secondary--darker-7) variable=--background-color-secondary--darker-7 value=#2D3239 baseVariable=background-color-secondary
baseVariables=[["size","20px"],["avatar-size","12px"],["usercolor1","#368BD6"],["usercolor2","#AC3BA8"],["usercolor3","#03B381"],["usercolor4","#E64F7A"],["usercolor5","#FF812D"],["usercolor6","#2DC2C5"],["usercolor7","#5C56F5"],["usercolor8","#74D12C"],["background-color-primary","#21262b"],["background-color-secondary","#2D3239"],["text-color","#fff"],["accent-color","#03B381"],["error-color","#FF4B55"],["fixed-white","#fff"],["room-badge","#61708b"],["link-color","#238cf5"]]
aliasMap=[["icon-color","background-color-secondary--darker-40"],["light-border","background-color-secondary--darker-5"],["light-text-color","background-color-secondary--darker-55"],["timeline-time-text-color","background-color-secondary--darker-35"],["icon-background","background-color-secondary--darker-7"],["right-panel-text-color","background-color-secondary--darker-35"]]
resolvedMap=[]
...
Generated an empty chunk: "index"

Dev notes

while yarn build:sdk; do :; done
bash -c "while bash scripts/sdk/build.sh; do :; done"

console.log('css-compile-variables once', count, root.source.input.from, root.toString().split("\n").slice(-100).join("\n"));

Platform details

Windows 10 using Cmder. It spawns a Git Bash window to run the build.sh script.

$ node --version
v16.14.0

$ yarn --version
1.22.18

$ git --version
git version 2.28.0.windows.1

The text was updated successfully, but these errors were encountered:

MadLittleMods · 2022-04-19T23:41:27Z

I've narrowed it down to to our appended :root { ... } variables just not showing up in the source for the PostCSS plugin sometimes.

I've confirmed that appendVariablesToCSS runs with all of the expected variables in both the failure and successful cases. But after the rollup load, it just isn't in the source for the PostCSS plugin to see sometimes.

https://github.com/vector-im/hydrogen-web/blob/480c5c1584c43c0f2d1ad82054e33b62a442a730/scripts/build-plugins/rollup-plugin-build-themes.js#L167-L172

To note: It doesn't appear to be another Windows path backslash problem like #721 (comment) even though there are a few more cases in our codebase with that same regex. Some code paths seem to get normalized forward slash paths anyway.

MadLittleMods · 2022-04-20T00:51:05Z

~~On Windows~~With vite@2.9.0+, all of the CSS files root.source.input.from resolve to C:/Users/MLM/Documents/GitHub/element/hydrogen-web/src/platform/web/ui/css/themes/element/theme.css without the extra query parameters we would see on other platforms:

/Users/eric/Documents/github/element/hydrogen-web/src/platform/web/ui/css/themes/element/theme.css?variant=dark&dark=true
/Users/eric/Documents/github/element/hydrogen-web/src/platform/web/ui/css/themes/element/theme.css?variant=light
/Users/eric/Documents/github/element/hydrogen-web/src/platform/web/ui/css/themes/element/theme.css?type=runtime

So the following check in our PostCSS plugin to skip the ?type=runtime file isn't ever hit and we run always run the plugin over the ?type=runtime file:

https://github.com/vector-im/hydrogen-web/blob/480c5c1584c43c0f2d1ad82054e33b62a442a730/scripts/postcss/css-compile-variables.js#L145-L149

And the reason it works sometimes (flakey), depends on the order that the files are processed. We also have the variable maps declared in the plugin instance scope which are shared across all of the files processed. So if we process one of the ?variant=light or ?variant=dark first, the variables are already defined in the baseVariables map and resolve for the other files which don't have them defined.

https://github.com/vector-im/hydrogen-web/blob/480c5c1584c43c0f2d1ad82054e33b62a442a730/scripts/postcss/css-compile-variables.js#L136-L139

We should move the map variables inside the file processing (Once) scope so they aren't shared across plugin runs. This would remove the flakey behavior as the plugin would fail every time. We should probably also pass them around directly to avoid concurrent parallel processing problems.

Then we also need to address a way to skip and detect ?type=runtime on Windows. Either fix the bug that is stripping the extra query parameter information or make it part of the file name. The query parameters are still intact in the id in the rollup load hook so it's happening sometime after that.

MadLittleMods · 2022-04-20T02:01:40Z

I've traced the problem to Vite at packages/vite/src/node/plugins/css.ts#L789 where it cleans the given id.

This behavior regressed or changed in vite@2.9.0 (via https://github.com/vitejs/vite/pull/7173/files#diff-2cfbd4f4d8c32727cd8e1a561cffbde0b384a3ce0789340440e144f9d64c10f6R760) so technically it's not a problem on the latest Hydrogen master which uses vite@2.6.14 but was a problem because I had a later version of vite@2.9.1 installed for #693.

MadLittleMods · 2022-04-20T05:27:41Z

Asked in vitejs/vite#7822 whether this cleanUrl(id) behavior is intended and here to stay so we know whether we have to refactor scripts/postcss/css-compile-variables.js#L145-L149 or not.

MadLittleMods · 2022-04-20T22:41:54Z

We should move the map variables inside the file processing (Once) scope so they aren't shared across plugin runs. This would remove the flakey behavior as the plugin would fail every time. We should probably also pass them around directly to avoid concurrent parallel processing problems.

@MidhunSureshR Any concern/interest about our custom PostCSS plugins having a bunch of plugin instance-level variables which are shared across all files being processed? Is this by design? It feels like ideally, we don't want this to be the case.

MidhunSureshR · 2022-04-21T12:03:50Z

We should move the map variables inside the file processing (Once) scope so they aren't shared across plugin runs. This would remove the flakey behavior as the plugin would fail every time. We should probably also pass them around directly to avoid concurrent parallel processing problems.

@MidhunSureshR Any concern/interest about our custom PostCSS plugins having a bunch of plugin instance-level variables which are shared across all files being processed? Is this by design? It feels like ideally, we don't want this to be the case.

This wasn't really by design. I just didn't want to pass the maps around and didn't really see an immediate issue with the approach. We can definitely bring the scope down for those variables 👍 .

Fix #722 Updating Vite to includes fixes from vitejs/vite#7822 -> vitejs/vite#7827

MadLittleMods · 2022-05-05T20:09:51Z

vite@2.9.8 now includes a fix vitejs/vite#7827 and restores the old functionality of not cleaning the ids before passing to the PostCSS functions so everything works again. The dependency is being updated in #693

bwindels · 2022-05-16T11:09:12Z

Hey @MidhunSureshR could you still have a look to move the scope of those variables down? I think it's a good thing to do regardless of it not being an immediate problem anymore now.

MadLittleMods · 2022-05-31T20:19:46Z

could you still have a look to move the scope of those variables down? I think it's a good thing to do regardless of it not being an immediate problem anymore now.

Created #745 to track this separately ⏩

MadLittleMods added bug Something isn't working sdk labels Apr 19, 2022

MadLittleMods changed the title ~~PostCSS css-compile-variables is flakey on Windows~~ PostCSS css-compile-variables is flakey with vite@2.9.0+ Apr 20, 2022

MadLittleMods mentioned this issue Apr 20, 2022

from id is always cleaned before being passed to the PostCSS plugins vitejs/vite#7822

Closed

7 tasks

MadLittleMods mentioned this issue Apr 20, 2022

Make the SDK friendly to locally link and develop on #693

Merged

MadLittleMods added a commit that referenced this issue May 5, 2022

Update Vite to avoid flakey errors in our PostCSS plugins

d053d43

Fix #722 Updating Vite to includes fixes from vitejs/vite#7822 -> vitejs/vite#7827

MadLittleMods self-assigned this May 5, 2022

bwindels closed this as completed in #693 May 30, 2022

MadLittleMods mentioned this issue May 31, 2022

Scope variables in Custom PostCSS plugins to each file processed #745

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PostCSS `css-compile-variables` is flakey with `vite@2.9.0`+ #722

PostCSS `css-compile-variables` is flakey with `vite@2.9.0`+ #722

MadLittleMods commented Apr 19, 2022 •

edited

Loading

MadLittleMods commented Apr 19, 2022 •

edited

Loading

MadLittleMods commented Apr 20, 2022 •

edited

Loading

MadLittleMods commented Apr 20, 2022 •

edited

Loading

MadLittleMods commented Apr 20, 2022

MadLittleMods commented Apr 20, 2022

MidhunSureshR commented Apr 21, 2022

MadLittleMods commented May 5, 2022

bwindels commented May 16, 2022

MadLittleMods commented May 31, 2022

PostCSS css-compile-variables is flakey with vite@2.9.0+ #722

PostCSS css-compile-variables is flakey with vite@2.9.0+ #722

Comments

MadLittleMods commented Apr 19, 2022 • edited Loading

Getting some more debugging info on which variable is failing

Dev notes

Platform details

MadLittleMods commented Apr 19, 2022 • edited Loading

MadLittleMods commented Apr 20, 2022 • edited Loading

MadLittleMods commented Apr 20, 2022 • edited Loading

MadLittleMods commented Apr 20, 2022

MadLittleMods commented Apr 20, 2022

MidhunSureshR commented Apr 21, 2022

MadLittleMods commented May 5, 2022

bwindels commented May 16, 2022

MadLittleMods commented May 31, 2022

PostCSS `css-compile-variables` is flakey with `vite@2.9.0`+ #722

PostCSS `css-compile-variables` is flakey with `vite@2.9.0`+ #722

MadLittleMods commented Apr 19, 2022 •

edited

Loading

MadLittleMods commented Apr 19, 2022 •

edited

Loading

MadLittleMods commented Apr 20, 2022 •

edited

Loading

MadLittleMods commented Apr 20, 2022 •

edited

Loading