Skip to content

Commit

Permalink
feat: remove raw loader and improve documentation and comments
Browse files Browse the repository at this point in the history
  • Loading branch information
@spence-s
spence-s committed Feb 2, 2021
1 parent 57a6a9a commit 6239e67
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 93 deletions.
14 changes: 9 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,21 @@ The defacto loader for compiling pug templates is [`pug-loader`](https://github.
`simple-pug-loader` does the same thing, only better for many use cases.

#### Problems with pug-loader

Pug loader, while it can work well, has not been well maintained. It suffers from many issues that cause it to not respect normal pug scoping rules and it also handles includes and mixins a bit clunkily. It uses its own - somewhat complicated - way of parsing out includes and mixins and then replaces them with `require` statements in order to get the code loaded into the module correctly. While this works satisfactorily many times, with more complexities in the pug code, you end up running into a lot of unexpected issues when the webpack `require` works completely different from pug `include`. Because of the odd way it requires some of the included pug templates, the templates are "orphaned" in the webpack dependency graph and don't respond correctly using `webpack --watch`, they either don't rebuild at all - or they rebuild the orphaned module and not the parent, so it makes watching untrust worthy. You can see a lot of the same gripes over and over in the [issues](https://github.com/pugjs/pug-loader/issues).

#### What does simple-pug-loader do better?

This loader is just a simple wrapper around pugs `compileClientWithDependenciesTracked` function. We compile directly using pug, then we hand the list of dependencies for the file straight to webpack so it never gets confused on what to build when a file changes. Because it's just pug and no sugar - it's just a lot more predictable.

#### Caveats
We _only_ slightly change pugs algorithm when an `include` is not referencing another pug file but rather a different file type altogether. This is called a `raw include` under the hood. We do this because in some cases, pug doesn't compile to a valid JS function when these files are processed and webpack will throw errors. In the case of raw includes, we make webpack require the raw module independently of pug as a string. Then we embed the raw file right in the pug template as a string (just as pug was originally trying to do) and the file is tracked as a dependency in webpack properly. As a consequence of this, locals and variables will not be available in the non-pug include. It is reccomended that you not `include` any other file type anyway, but rather pass `require` as a global variable into your template and require any non-pug assets instead of including them, unless you need to render the file on the server sometimes as well.

We _only_ slightly change pugs algorithm when an `include` is not referencing another pug file but rather a different file type altogether. This is called a `raw include` under the hood. We do this because in some cases, pug doesn't compile to a valid JS function when these files are processed and webpack will throw errors. In the case of raw includes, we make webpack require the raw module independently. If you have not defined a way for webpack to handle the included file-type - an error will be thrown. The file is tracked as a dependency in webpack properly according to whatever rules you have set.

As a consequence of this, locals and variables will not be available in the non-pug include. If you, for instance, include an svg file in your template like `include ../imgs/logo.svg`. You would need to have a block in webpack telling it to handle svgs by adding an [asset type](https://webpack.js.org/guides/asset-modules/) for svg in webpack 5 or using raw or file loader in webpack 4 and below.

#### Personal Needs

While I haven't seen the exact reasons for `pug-loader` forcing webpack requires in the situations it does, I'm sure there are good reasons for it and if you run across cases where this loader doesn't work for you and `pug-loader` does, please file an issue! I will try to prioritize getting anything working for anyone who wants to use a more stable and maintained pug loader.

My use personal use case is that I render many templates both server and client side and I want predictability from both and everything I have tested thus far has benefitted from this simpler approach that I am employing with this plugin.
Expand Down Expand Up @@ -94,16 +100,14 @@ The following [options] are available to be set for the loader. They are all map

### Embedded resources

Try to use `require` for all your embedded resources, to process them with webpack.
For clarity you should try to use `require` for all your raw includes and embedded resources.

```pug
div
img(src=require("./my/image.png"))
```

Remember, you need to configure loaders for these file types too. You might be interested in the [file loader][file-loader].

If a non-pug resource is included with `include resource.whatever`, `simple-pug-loader` will load it as a raw string automatically.
If a non-pug resource is included with `include resource.whatever`, `simple-pug-loader` will change it to a require call under the hood automatically.

## Contributors

Expand Down
45 changes: 32 additions & 13 deletions index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,24 @@ const nodeResolve = require('resolve').sync;
const dedent = require('dedent');
const walk = require('pug-walk');

/*
* Enusre pug and runtime are from same version
*/
const pugPath = require.resolve('pug');

const pug = require(pugPath);
const runtimePath = nodeResolve('pug-runtime', {
basedir: path.dirname(pugPath)
});

const rawLoaderPath = nodeResolve('raw-loader', { basedir: __dirname });

const pug = require(pugPath);

module.exports = function (source) {
// All the cool loaders do it
/*
* All the cool loaders do it
*/
if (this.cacheable) this.cacheable(true);

// Options and context
/*
* Options and context
*/
const loaderContext = this;
const options = loaderUtils.getOptions(this) || {};
const filename = loaderContext.resourcePath;
Expand All @@ -27,6 +30,12 @@ module.exports = function (source) {

source = typeof source === 'string' ? source : source.toString();

/*
* The plugin is hooked into right before pug "links" the included files,
* which means right before it inlines the included file content into the
* template. Here, we remove raw includes and replace them with
* "- require("path/to/include.notpug")
*/
const plugin = {
preLink(ast) {
return walk(ast, (node, replace) => {
Expand All @@ -37,7 +46,7 @@ module.exports = function (source) {
) {
const val = `require(${loaderUtils.stringifyRequest(
loaderContext,
rawLoaderPath + '?esModule=false!' + node.file.fullPath
node.file.fullPath
)})`;

replace({
Expand Down Expand Up @@ -68,32 +77,42 @@ module.exports = function (source) {
plugins: [plugin].concat(options.plugins || [])
};

// Compile the pug
/*
* Compile the pug
*/
const compilation = pug.compileClientWithDependenciesTracked(
source,
pugOptions
);

func = compilation.body;

// Let webpack know to watch the dependencies
/*
* Let webpack know to watch the dependencies
*/
if (compilation.dependencies && compilation.dependencies.length > 0)
compilation.dependencies.forEach((dep) =>
loaderContext.addDependency(dep)
);
} catch (error) {
// Catch errors if needed
/*
* Catch errors if needed
*/
loaderContext.callback(error);
return;
}

// Add the runtime dependency
/*
* Add the runtime dependency
*/
const requireRuntimeString =
'var pug = require(' +
loaderUtils.stringifyRequest(loaderContext, '!' + runtimePath) +
');\n\n';

// Return the compiled function to be processes as a JS module now
/*
* Return the compiled function to be processes as a JS module now
*/
loaderContext.callback(
null,
dedent`
Expand Down
8 changes: 5 additions & 3 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,8 @@
],
"dependencies": {
"dedent": "^0.7.0",
"file-type": "^16.2.0",
"loader-utils": "^2.0.0",
"pug-walk": "^2.0.0",
"raw-loader": "^4.0.2",
"resolve": "^1.19.0"
},
"devDependencies": {
Expand Down Expand Up @@ -69,7 +67,11 @@
}
},
"rules": {
"unicorn/prevent-abbreviations": "off"
"unicorn/prevent-abbreviations": "off",
"multiline-comment-style": [
"error",
"starred-block"
]
}
}
}
78 changes: 6 additions & 72 deletions yarn.lock
View file
Original file line number Diff line number Diff line change
Expand Up @@ -409,11 +409,6 @@
dependencies:
defer-to-connect "^1.0.1"

"@tokenizer/token@^0.1.1":
version "0.1.1"
resolved "https://registry.yarnpkg.com/@tokenizer/token/-/token-0.1.1.tgz#f0d92c12f87079ddfd1b29f614758b9696bc29e3"
integrity sha512-XO6INPbZCxdprl+9qa/AAbFFOMzzwqYxpjPgLICrMD6C2FCw6qfJOPcBk6JqqPLSaZ/Qx87qn4rpPmPMwaAK6w==

"@types/anymatch@*":
version "1.3.1"
resolved "https://registry.yarnpkg.com/@types/anymatch/-/anymatch-1.3.1.tgz#336badc1beecb9dacc38bea2cf32adf627a8421a"
Expand All @@ -431,11 +426,6 @@
dependencies:
"@types/babel-types" "*"

"@types/debug@^4.1.5":
version "4.1.5"
resolved "https://registry.yarnpkg.com/@types/debug/-/debug-4.1.5.tgz#b14efa8852b7768d898906613c23f688713e02cd"
integrity sha512-Q1y515GcOdTHgagaVFhHnIFQ38ygs/kmxdNpvpou+raI9UO3YZcHDngBSYKQklcKlvA7iuQlmIKbzvmxcOE9CQ==

"@types/eslint-scope@^3.7.0":
version "3.7.0"
resolved "https://registry.yarnpkg.com/@types/eslint-scope/-/eslint-scope-3.7.0.tgz#4792816e31119ebd506902a482caec4951fabd86"
Expand Down Expand Up @@ -500,14 +490,6 @@
resolved "https://registry.yarnpkg.com/@types/parse-json/-/parse-json-4.0.0.tgz#2f8bb441434d163b35fb8ffdccd7138927ffb8c0"
integrity sha512-//oorEZjL6sbPcKUaCdIGlIUeH26mgzimjBB77G6XRgnDl/L5wOnpyBGRe/Mmf5CVW3PwEBE1NjiMZ/ssFh4wA==

"@types/readable-stream@^2.3.9":
version "2.3.9"
resolved "https://registry.yarnpkg.com/@types/readable-stream/-/readable-stream-2.3.9.tgz#40a8349e6ace3afd2dd1b6d8e9b02945de4566a9"
integrity sha512-sqsgQqFT7HmQz/V5jH1O0fvQQnXAJO46Gg9LRO/JPfjmVmGUlcx831TZZO3Y3HtWhIkzf3kTsNT0Z0kzIhIvZw==
dependencies:
"@types/node" "*"
safe-buffer "*"

"@types/source-list-map@*":
version "0.1.2"
resolved "https://registry.yarnpkg.com/@types/source-list-map/-/source-list-map-0.1.2.tgz#0078836063ffaf17412349bba364087e0ac02ec9"
Expand Down Expand Up @@ -3186,16 +3168,6 @@ file-entry-cache@^6.0.0:
dependencies:
flat-cache "^3.0.4"

file-type@^16.2.0:
version "16.2.0"
resolved "https://registry.yarnpkg.com/file-type/-/file-type-16.2.0.tgz#d4f1da71ddda758db7f15f93adfaed09ce9e2715"
integrity sha512-1Wwww3mmZCMmLjBfslCluwt2mxH80GsAXYrvPnfQ42G1EGWag336kB1iyCgyn7UXiKY3cJrNykXPrCwA7xb5Ag==
dependencies:
readable-web-to-node-stream "^3.0.0"
strtok3 "^6.0.3"
token-types "^2.0.0"
typedarray-to-buffer "^3.1.5"

file-uri-to-path@1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz#553a7b8446ff6f684359c445f1e37a05dacc33dd"
Expand Down Expand Up @@ -3824,7 +3796,7 @@ iconv-lite@0.4.24:
dependencies:
safer-buffer ">= 2.1.2 < 3"

ieee754@^1.1.13, ieee754@^1.1.4, ieee754@^1.2.1:
ieee754@^1.1.13, ieee754@^1.1.4:
version "1.2.1"
resolved "https://registry.yarnpkg.com/ieee754/-/ieee754-1.2.1.tgz#8eb7a10a63fff25d15a57b001586d177d1b0d352"
integrity sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==
Expand Down Expand Up @@ -5714,11 +5686,6 @@ pbkdf2@^3.0.3:
safe-buffer "^5.0.1"
sha.js "^2.4.8"

peek-readable@^3.1.3:
version "3.1.3"
resolved "https://registry.yarnpkg.com/peek-readable/-/peek-readable-3.1.3.tgz#932480d46cf6aa553c46c68566c4fb69a82cd2b1"
integrity sha512-mpAcysyRJxmICBcBa5IXH7SZPvWkcghm6Fk8RekoS3v+BpbSzlZzuWbMx+GXrlUwESi9qHar4nVEZNMKylIHvg==

picomatch@^2.0.4, picomatch@^2.0.5, picomatch@^2.2.1, picomatch@^2.2.2:
version "2.2.2"
resolved "https://registry.yarnpkg.com/picomatch/-/picomatch-2.2.2.tgz#21f333e9b6b8eaff02468f5146ea406d345f4dad"
Expand Down Expand Up @@ -6108,14 +6075,6 @@ raw-body@2.4.0:
iconv-lite "0.4.24"
unpipe "1.0.0"

raw-loader@^4.0.2:
version "4.0.2"
resolved "https://registry.yarnpkg.com/raw-loader/-/raw-loader-4.0.2.tgz#1aac6b7d1ad1501e66efdac1522c73e59a584eb6"
integrity sha512-ZnScIV3ag9A4wPX/ZayxL/jZH+euYb6FcUinPcgiQW0+UBtEv0O6Q3lGd3cqJ+GHH+rksEv3Pj99oxJ3u3VIKA==
dependencies:
loader-utils "^2.0.0"
schema-utils "^3.0.0"

rc@^1.2.8:
version "1.2.8"
resolved "https://registry.yarnpkg.com/rc/-/rc-1.2.8.tgz#cd924bf5200a075b83c188cd6b9e211b7fc0d3ed"
Expand Down Expand Up @@ -6184,14 +6143,6 @@ readable-stream@^2.0.1, readable-stream@^2.0.2, readable-stream@^2.3.3, readable
string_decoder "~1.1.1"
util-deprecate "~1.0.1"

readable-web-to-node-stream@^3.0.0:
version "3.0.1"
resolved "https://registry.yarnpkg.com/readable-web-to-node-stream/-/readable-web-to-node-stream-3.0.1.tgz#3f619b1bc5dd73a4cfe5c5f9b4f6faba55dff845"
integrity sha512-4zDC6CvjUyusN7V0QLsXVB7pJCD9+vtrM9bYDRv6uBQ+SKfx36rp5AFNPRgh9auKRul/a1iFZJYXcCbwRL+SaA==
dependencies:
"@types/readable-stream" "^2.3.9"
readable-stream "^3.6.0"

readdirp@^2.2.1:
version "2.2.1"
resolved "https://registry.yarnpkg.com/readdirp/-/readdirp-2.2.1.tgz#0e87622a3325aa33e892285caf8b4e846529a525"
Expand Down Expand Up @@ -6440,16 +6391,16 @@ rxjs@^6.6.3:
dependencies:
tslib "^1.9.0"

safe-buffer@*, safe-buffer@>=5.1.0, safe-buffer@^5.0.1, safe-buffer@^5.1.0, safe-buffer@^5.1.1, safe-buffer@^5.1.2, safe-buffer@^5.2.0, safe-buffer@~5.2.0:
version "5.2.1"
resolved "https://registry.yarnpkg.com/safe-buffer/-/safe-buffer-5.2.1.tgz#1eaf9fa9bdb1fdd4ec75f58f9cdb4e6b7827eec6"
integrity sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==

safe-buffer@5.1.2, safe-buffer@~5.1.0, safe-buffer@~5.1.1:
version "5.1.2"
resolved "https://registry.yarnpkg.com/safe-buffer/-/safe-buffer-5.1.2.tgz#991ec69d296e0313747d59bdfd2b745c35f8828d"
integrity sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==

safe-buffer@>=5.1.0, safe-buffer@^5.0.1, safe-buffer@^5.1.0, safe-buffer@^5.1.1, safe-buffer@^5.1.2, safe-buffer@^5.2.0, safe-buffer@~5.2.0:
version "5.2.1"
resolved "https://registry.yarnpkg.com/safe-buffer/-/safe-buffer-5.2.1.tgz#1eaf9fa9bdb1fdd4ec75f58f9cdb4e6b7827eec6"
integrity sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==

safe-regex@^1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/safe-regex/-/safe-regex-1.1.0.tgz#40a3669f3b077d1e943d44629e157dd48023bf2e"
Expand Down Expand Up @@ -7037,15 +6988,6 @@ strip-json-comments@~2.0.1:
resolved "https://registry.yarnpkg.com/strip-json-comments/-/strip-json-comments-2.0.1.tgz#3c531942e908c2697c0ec344858c286c7ca0a60a"
integrity sha1-PFMZQukIwml8DsNEhYwobHygpgo=

strtok3@^6.0.3:
version "6.0.8"
resolved "https://registry.yarnpkg.com/strtok3/-/strtok3-6.0.8.tgz#c839157f615c10ba0f4ae35067dad9959eeca346"
integrity sha512-QLgv+oiXwXgCgp2PdPPa+Jpp4D9imK9e/0BsyfeFMr6QL6wMVqoVn9+OXQ9I7MZbmUzN6lmitTJ09uwS2OmGcw==
dependencies:
"@tokenizer/token" "^0.1.1"
"@types/debug" "^4.1.5"
peek-readable "^3.1.3"

supertap@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/supertap/-/supertap-2.0.0.tgz#8b587d6e14b8e885fa5183a9c45abf429feb9f7f"
Expand Down Expand Up @@ -7263,14 +7205,6 @@ token-stream@0.0.1:
resolved "https://registry.yarnpkg.com/token-stream/-/token-stream-0.0.1.tgz#ceeefc717a76c4316f126d0b9dbaa55d7e7df01a"
integrity sha1-zu78cXp2xDFvEm0LnbqlXX598Bo=

token-types@^2.0.0:
version "2.1.1"
resolved "https://registry.yarnpkg.com/token-types/-/token-types-2.1.1.tgz#bd585d64902aaf720b8979d257b4b850b4d45c45"
integrity sha512-wnQcqlreS6VjthyHO3Y/kpK/emflxDBNhlNUPfh7wE39KnuDdOituXomIbyI79vBtF0Ninpkh72mcuRHo+RG3Q==
dependencies:
"@tokenizer/token" "^0.1.1"
ieee754 "^1.2.1"

trim-newlines@^3.0.0:
version "3.0.0"
resolved "https://registry.yarnpkg.com/trim-newlines/-/trim-newlines-3.0.0.tgz#79726304a6a898aa8373427298d54c2ee8b1cb30"
Expand Down

0 comments on commit 6239e67

Please sign in to comment.