Using nearley-test on Windows PowerShell or Bash

[jtheis85]

I have a very simple grammar which seems to compile and work as expected. But we have a mix of Windows and Mac dev machines on our team, and while I can get nearley-test to work on Mac with the instructions in the docs, I have to jump through some hoops on Windows:

On Mac
Terminal: VS Code Integrated
Shell: Bash
npm -v: 6.0.0
node -v: v6.11.1 (I realize this is an old version of node, but this is the machine it works on)
nearleyc -v: 2.19.0

nearley-test <(nearleyc my-grammar.ne) -i "A test string"      ✅ Works fine

On Windows
Terminal: VS Code Integrated
npm -v: 6.4.1
node -v: v10.14.1
nearleyc -v: 2.19.0

I'm not sure I'm doing this properly. I THINK the | character is supposed to pipe the output to the next command in powershell

On Windows (PowerShell)

nearleyc my-grammar.ne | nearley-test -i "A test string"      ❌ Breaks

path.js:39
    throw new ERR_INVALID_ARG_TYPE('path', 'string', path);
    ^

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
    at assertPath (path.js:39:11)
    at Object.resolve (path.js:166:7)
    at Object.<anonymous> (C:\Users\jathe\AppData\Roaming\npm\node_modules\nearley\bin\nearley-test.js:23:32)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:741:12)
    at startup (internal/bootstrap/node.js:285:19)

So I tried doing it in 2 steps, first compile (which works), then point nearley-test at the compiled file:

On Windows (PowerShell)

nearleyc my-grammar.ne > test.js      ✅ Works fine
nearley-test test.js -i "A test string"      ❌ Breaks

(function (exports, require, module, __filename, __dirname) { ��/
                                                              ^
SyntaxError: Invalid or unexpected token
    at new Script (vm.js:79:7)
    at createScript (vm.js:251:10)
    at Object.runInThisContext (vm.js:303:10)
    at Module._compile (internal/modules/cjs/loader.js:656:28)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at Module.require (internal/modules/cjs/loader.js:636:17)
    at require (internal/modules/cjs/helpers.js:20:18)

So I try to switch to Bash on Windows instead of PowerShell. The first thing I try is the command in the docs again:

On Windows (Bash)

nearley-test <(nearleyc my-grammar.ne) -i "A test string"      ❌ Breaks

internal/modules/cjs/loader.js:582
    throw err;
    ^

Error: Cannot find module 'C:\proc\27880\fd\63'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)
    at Function.Module._load (internal/modules/cjs/loader.js:506:25)
    at Module.require (internal/modules/cjs/loader.js:636:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (C:\Users\jathe\AppData\Roaming\npm\node_modules\nearley\bin\nearley-test.js:24:44)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
events.js:167
      throw er; // Unhandled 'error' event
      ^

Error: EPIPE: broken pipe, write
    at Socket._write (internal/net.js:58:25)
    at doWrite (_stream_writable.js:410:12)
    at writeOrBuffer (_stream_writable.js:394:5)
    at Socket.Writable.write (_stream_writable.js:294:11)
    at StreamWrapper.<anonymous> (C:\Users\jathe\AppData\Roaming\npm\node_modules\nearley\bin\nearleyc.js:37:16)
    at StreamWrapper.emit (events.js:187:15)
    at finishMaybe (_stream_writable.js:641:14)
    at endWritable (_stream_writable.js:649:3)
    at StreamWrapper.Writable.end (_stream_writable.js:589:5)
    at ReadStream.onend (_stream_readable.js:628:10)
Emitted 'error' event at:
    at onwriteError (_stream_writable.js:425:12)
    at onwrite (_stream_writable.js:456:5)
    at Socket._write (internal/net.js:62:14)
    at doWrite (_stream_writable.js:410:12)
    [... lines matching original stack trace ...]
    at endWritable (_stream_writable.js:649:3)

So then I try again in Bash on Windows to split it into 2 parts:

On Windows (Bash)

nearleyc my-grammar.ne > test.js      ✅ Works fine
nearley-test test.js -i "A test string"      ✅ Works fine

Any idea what I'm doing wrong with Win+PowerShell that will make it work (either in one OR two stages?)

Also, any idea what could be done with Win+Bash to make it work in a single command like the docs? I guess I could just squash the 2 commands together like nearleyc my-grammar.ne > test.js && nearley-test test.js -i "A test string" which seems to work, although the intermediate file is annoying.

[khoidt]

I stumbled upon a similar issue while testing a nearley grammar in a React application on Win 10 ( ✔️) and calling it with node in Powershell (:x:) respectively. In the latter case, I got the SyntaxError for a subset of strings that were successfully parsed in React.

[khoidt]

It seems that - at least in my case - this had to do with different line break definitions (\r\n on Win vs. '\n' Bash), that were not taken into account in my tokenizer. The easiest way to avoid this would be preprocessing the string before parsing, e.g. fs.readFileSync(path, 'utf8').replace(/[ ]*(\r\n|\n|\r)/gm, '\n') on load.