Unable to debug rules using the Run TSLint Tests template

[noamyogev84]

Bug Report

• tslint-microsoft-contrib version: latest
• TSLint version: 5.11.0
• TypeScript version: 3.1.1
• Running TSLint via: VSCode
• Node.js version:8.12.0
• OS: win10 x64

Trying to debug tests under tests folder with no success.
Command: node.exe --inspect-brk=27157 node_modules\tslint\bin\tslint --test -r dist/src tests/**
Output: Debugger listening on ws://127.0.0.1:27157/da6329d9-5a8f-4729-965a-9339e2d17953

Debug console lists all tests with passed, no breakpoint is being hit.

[reduckted]

How are you running that command? Selecting the "Run TSLint Tests" debugging configuration in VS Code and starting debugging works fine for me. Here's the steps I took:

[noamyogev84]

Hi @reduckted , exactly as you describe.
No breakpoint is being hit in my rule though.

Can you share the test folder content that's associated with what your'e showing?

Thanks 🏆

[reduckted]

What file are you putting the breakpoint in?

Here's the file listing of the dist/src directory:

D:\Code\GitHub\tslint-microsoft-contrib>dir dist\src

2019-01-02  10:07    <DIR>          .
2019-01-02  10:07    <DIR>          ..
2019-01-02  10:07               293 chaiPreferContainsToIndexOfRule.d.ts
2019-01-02  10:07             3,427 chaiPreferContainsToIndexOfRule.js
2019-01-02  10:07             1,992 chaiPreferContainsToIndexOfRule.js.map
2019-01-02  10:07               293 chaiVagueErrorsRule.d.ts
2019-01-02  10:07             4,507 chaiVagueErrorsRule.js
2019-01-02  10:07             2,641 chaiVagueErrorsRule.js.map
...<snip>...
2019-01-02  10:07               329 noRegexSpacesRule.d.ts
2019-01-02  10:07             2,431 noRegexSpacesRule.js
2019-01-02  10:07             1,153 noRegexSpacesRule.js.map
2019-01-02  10:07               293 noRelativeImportsRule.d.ts
2019-01-02  10:07             4,494 noRelativeImportsRule.js
2019-01-02  10:07             2,495 noRelativeImportsRule.js.map
2019-01-02  10:07               417 noReservedKeywordsRule.d.ts
2019-01-02  10:07             3,315 noReservedKeywordsRule.js
2019-01-02  10:07             1,608 noReservedKeywordsRule.js.map
...<snip>...
2019-01-02  10:07               293 useSimpleAttributesRule.d.ts
2019-01-02  10:07             4,641 useSimpleAttributesRule.js
2019-01-02  10:07             3,034 useSimpleAttributesRule.js.map
2019-01-02  10:07    <DIR>          utils
2019-01-02  10:07               400 validTypeofRule.d.ts
2019-01-02  10:07             4,480 validTypeofRule.js
2019-01-02  10:07             3,600 validTypeofRule.js.map
             288 File(s)        725,008 bytes
               4 Dir(s)

Here's the output in the debug console:

C:\Program Files\nodejs\node.exe --inspect-brk=20670 node_modules\tslint\bin\tslint --test -r dist/src tests/** 
Debugger listening on ws://127.0.0.1:20670/5bf8869f-c4c2-47ac-8659-aa5363c98645

For help, see: https://nodejs.org/en/docs/inspector

Warning: no-unnecessary-bind rule is deprecated. Replace your usage with the TSLint unnecessary-bind rule.
Warning: no-var-self rule is deprecated. Replace your usage with the TSLint no-this-assignment rule.
Warning: valid-typeof rule is deprecated. Replace your usage with the TSLint typeof-compare rule.
tests/export-name/defaults/re-export-external.ts.lint: Passed
tests/no-relative-imports/allow-siblings/test.js.lint: Passed
tests/no-relative-imports/allow-siblings/test.ts.lint: Passed
tests/no-relative-imports/default/test.js.lint: Passed
tests/no-relative-imports/default/test.ts.lint: Passed
tests/no-unnecessary-bind/test.ts.lint: Passed
tests/no-unnecessary-field-initialization/test.ts.lint: Passed
tests/no-unnecessary-local-variable/test.ts.lint: Passed
tests/no-unnecessary-override/test.ts.lint: Passed
tests/no-unsupported-browser-code/test.ts.lint: Passed
tests/no-useless-files/code/test.ts.lint: Passed
tests/no-useless-files/empty/test.ts.lint: Passed
tests/no-useless-files/mixed comments/test.ts.lint: Passed
tests/no-useless-files/multi line comments/test.ts.lint: Passed
tests/no-useless-files/single line comments/test.ts.lint: Passed
tests/no-var-self/default/test.ts.lint: Passed
tests/no-var-self/negation parameter/test.ts.lint: Passed
tests/no-var-self/parameter/test.ts.lint: Passed
tests/no-with-statement/test.ts.lint: Passed
tests/non-literal-fs-path/test.ts.lint: Passed
tests/non-literal-require/test.ts.lint: Passed
tests/possible-timing-attack/test.ts.lint: Passed
tests/prefer-array-literal/allow-type-parameters/test.ts.lint: Passed
tests/prefer-array-literal/default/test.ts.lint: Passed
tests/promise-must-complete/test.ts.lint: Passed
tests/underscore-consistent-invocation/default/test.ts.lint: Passed
tests/underscore-consistent-invocation/static/test.ts.lint: Passed
tests/use-named-parameter/test.ts.lint: Passed
tests/valid-typeof/test.ts.lint: Passed
Debugger attached.

Waiting for the debugger to disconnect...

[IllusionMH]

I was able to reproduce this issue.
In my case it may work for some lines, and doesn't work for other.

For example I tried to place 3 breakpoints in src/validTypeofRule.ts

public apply(sourceFile: ts.SourceFile): Lint.RuleFailure[] {
    if (Rule.isWarningShown === false) { // <- Doesn't work (never hit).
        console.warn('Warning: valid-typeof rule is deprecated. Replace your usage with the TSLint typeof-compare rule.');
        Rule.isWarningShown = true; // <- Inconsistent (few hits, but rarely)
    }
    return this.applyWithFunction(sourceFile, walk); // <- Works almost always (after few rebuilds stopped to work)
}

These are simple lines that should definitely hit at least once, but e.g. first line never worked for me.

As workaround I've added debugger at top level of the file.

import { ExtendedMetadata } from './utils/ExtendedMetadata';
debugger;
export class Rule extends Lint.Rules.AbstractRule {

rebuild sources and sourcemaps (npm run tsc:src for short) and after this "forced" breackpoint all other breakpoints are correctly hit.

@noamyogev84 could you please try same steps and see if this workaround works for you?

UPD. Just to clarify - I will continue investigation to determine root cause. At this time it may be quick workaround and check some theories.

[reduckted]

In my case it may work for some lines, and doesn't work for other.

Nice find! I guess I got lucky with what I tried out. I can confirm that it doesn't hit the breakpoints when I put them in any of those places in validTypeOfRule.ts.

I took a look at what tslint uses for its debugging of tests, and it appears that the outFiles property is required. Using this configuration, the breakpoints are hit:

{
    "type": "node",
    "request": "launch",
    "name": "Run TSLint Tests",
    "program": "${workspaceFolder}/node_modules/tslint/bin/tslint",
    "args": ["--test", "-r", "dist/src", "tests/**"],
    "outputCapture": "std",
    "outFiles": ["${workspaceRoot}/dist/src/**/*.js"],
    "cwd": "${workspaceFolder}",
    "runtimeArgs": ["--nolazy"]
}

I've also added the cwd and runtimeArgs properties that tslint also uses.

[JoshuaKGoldberg]

/cc @roblourens - good case study in why VS Code debuggering is still a bit confusing.

[reduckted]

good case study in why VS Code debuggering is still a bit confusing.

Indeed. Not sure why it worked for some files, but failed for others.

I also discovered that, without outFiles, if you put a breakpoint in the compiled JavaScript file, VS Code would hit the breakpoint, but would stop at the corresponding place in the original TypeScript file instead of at the breakpoint in the JavaScript file. 🤯

[roblourens]

Glad you figured it out. Any ideas on how I could have made it more obvious? outFiles is not required but it is recommended to be used when possible.

vscode will still load the script's sourcemaps when the script loads, but there is a race between loading the sourcemap and running the line of code with the breakpoint. outFiles lets us pre-load sourcemaps off disk before running the program.

[JoshuaKGoldberg]

How about a suggestion for this particular case of not including the .ts files? If some or all of these fields aren't filled out (read: if the user is putting breakpoints in a file that won't be included), and the user is placing breakpoints that aren't being hit, maybe some kind of informative dialog?

Right now we just get a generic "breakpoint won't be loaded" complaint without any indication of why., or easy access to the list of files that would be included.

Just a quick thought - seems rather hardcoded to this scenario, and it'd be nice to have a more general solution.

[roblourens]

Yeah but I don't know whether anything has actually gone wrong yet, it may be that the file is just not loaded. And if the breakpoint was supposed to be hit, that describes many possible issues with the setup or the sourcemaps.

[JoshuaKGoldberg]

the file is just not loaded

Is there a way we could detect our use case? Suppose you have source.js with not-loaded breakpoints along with any (or maybe just both?) of...

• //# sourceMappingURL=nodes.js.map that matches up to a neighboring source.ts
• A project using TypeScript such as with tsconfig.json or an equivalent

[roblourens]

The breakpoints will be in source.ts (if I understand you correctly) and the whole problem is that it hasn't been associated with a source.js file.

If we want to hint that the outFiles field could have been used, maybe we would want to detect when a sourcemap is loaded from disk but was not preloaded, then show a hint. But it is kind of specific.