Add a switch for reporting expensive statements #37785

amcasey · 2020-04-04T01:06:37Z

Multiple customers have reported that having expensive statements identified (thus far, with private builds) has been helpful in reducing their check time - even without explanations of why the statements are expensive. This PR attempts to formalize that instrumentation so that it can be used more widely.

amcasey · 2020-04-04T01:06:52Z

Marked as a draft because I guessed a bunch of things.

amcasey · 2020-04-04T01:07:42Z

src/executeCommandLine/executeCommandLine.ts

        let statistics: Statistic[];
        const compilerOptions = program.getCompilerOptions();
+
+        if (compilerOptions.expensiveStatements) {


They're really only diagnostics for the pretty-printing. Otherwise, they seem like they belong with the statistics.

amcasey · 2020-04-04T01:08:19Z

src/compiler/diagnosticMessages.json

@@ -5733,5 +5737,9 @@
    "An optional chain cannot contain private identifiers.": {
        "category": "Error",
        "code": 18030
+    },
+    "Checking this statement may result in the creation of as many as {0} types and {1} symbols.": {
+        "category": "Warning",


This is the first Warning. It feels pretty distinct to me, but it doesn't really matter one way or the other.

amcasey · 2020-04-04T01:08:37Z

src/compiler/diagnosticMessages.json

+    },
+    "Checking this statement may result in the creation of as many as {0} types and {1} symbols.": {
+        "category": "Warning",
+        "code": 19000


Again, since this felt like a new kind of diagnostic, I added a new code range.

amcasey · 2020-04-04T01:09:24Z

src/compiler/checker.ts

                checkSourceElementWorker(node);
+
+                if (node.kind >= SyntaxKind.FirstStatement && node.kind <= SyntaxKind.LastStatement) {


This could be more aggressive about skipping work if the statements will never be reported.

amcasey · 2020-04-04T01:10:13Z

src/compiler/checker.ts

+
+                        let i = 0;
+                        for (const record of expensiveStatements) {
+                            if (record.typeDelta < typeDelta) {


In teamroom discussions, there has seemed to be more interest in type count than symbol count, so I used that for sorting.

I wonder if it'll be able to detect groups of statements that are cheap individually but are executed so often that they are actually expensive.

eg. I have some type ConvertNumbersToStringsInInterface<I> that will be not so expensive as individual call.

But let's say I use it on ViewStyle interface which can be included 100s of properties.

And then let's say I use it in every view I've got in my app. Then I can end up having 1000s of relatively cheap operations instead of 1 huge one.

Maybe it'll be possible to track the total time of each type being 'part of' for other operations?

@pie6k I had an earlier version that listed top calls by both individual invocation and aggregate, but it proved to be too difficult to read the output. Anecdotally, this unsophisticated heuristic has been helpful in some real world projects, but, as you point out, it has numerous limitations.

amcasey · 2020-04-04T01:10:51Z

@sheetalkamat Yes, I'll add a test if people are happy with the design. 😄

pie6k · 2020-04-05T09:40:55Z

Hey, do you think it'd be possible/good idea to somehow group diagnostic report items?

eg. Some projects might have relatively 'cheap' typing operation but used so many times that it might take most of the time spent processing types (it can be the case with packages like styled-components that are often used 100s of times in the codebase).

In such cases, being able to detect that and extract as a group could be a good insight. Also, optimizing ts server for such cases even a little bit could make a big difference.

Another question is if it's possible to run those diagnostics in 'ts server mode'? I mean running it for single operations in IDE (eg. requesting auto suggestions).

Currently, I was able to run your diagnostics with tsc command and it showed info about 'initial build' (taking around 8s). While it's quite a lot of time, I actually don't mind initialization time so much as 'real-time' delay of suggestions, auto-imports, issues underlining, etc.

In my case, being able to have diagnostics for single editor operations could be very helpful.

Or maybe it could be possible to 'detect' long editor operations and then prepare diagnostic info for those (and send them to you guys?).

In general, my priorities about 'performance' are way more focused on 'real-time' performance than on initial load/preparing ts-server for the first time. I'd be happy waiting twice as long for ts-server init and then having twice shorter real-time editor operations.

@amcasey - well... if you want, I can give you access to the private repo I've got issues with. I'd like to avoid any leaks of it, but it's not rocket science or top-secret code. DM me on Twitter (@pie6k) if you'd like to set it up.

Another thing is, that some of the issues I've got in my codebase are actually quite tricky to reproduce - ie they happen relatively often, but I have no idea what causes them. I don't want to run verbose logging all the time (as I think, but not sure, this is slowing everything even more) and makes logs so huge I have no idea what's going there at all. But if I have logs turned off and I have the issue - in order to collect logs I have to restart ts-server which wipes out the issue as it starts to work fine after ts-server restart.

orta · 2020-04-14T20:09:14Z

@typescript-bot pack this

typescript-bot · 2020-04-14T20:09:15Z

Heya @orta, I've started to run the tarball bundle task on this PR at f6ada20. You can monitor the build here.

orta · 2020-04-14T20:41:23Z

Ah, it's possible this can't be done because it's a draft PR

amcasey · 2020-04-14T20:43:17Z

@orta I don't think so - I routinely pack draft PRs

amcasey · 2020-04-14T20:53:27Z

@pie6k Sorry for the delay, I thought I'd already posted replies - I'd certainly thought through them.

We're starting with batch/command line metrics because the scenarios are much more clear cut and reproducible. The problem with interactive scenarios is that a lot of operations are incremental and the exact order of operations makes it very tricky to identify or reproduce the causes of problems.

Building up a profiling ecosystem for TypeScript will be a long journey and this is just the first step.

Regarding your particular setup, if you're on Windows, it may be possible to keep very-inexpensive ETW logging running in the background that would give us nearly all the information that the text log would contain. Setup is a bit involved, but let me know if you're interested and I can write something up.

pie6k · 2020-04-14T21:03:02Z

@amcasey I'm using macOS. If something similar is possible - let me know, I'd be happy to do it, even if the setup is involved.

amcasey · 2020-04-14T21:28:27Z

@pie6k We've had trouble finding posix equivalents to ETW, but we're open to suggestions. DTrace seemed like the most obvious, but it didn't play nicely with our other infrastructure, so we've been sticking with text logs for now.

orta · 2020-04-15T15:27:29Z

Specific to Apple, it has a new and reasonably similar logging system - I took a look at trying to access it form a C++ cli app, and it relies on the objc runtime - which I originally thought was a blocker. Looks like @rebornix got that working in napi-spellchecker. So I think I could probably build this if we need it.

amcasey · 2020-05-04T21:55:36Z

@typescript-bot pack this

typescript-bot · 2020-05-04T21:55:39Z

Heya @amcasey, I've started to run the tarball bundle task on this PR at f6ada20. You can monitor the build here.

amcasey · 2020-05-05T01:20:49Z

@typescript-bot pack this

typescript-bot · 2020-05-05T01:21:05Z

Heya @amcasey, I've started to run the tarball bundle task on this PR at f6ada20. You can monitor the build here.

amcasey · 2020-05-05T23:41:04Z

@typescript-bot pack this

typescript-bot · 2020-05-05T23:41:10Z

Heya @amcasey, I've started to run the tarball bundle task on this PR at f6ada20. You can monitor the build here.

amcasey · 2020-05-06T01:22:13Z

I thought it was yesterday's GH instability, but it looks like the bot really can't pack this. I'll see if rebasing helps.

amcasey · 2020-05-06T01:37:44Z

@typescript-bot pack this

DanielRosenwasser · 2020-07-23T02:45:50Z

@typescript-bot pack this

typescript-bot · 2020-07-23T02:46:17Z

Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at beb01a3. You can monitor the build here.

amcasey · 2020-07-23T18:17:39Z

@typescript-bot pack this

typescript-bot · 2020-07-23T18:17:44Z

Heya @amcasey, I've started to run the tarball bundle task on this PR at 2e794fe. You can monitor the build here.

typescript-bot · 2020-07-23T18:34:16Z

Hey @amcasey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/80545/artifacts?artifactName=tgz&fileId=3DD627415FC1A179E14077E55421345032261178C9B7B2FCC5B5F51DDC52A82502&fileName=/typescript-4.0.0-insiders.20200723.tgz"
    }
}

and then running npm install.

There is also a playground for this build.

amcasey · 2020-07-23T20:04:54Z

@typescript-bot pack this

typescript-bot · 2020-07-23T20:04:58Z

Heya @amcasey, I've started to run the tarball bundle task on this PR at 5a8d335. You can monitor the build here.

amcasey · 2020-07-23T21:39:02Z

@typescript-bot pack this

typescript-bot · 2020-07-23T21:39:05Z

Heya @amcasey, I've started to run the tarball bundle task on this PR at eabb93d. You can monitor the build here.

typescript-bot · 2020-07-23T21:55:22Z

Hey @amcasey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/80565/artifacts?artifactName=tgz&fileId=A7B6365D0EBF5ACC9FDA621ED49A94C8E1B7ADBA9DC8104D2D66601560A284BF02&fileName=/typescript-4.0.0-insiders.20200723.tgz"
    }
}

and then running npm install.

There is also a playground for this build.

...rather than just dropping the first one, now that .d.ts file checking is front-loaded.

@RyanCavanaugh

Thanks to @RyanCavanaugh for the suggestion!

amcasey · 2020-08-15T00:02:22Z

Superseded by #40063.

amcasey commented Apr 4, 2020

View reviewed changes

amcasey requested review from sheetalkamat, DanielRosenwasser and ahejlsberg April 4, 2020 01:10

orta mentioned this pull request Apr 14, 2020

Debugging long check times with TypeScript? #24435

Closed

DanielRosenwasser mentioned this pull request May 4, 2020

Guidance on reducing heap usage of large typings? #37214

Open

amcasey force-pushed the ExpensiveStatements branch from f6ada20 to d99f547 Compare May 6, 2020 01:34

amcasey force-pushed the ExpensiveStatements branch from beb01a3 to 2e794fe Compare July 23, 2020 18:17

typescript-bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Jul 23, 2020

amcasey mentioned this pull request Jul 23, 2020

Slow compilation likely caused by too many types #39678

Closed

amcasey mentioned this pull request Jul 23, 2020

Custom instrumentation to identify expensive overload resolution instances #35488

Closed

amcasey added 10 commits August 4, 2020 12:45

Track a fixed number of most-expensive statements

52e2ce9

Take the number of statements to report from a switch

ebf8288

Report the most expensive statements as diagnostics

86fbb03

Process .d.ts files before other files

3fac294

Ignore expensive statements in .d.ts files

3ddf1ff

...rather than just dropping the first one, now that .d.ts file checking is front-loaded.

Include type alias and interface declarations

36b2865

Thanks to @RyanCavanaugh for the suggestion!

Update baselines to reflect earlier processing of .d.ts files

8f5e138

Suppress ancestors of expensive statements to reduce noise

2e2dfd4

Skip unnecessary work more thoroughly

906c503

Add experimental support for expensive expressions

9d24de1

amcasey force-pushed the ExpensiveStatements branch from eabb93d to 9d24de1 Compare August 5, 2020 00:02

amcasey added 2 commits August 4, 2020 17:30

Add missing semicolon

e15bec8

Discount literal types

ca645f8

amcasey mentioned this pull request Aug 15, 2020

[Don't Merge] Print out type and symbol count deltas for each statement #37417

Closed

amcasey closed this Aug 15, 2020

		checkSourceElementWorker(node);

		if (node.kind >= SyntaxKind.FirstStatement && node.kind <= SyntaxKind.LastStatement) {

Add a switch for reporting expensive statements #37785

Add a switch for reporting expensive statements #37785

Conversation

amcasey commented Apr 4, 2020

amcasey commented Apr 4, 2020

amcasey Apr 4, 2020

Choose a reason for hiding this comment

amcasey Apr 4, 2020

Choose a reason for hiding this comment

amcasey Apr 4, 2020

Choose a reason for hiding this comment

amcasey Apr 4, 2020

Choose a reason for hiding this comment

amcasey Apr 4, 2020

Choose a reason for hiding this comment

pie6k Apr 5, 2020

Choose a reason for hiding this comment

amcasey Apr 14, 2020

Choose a reason for hiding this comment

amcasey commented Apr 4, 2020

pie6k commented Apr 5, 2020 • edited Loading

orta commented Apr 14, 2020

typescript-bot commented Apr 14, 2020 • edited Loading

orta commented Apr 14, 2020

amcasey commented Apr 14, 2020

amcasey commented Apr 14, 2020

pie6k commented Apr 14, 2020

amcasey commented Apr 14, 2020

orta commented Apr 15, 2020 • edited Loading

amcasey commented May 4, 2020

typescript-bot commented May 4, 2020 • edited Loading

amcasey commented May 5, 2020

typescript-bot commented May 5, 2020 • edited Loading

amcasey commented May 5, 2020

typescript-bot commented May 5, 2020 • edited Loading

amcasey commented May 6, 2020

amcasey commented May 6, 2020

DanielRosenwasser commented Jul 23, 2020

typescript-bot commented Jul 23, 2020 • edited Loading

amcasey commented Jul 23, 2020

typescript-bot commented Jul 23, 2020 • edited Loading

typescript-bot commented Jul 23, 2020 • edited Loading

amcasey commented Jul 23, 2020

typescript-bot commented Jul 23, 2020 • edited Loading

amcasey commented Jul 23, 2020

typescript-bot commented Jul 23, 2020 • edited Loading

typescript-bot commented Jul 23, 2020 • edited Loading

amcasey commented Aug 15, 2020

pie6k commented Apr 5, 2020 •

edited

Loading

typescript-bot commented Apr 14, 2020 •

edited

Loading

orta commented Apr 15, 2020 •

edited

Loading

typescript-bot commented May 4, 2020 •

edited

Loading

typescript-bot commented May 5, 2020 •

edited

Loading

typescript-bot commented May 5, 2020 •

edited

Loading

typescript-bot commented Jul 23, 2020 •

edited

Loading

typescript-bot commented Jul 23, 2020 •

edited

Loading

typescript-bot commented Jul 23, 2020 •

edited

Loading

typescript-bot commented Jul 23, 2020 •

edited

Loading

typescript-bot commented Jul 23, 2020 •

edited

Loading

typescript-bot commented Jul 23, 2020 •

edited

Loading