Very slow tests since vesion 12.2.0

[lousley]

Hello together

Since we updated ng-mocks to version 12.2.0 on our project (Angular 12 with unittests jasmine / karma), we are seeing extreme performance issues. Do you have any spontaneous idea what this could be?
I did some debugging and saw that defineTouches / generateTouches fills the touches SET with extremely many entries.

We only use in some tests ngMocks and MockComponent.

Thanks for any feedback.

[satanTime]

Hi @lousley , might I ask you to share a test which demonstrates the degradation?

[satanTime]

Also, to confirm - 12.5.0 also has the issue, right?

[lousley]

yes i have the problem also with version 12.5. interestingly i just saw that if i have a testfile without ngmocks imports or functionalities but i have many imports and providers it becomes extremely slow. defineTouches and geneateTouches is probably the problem even if no ng-mocks is needed in the corresponding file?

[lousley]

performance profile:

[lousley]

is there any way to turn off/override this generatetouches?

[LPCmedia]

seeing the same thing in ng 13.0.2 + ng-mocks 12.5.0. Performance is especially slow when MockModule is added.

Same Unit Test using:	Seconds
Mock Module + MockComponent	38.351
MockComponents	6.999
Benchmarks:
NO_ERRORS_SCHEMA + CUSTOM_ELEMENTS_SCHEMA	3.742s
using actual modules	6.558s

with ( MockModule + MockComponent)

with (MockComponents)

without:

using the actual modules:

[satanTime]

Hi there, thanks for the info and support.

might you also downgrade ng-mocks to <12.2.0 and share results?

looks like another change in A13 will cause huge refactoring, so I could try to find the issue which impacts performance degradation and to fix it there too.

[satanTime]

Regarding MockModule, it depends in the amount of declarations recursively. Because it goes through the whole declaration tree of the module and mocks every thing. So to mock AppModule will take much more time than ButtonModule.
Another issue here is jest, because it runs tests in isolated runtimes, it bootstraps every runtime from scratch and causes mocking for the same things. That also kills performance, but it's a separate story.

Considering that a simple test takes 6.5 seconds on your side, I would say that your app looks heavy and can have a lot of declarations, and that's why ng-mocks needs enormous time to mock all of them.

I'll try to find a way to add metrics so we could debug where ng-mocks looses time.

[LPCmedia]

sounds like a good plan on the metrics.

As a guideline we will try to avoid MockModule, we are supposed to be shallow testing anyway 🤷 . But the atomic option is to appealing.

Is there a separate issue logged for jest performance improvements ?

[lousley]

In my case i have the following TestBed

I think the problem is the AwbTestingModule with many declarations / imports / exports and providers as:

If I remove the TestingModule and use only the required declarations / imports / providers in the corresponding test file, then the test is already much faster. I also see that in this testfile no ng-mocks stuff is needed at all, nevertheless it is very slow.
We use this TestModule in quite a few places, because it is easier and more convenient than defining the same things over and over again.
Is there a way to tell ng-mocks to ignore this module when generatingTouches / definingTouches? Or am I looking at this wrong and the problem is somewhere else?

[LPCmedia]

Hi there, thanks for the info and support.

might you also downgrade ng-mocks to <12.2.0 and share results?

looks like another change in A13 will cause huge refactoring, so I could try to find the issue which impacts performance degradation and to fix it there too.

assuming you mean : ng-mocks 12.1.2 with ng13. If that is the case the tests will not run:

ng-mocks cannot install own spec reporter. This affects its core features for MockInstance and MockRender. Please report an issue on github. If you use jest v27, please add to its config testRunner=jest-jasmine2 for now and upvote the issue on github: jestjs/jest#11483.

I can downgrade to 12.4.0 only. I re-ran the same test on both versions below. The 30+ seconds with MockModule above seems an anomaly. But its still significantly slower in 12.5.0.

Test	12.5.0 + ng13.0.2	12.4.0 + ng13.0.2
Mock Module + MockComponent	14.386 s	5.34 s
MockComponents	6.618 s	4.23 s
Benchmarks:
NO_ERRORS_SCHEMA + CUSTOM_ELEMENTS_SCHEMA	3.742 s	3.104 s
using actual modules	6.558 s	6.383 s

[gagle]

I can also confirm that. We are using Angular 12 + ng-mocks 12.1.0 and any version above 12.1.0 is extremely slowing down tests.

[satanTime]

Hey @gagle, might you provide how much time it needed for the test above?

[lousley]

@satanTime: i would also like to share my time difference:

Version 12.1.2: 277.31s
Version >12.1.2: 1200s

This based on 5500 tests

[satanTime]

thanks for the numbers, ill try to figure out how to optimise that.

[lousley]

any updates? :S

[satanTime]

Hi,

not much, jest doesn't want to merge the fix which would simplify the flow. And I haven't had time yet to dig deeper and to find alternative solutions.

Also, feel free to post any findings here. For example, if you see that some functions cause the issue and others aren't. It would really help once I'm on fixing it.

[rbirkgit]

I cannot get our tests to even start after updating to angular 13 and ng-mocks v13.

I start the unit tests with yarn test and after compilation the chrome browser starts up and then just hangs (using cpu). Only after I remove all references to ng-mocks I can get the tests running again. Doesn't seem to matter what you mock. The moment we add a single reference to ng-mocks (e.g. MockComponent, MockProcider etc), all our tests hangs right away.

I tried downgrading to ng-mocks 12.5 but that doesn't seem compatible with ang13. Getting "declaration has been passed into ng-mocks without Angular decorators. " errors.

In our ang12 version and ng-mocks 12.2 all worked fine.

[satanTime]

Hi @rbirkgit,

Thanks for the info. Looks like there is a recursion.
Would it be possible to share your project with me for investigation?

[rbirkgit]

Unfortunately it's a big project with many dependencies and don't think we can share the code. The strange thing is, removing all references to ng-mocks all is fine (just to get a base line). Then all I do is adding one place with MockComponent(xxx) and boom, it hangs using v13.

Not sure if there are some ways to get more debug info or how to test what could cause the issue?

[rbirkgit]

All I do is add this in one of our tests and the whole thing hangs before even the first test starts. No other use of ng-mocks anywhere.

  import { MockComponent } from 'ng-mocks';

  beforeEach(() => {
    TestBed.configureTestingModule({
      declarations: [MockComponent(FileInputComponent)],

[satanTime]

Yep, I think some root installs don't work well. could you check for example not MockComponent, but MockRender or ngMocks.*?

I'll prepare a special debug version which should shed light on the source of the issue, would it be possible for you to run it and share with me its output?

[rbirkgit]

I would be happy to test and run debug versions and share results.

[rbirkgit]

Played more today and my latest conclusion is that it's not actually hanging, it really really slow. We have a 60s timeout where it will disconnect the test and that what I am experiencing. And the browser then just sits there.

So we have a shared MockModule where we have mocked the most common services etc we use. Then I went ahead and deleted every spec file we have except one, which is a bare minimum. Just test that component is created.

Now I added a dummy empty component and added MockComponent(MyTestComponent) to this minimal test. Still hangs, or rather timeouts ater 60sec.

So started removing mocks in our mock module and after removing enough of them, the app.component test passed taking 25s. It usually is much faster, much less than 1sec. When we add the MockComponent things explode exponentially.

[satanTime]

Do you mean 25s without ng-mocks? Or without it it's 1s?

[rbirkgit]

The test takes 0.3sec normally. The moment I add a single line mocking a dummy empty component, it takes 25.4sec

MockComponent(MyTestComponent)

[rbirkgit]

This is my code in my test:

  beforeEach(() => {
    TestBed.configureTestingModule({
      declarations: [AppComponent, MockComponent(MyTestComponent)],
      imports: [MockModule]

I had to delete mocks in our MockModule one by one until the test got under the 60sec timeout. Not sure why code in our MockModule should affect anything when adding the MockComponent(xxx) call.

[lousley]

we had the same problem, but since we upgraded to angular 13.2.4, this problem is no longer so extreme. What version are you running and what does your test.ts file look like?

[satanTime]

Aha.

One question. In test.ts, where it has initTestEnvironment, do you have BrowserDynamicTestingModule or a different one?

Also it would be nice to see all dependencies in your project from package.json.

[rbirkgit]

We are using Angular 13.3.0

test.ts is the default one:

declare const require: {
  context(
    path: string,
    deep?: boolean,
    filter?: RegExp
  ): {
    <T>(id: string): T;
    keys(): string[];
  };
};

getTestBed().initTestEnvironment(BrowserDynamicTestingModule, platformBrowserDynamicTesting());
const context = require.context('./', true, /\.spec\.ts$/);
context.keys().map(context);

[satanTime]

Hi @rbirkgit,

could give me a favor and test the next conditions:

preconditions: you have one spec file which needs 25 secs to be run.
in its test, please, remove MockComponent and simply pass its component into TestBed like a test without ng-mocks.
Before the very first describe in this file, please add ngMocks.autoSpy('jasmine');.
The idea is that ng-mocks has been imported, but isn't used directly.

Now, please check how much time is needed to execute the test.
if it needs more than 1 sec, please keep beforeEach as it is, but delete all it.
Now add one it like that:

it('test', () => {
  expect(1).toEqual(1);
});

Please check how much time it needs.

after that, please add the same it again, so there would be 2 of them.

Please check how much time it needs.

---

At the end, you should have 3 numbers:

• how much test needs with one import of ng-mocks
• how much beforeEach consumes with one simple test
• how much beforeEach consumes with two simple tests

[rbirkgit]

I did the first step as you described. I removed all spec files except one simple one with one it that only tests for component creation (default test). This runs in less than a second. Added the line you described above the describe:

import { ngMocks } from 'ng-mocks';

ngMocks.autoSpy('jasmine');

describe('AppComponent', () => {
...

Now tests disconnects after 60s (timeout). Without the ngMocks line, again it's less than a second.

[satanTime]

Thanks. I think I know where the issue is now.

Not an easy fix, but I'll try my best.

[satanTime]

And to confirm: MockModule is super fat, isn't it?

[rbirkgit]

Yes, our MockModule is fat. I started removing code from it and it got faster, but still very slow. Removed enough, I got it to 25sec. Removed most, 15sec. We also have what may or may not be a separate issue that after the test, it still hangs. Meaning it says the it() passed but testing is still sitting there not exiting. Removing ng-mocks, it exits right away.

I know this is not much to go on but thought I try to describe. Our tests could be optimized with mocking things more properly. Right now it's not mocking a component's every dependent service and some are running during the tests, which using other services, along with observables etc. That where we wanted to cleanup the tests with ng-mocks. It was very promising and worked great in the ang12 env. Thanks for looking into it. Would be awesome if we can continue using it. It's a very nice tool.

[satanTime]

Thanks for the feedback. I'll keep you all posted.

[satanTime]

Hi @rbirkgit,

not sure if it helps, nevertheless, could you try one of those things:
in the hanging file before the very first describe to add:
ngMocks.globalKeep(MockModule);,
and if it doesn't help then to try ngMocks.globalReplace(MockModule, MockModule);.

[rbirkgit]

Tested both options. No difference.

[satanTime]

I was able to reproduce the issue. Working on a fix.

[satanTime]

Hi all,

could you verify that this build solves the issue?
simply put ng-mocks from the archive to node_modules of your project: ng-mocks.zip.

Also, please delete .angular in the root directory of your project.

[rbirkgit]

I will try when I get back to my computer later today. Thanks!!

[rbirkgit]

Tested it. Success!! It's as fast as v12 now and works! I need to pinch myself and try again :)

[satanTime]

Awesome!!! Thanks for confirming. I'll release it tomorrow.

[rbirkgit]

Now all tests run and normal speed. Got 6 tests failing of 350 in this project. Need to figure out next if it's a new issue or we did something wrong that now got exposed. Four them has MockModule(SDKModule) which is one of our modules that contains some services etc. It's causing:

NullInjectorError: R3InjectorError(DynamicTestModule)[ApplicationModule -> ApplicationRef -> ApplicationInitStatus -> InjectionToken Application Initializer -> [object Object] -> TimeoutService -> SessionService -> HttpClient -> HttpClient]: 
NullInjectorError: No provider for HttpClient!

This was working before. I'm not sure yet if it's an ang13 update issue, or a new ngMocks problem or our code just bad. But thought I post here just in case. Will look at it more on Monday.

[satanTime]

Why do you not want to use MockBuilder? It resolves the hell of dependencies.

Anyway, if there is a bug, please create a new issue.

[satanTime]

Hi @lousley and @LPCmedia, could you also check whether it works for you?

And I would like to thank your for helping with debug information. It was very useful.

[lousley]

HI @satanTime i can confirm that the performance is back as before. thanks for the fix. really nice!

[satanTime]

v13.3.0 has been released and contains a fix for the issue. Feel free to reopen the issue or to submit a new one if you meet any problems.