[prism-cli] Memory leak while validating (Deluxe Edition)

[nulltoken]

Describe the bug

This is a follow up to #1881.

We've discovered another leaky scenario. See below for the repro case.

Note: This may look like a contrived scenario but this is the trimmed down version of a production real issue.
Basically we host Prism and use it to validate the traffic (requests/responses). Every time the specs are updated, which may happen many times a week, we trigger a "refresh" which makes Prism load and parse the new version of the specs and build a new array of IHttpOperation from it. Each time this happens, we leak a little memory.

To Reproduce

A complete standalone repro case is available at https://github.com/nulltoken/prism_repro

It triggers a loop of 5000 request and response validations, capturing the amount of used Heap every 10 steps, and triggering a garbage collection between every step.

Running yarn repro shows the following output where one can clearly see the heap growing.

yarn run v1.22.5
$ yarn node -r ts-node/register --unhandled-rejections=strict --expose-gc ./src/repro.ts

i       HeapUsed
0       113844232
10      108756224
20      108771128
30      108804208
40      108840448
50      108873944
60      108886160
[...snipped for brevity...]
4930    120451312
4940    120473336
4950    120536456
4960    120518088
4970    120545088
4980    120568632
4990    120589472

Running prism v4.4.1
Done in 27.91s.

Expected behavior

As far as my understanding goes, there are at least two levels of JsonSchema/ValidatingFunction caching: one at the Prism level and a second one at Ajv level.

In our scenario, I believe that what would make the most sense would be a way to "clear" the caches every time we "refresh" the list of operations.

Thoughts?

/cc @chohmann @radzserg @lukasz-kuzynski-11sigma

[nulltoken]

@radzserg @lukasz-kuzynski-11sigma 👋 Any feedback? Have you been able to repro this on your side?

Sadly this compelled us to rollback to Prism 4.1.2. :-/

[chohmann]

Hey team! Please add your planning poker estimate with ZenHub @Amjcraft @lukasz-kuzynski-11sigma @radzserg @rainum @EdVinyard

[ghost]

Hi @nulltoken

Thank You for your patience and a finding of a bug! :)

I can confirm that once prism gets reloaded upon spec change and performing some requests there is some memory leak. The source of it is https://www.npmjs.com/package/ajv package that creates new function for each validated schema. The way the validation function is being created (via new Function - similar to eval) prevents GC from removing it. I've performed synthetic tests and isolated ajv to confirm. There is no change for ajv to change that approach. Cleaning cache wasn't helping, removing schema also. That is just nature of creating a new code through eval or new Function

Possible solutions we took into account:

1. Changing validator I couldn't find a validator that match our needs
2. Moving prism to seperate process - too big change ATM
3. Moving validation to seperate process and restart it on spec change. That would help and probably is the only solution feasible solution but that change it still too big to introduce right now. If you're happy to do so, then PRs are welcome.

Moreover what I found is that I'm really curious why you went back to version 4.1.2? AJV was there and causing memory leaks on spec reload too. Why is running 4.4.x is problematic? How big is memory growth?

Also I'm very curious about the general increase of memory usage. I've tested it locally by changing the spec every second and making few requests in that time and the increase wasn't so big.

Is it feasible for You to just restart the prism once a day?

The cache introduced by my previous fix is not a source of a leak since it relies on WeakMap that releases memory once cached keys (schemas from operations) are not available and I've confirmed that.

Closing for now. If we find this issue serious this will be obviously reopened.