Release 0.5.3

[nnansters]

Hey all,

Niels from NannyML engineering here to inform you about our 0.5.3 release 🚀 (our release rocket is currently outpacing Artemis 1!).

Installing / upgrading

You can get this latest version by using pip:

pip install -U nannyml

An older NannyML version (0.5.0) is available on conda-forge as well, but we have some work on getting the newer versions to build. More on that soon!

What's new?

We've completely overhauled the way we determine the "stability" of our estimations, i.e. the degree of confidence you should put in an estimation.

We used to do this by determining a minimum Chunk size for a given estimation. When a Chunk would contain fewer observations than this minimum, we would raise a warning about the quality of the estimations.
This was quite complex however, it caused significant complications in our codebase and on top of that was not very intuitive and transparent for our users.

Our new revamped approach estimates the standard error for our estimators as a way to measure sampling error. Some results will now include a sampling error value for each Chunk as a measure of reliability. When the sampling error gets bigger, so do the odds of some estimated performance drop actually being an effect of sampling.

So, what does that mean for you?

• A sampling error value will be provided per metric per Chunk in the result data for reconstruction error multivariate drift calculator, all performance calculation metrics and all performance estimation metrics.
• Confidence bands are now also based on this sampling error. It is calculated as +/- 3 times the standard error. It is displayed as a range around the calculated values in CBPE and reconstruction error multivariate drift calculator.

Be sure to check out our in-depth documentation on how it works or dive right into the implementation.

What's changed?

An outdated version of Numpy was causing trouble so we've been spending some time on updating our dependencies where possible. Since our statistics still show quite some usage of Python 3.7 we're making sure to keep our dependencies compliant as long as we can.

What's up next?

We've got some big, very exciting stuff releasing later this week, so keep your eyes peeled! 👀

👋 Reliable greetings! 👋

Niels