👌 More robust covariance_matrix calculation

[s-weigand]

Follow up PR for #704 , which implements a more robust calculation of the covariance_matrix for results.

Change summary

• 🧪 Adds benchmarks for glotaran.analysis.optimize._create_result
• 👌 Uses SVD of the estimated jacobian to calculate covariance_matrix

Not sure what was up with the calculation of the mask in the original StackOverflow answer

tol = np.finfo(float).eps*s[0]*max(res.jac.shape)
w = s > tol

So I changed it to

w = s**2 > np.finfo(float).eps

which should be sufficient to prevent ZeroDivisionError.

If any of you can think of a reason why the biggest singular value and the shape of the jacobian might be of importance to create a filter mask please tell me 😄

Checklist

• ✔️ Passing the tests (mandatory for all PR's)
• 🧪 Adds new tests for the feature (mandatory for ✨ feature and 🩹 bug fix PR's)

[github-actions]

👈 Launch a binder notebook on branch s-weigand/pyglotaran/roust-stderror

[codecov]

Codecov Report

Merging #706 (b33750e) into main (ed035d6) will not change coverage.
The diff coverage is 100.0%.

@@          Coverage Diff          @@
##            main    #706   +/-   ##
=====================================
  Coverage   81.7%   81.7%           
=====================================
  Files         79      79           
  Lines       3921    3921           
  Branches     689     689           
=====================================
  Hits        3206    3206           
  Misses       595     595           
  Partials     120     120           

Impacted Files	Coverage Δ
glotaran/analysis/optimize.py	89.2% <100.0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed035d6...b33750e. Read the comment docs.

[s-weigand]

Also if we would refactor optimize_problem to return problem, ls_result, free_parameter_labels, termination_reason instead of
_create_result(problem, ls_result, free_parameter_labels, termination_reason) and just chain them together in optimize, we could get rid of using pickle in the benchmarks which would also make it more maintainable (+ codacy won't complain anymore :P)

[jsnel]

We will have to dive a little deeper this time and not only check if the examples run, but how they run (minimization pathway) and compare the two.

[sourcery-ai]

Sourcery Code Quality Report

✅  Merging this PR will increase code quality in the affected files by 1.97%.

Quality metrics	Before	After	Change
Complexity	5.00 ⭐	4.20 ⭐	-0.80 👍
Method Length	78.18 🙂	73.38 🙂	-4.80 👍
Working memory	13.43 😞	13.18 😞	-0.25 👍
Quality	54.06% 🙂	56.03% 🙂	1.97% 👍

Other metrics	Before	After	Change
Lines	274	304	30

Changed files	Quality Before	Quality After	Quality Change
benchmark/benchmarks/integration/ex_two_datasets/benchmark.py	86.69% ⭐	83.02% ⭐	-3.67% 👎
glotaran/analysis/optimize.py	48.87% 😞	48.42% 😞	-0.45% 👎
glotaran/builtin/models/kinetic_spectrum/test/test_spectral_constraints.py	51.06% 🙂	51.25% 🙂	0.19% 👍

Here are some functions in these files that still need a tune-up:

File	Function	Complexity	Length	Working Memory	Quality	Recommendation
glotaran/analysis/optimize.py	_create_result	14 🙂	241 ⛔	23 ⛔	28.73% 😞	Try splitting into smaller methods. Extract out complex expressions
glotaran/builtin/models/kinetic_spectrum/test/test_spectral_constraints.py	test_spectral_constraint	0 ⭐	302 ⛔	11 😞	51.25% 🙂	Try splitting into smaller methods. Extract out complex expressions
glotaran/analysis/optimize.py	optimize_problem	3 ⭐	141 😞	14 😞	54.29% 🙂	Try splitting into smaller methods. Extract out complex expressions
benchmark/benchmarks/integration/ex_two_datasets/benchmark.py	IntegrationTwoDatasets.setup	0 ⭐	95 🙂	10 😞	69.70% 🙂	Extract out complex expressions

Legend and Explanation

The emojis denote the absolute quality of the code:

• ⭐ excellent
• 🙂 good
• 😞 poor
• ⛔ very poor

The 👍 and 👎 indicate whether the quality has improved or gotten worse with this pull request.

---

Please see our documentation here for details on how these metrics are calculated.

We are actively working on this report - lots more documentation and extra metrics to come!

Help us improve this quality report!

[github-actions]

Benchmark is done. Checkout the benchmark result page.
Benchmark differences below 5% might be due to CI noise.

Benchmark diff

All benchmarks:

       before           after         ratio
     [ed035d6e]       [b33750e0]
             166M             179M     1.08  IntegrationTwoDatasets.peakmem_create_result
             196M             195M     0.99  IntegrationTwoDatasets.peakmem_optimize
+         198±4ms          224±6ms     1.13  IntegrationTwoDatasets.time_create_result
        4.39±0.1s       4.52±0.01s     1.03  IntegrationTwoDatasets.time_optimize

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[jsnel]

Tested ok!