Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release 0.5.2 #1406

Merged
merged 29 commits into from
May 31, 2024
Merged

release 0.5.2 #1406

merged 29 commits into from
May 31, 2024

Conversation

PaulJonasJost
Copy link
Collaborator

@PaulJonasJost PaulJonasJost commented May 26, 2024

dweindl and others added 26 commits April 17, 2024 10:03
Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>
…ectly. (#1364)

Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>
* Fix log of zero in hierarchical calculate_nllh by masking

* Fix in `compute_bounded_optimal_scaling_offset_coupled`

* Update numerical solver with data mask

---------

Co-authored-by: Jonas Arruda <69197639+arrjon@users.noreply.github.com>
* Added option to sample startpoints of a problem, from the problem directly. Also safety checks for startindices like "all" or "clustered"

* Fixed error in correlation matrix, if there were nn values. Also safety checks in general for start indices. Added a test for correlation matrix

* log filtering in process_start_indices

* update docstring

---------

Co-authored-by: Maren Philipps <maren.philipps@uni-bonn.de>
For ESSOptimizer and SacessOptimizer:
* supporess divide-by-zero warnings
* report number of function evaluations
* Update setup.cfg

Remove python 3.9

* Update ci.yml

Remove python 3.9 and update to 3.11

* Update deploy.yml

Change python 3.9 by 3.11

* Update ci.yml

Add python 3.10

* Update setup.cfg

Add python 3.12

* Update ci.yml

Update from python 3.11 to 3.12

* Update deploy.yml

Update from python 3.11 to 3.12

* Skip aesara tests on case of python >= 3.12 and use 3.11 for building the documentation.

* Update test/base/test_objective.py

Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>

* Update .github/workflows/ci.yml

---------

Co-authored-by: PaulJonasJost <jost.pauljonas@gmail.com>
Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>
Prevents unclear timeouts that occur with `macos-14-arm64` (see #1383).
So far, most stats from different SacessOptimizer workers have only been available from the logs.
Now they are also available via `SacessOptimizer.worker_results`.
Additionally, the total number of objective evaluations across all workers ís logged.
For `nfft<=3`, this function computed `k` by converting infinity to int64.

For the last [4 years](fdbacb4), this seemed to not cause any trouble, because `np.float64("inf").astype(int)` yielded -9223372036854775808 and the loop did not run. However, with the current `macos-14` GitHub runner, this yields 9223372036854775807, and accordingly, rather long computation times (this is what caused #1383).

I am not really sure what caused this change. macos-12 vs macos-14, arm64 vs x86, ...? all should be IEEE 754-compliant. Maybe some different integer type? 🤷‍♂️

With this change `macos-14` runners yield the same results as the other runners. However, I am not familiar with computing power spectral density and I don't know if this is indeed what should happen. It would be great if somebody more familiar with that topic could double-check. (Also, can't we use [scipy.signal.welch](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.welch.html#scipy.signal.welch) here?)
* switch to `macos-14`
  previous issues with macos-14 were a mix of #1388 and caching with seemingly random switching between macos-12 and macos-14 runners when using macos-latest
* include arch in cache keys; use uniform cache keys
* change some imports in `test/sample/test_sample.py` to be able to run subsets of tests despite missing optional dependencies
* adjust resource limits to avoid [random](https://github.com/ICB-DCM/pyPESTO/actions/runs/8945258779/job/24573887731) [failures](https://github.com/ICB-DCM/pyPESTO/actions/runs/8945870387/job/24575587898?pr=1387) such as:
```
/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/_pytest/main.py:339: PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: _cov, Hook: pytest_runtestloop
DataError: Couldn't use data file '/Users/runner/work/pyPESTO/pyPESTO/.coverage.Mac-1714771409828.local.9672.XvRVVmRx': unable to open database file
For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
  config.hook.pytest_runtestloop(session=session)
..FFFF                                                                   [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqlitedb.py", line 52, in _connect
INTERNALERROR> sqlite3.OperationalError: unable to open database file
INTERNALERROR> 
INTERNALERROR> The above exception was the direct cause of the following exception:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/_pytest/main.py", line 285, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/_pytest/main.py", line 339, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pluggy/_callers.py", line 156, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pytest_cov/plugin.py", line 339, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pytest_cov/engine.py", line 46, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pytest_cov/engine.py", line 256, in finish
INTERNALERROR>     self.cov.save()
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/control.py", line 785, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>            ^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/control.py", line 865, in get_data
INTERNALERROR>     if self._collector.flush_data():
INTERNALERROR>        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/collector.py", line 535, in flush_data
INTERNALERROR>     self.covdata.add_lines(self.mapped_file_dict(line_data))
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 124, in _wrapped
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 495, in add_lines
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 564, in _choose_lines_or_arcs
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 344, in _connect
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 287, in _open_db
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 291, in _read_db
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqlitedb.py", line 88, in __enter__
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqlitedb.py", line 54, in _connect
INTERNALERROR> coverage.exceptions.DataError: Couldn't use data file '/Users/runner/work/pyPESTO/pyPESTO/.coverage.Mac-1714771409828.local.9672.XvRVVmRx': unable to open database file
```
* add load method to Hdf5AmiciHistory
Removed a spacebar preventing header to be seen as such (#1394)
* remove random.see in favor of np.random.seed

* added import numpy as np import
* update readme links
* Add relative to READ_ME.md

---------

Co-authored-by: Doresic <domagoj.doresic@gmail.com>
* Made get_full_vector more intuitive

* Corrected he usage of get_full vector. Removed it for lb and ub in favor of lb_full and ub_full.

* Apply suggestions from code review

Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>

---------

Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>
Papers using pypesto
* As a first test change the mac version

* Add a remark to revert the runs-on

* Set Runs-on to macos-13
* Fixed doc example from generic call (that did not exists) to apropriate call

* Marked code as codeblocks
* fixing call of prior in sampling

* check if priors not None

* remove deepcopy
* Lowered samples, removed comparison of two different amount of chains

* Removed even argument, due to scipy wanting to go for "simpson" and depracating "even"

* Removed warning for `test_thermodynamic_integration`

---------

Co-authored-by: Jonas Arruda <69197639+arrjon@users.noreply.github.com>
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
* variational inference fit

* remove variational from sample

* make pymc object accessible

* save as McmcPtResult

* tests added

* add warning in write_result()
* Saves pypesto and python version to the problem.

* Update pypesto/problem/base.py

Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>

---------

Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>
@codecov-commenter
Copy link

codecov-commenter commented May 26, 2024

Codecov Report

Attention: Patch coverage is 48.75622% with 103 lines in your changes are missing coverage. Please review.

Project coverage is 83.92%. Comparing base (d8333bd) to head (c966568).

Files Patch % Lines
pypesto/variational/pymc.py 0.00% 53 Missing ⚠️
pypesto/variational/variational_inference.py 0.00% 35 Missing ⚠️
pypesto/history/amici.py 33.33% 4 Missing ⚠️
pypesto/visualize/spline_approximation.py 25.00% 3 Missing ⚠️
pypesto/variational/__init__.py 0.00% 2 Missing ⚠️
pypesto/visualize/misc.py 77.77% 2 Missing ⚠️
pypesto/objective/amici/amici_util.py 75.00% 1 Missing ⚠️
pypesto/store/save_to_hdf5.py 66.66% 1 Missing ⚠️
pypesto/visualize/model_fit.py 50.00% 1 Missing ⚠️
pypesto/visualize/ordinal_categories.py 50.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1406      +/-   ##
==========================================
- Coverage   84.50%   83.92%   -0.58%     
==========================================
  Files         157      160       +3     
  Lines       12917    13061     +144     
==========================================
+ Hits        10915    10962      +47     
- Misses       2002     2099      +97     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@PaulJonasJost PaulJonasJost changed the title release 0.5.1 release 0.5.2 May 27, 2024
GiacomoFabrini and others added 2 commits May 29, 2024 17:18
* Add pre_post_processor.setter and base_objective wrapper functions

* fix docstring & simplify implementation

* extend tests

* fix tests

* add returns to docstring

* fix max sensi orders, add test

---------

Co-authored-by: Paul Jonas Jost <70631928+PaulJonasJost@users.noreply.github.com>
Co-authored-by: Fabian Fröhlich <fabian@schaluck.com>
Co-authored-by: Fabian Fröhlich <fabian.frohlich@crick.ac.uk>
* Update Changelog and Version

---------

Co-authored-by: Fabian Fröhlich <fabian.frohlich@crick.ac.uk>
Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>
@PaulJonasJost PaulJonasJost marked this pull request as ready for review May 31, 2024 13:24
@PaulJonasJost PaulJonasJost merged commit d38005e into main May 31, 2024
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants