Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>

)

The most recent arviz should now work with the most recent scipy (but requires python>=3.10): https://github.com/arviz-devs/arviz/blob/main/CHANGELOG.md#maintenance-and-fixes-1


Closes #1354

…ectly. (#1364)

Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>

* Fix log of zero in hierarchical calculate_nllh by masking

* Fix in `compute_bounded_optimal_scaling_offset_coupled`

* Update numerical solver with data mask

---------

Co-authored-by: Jonas Arruda <69197639+arrjon@users.noreply.github.com>

* Added option to sample startpoints of a problem, from the problem directly. Also safety checks for startindices like "all" or "clustered"

* Fixed error in correlation matrix, if there were nn values. Also safety checks in general for start indices. Added a test for correlation matrix

* log filtering in process_start_indices

* update docstring

---------

Co-authored-by: Maren Philipps <maren.philipps@uni-bonn.de>

For ESSOptimizer and SacessOptimizer:
* supporess divide-by-zero warnings
* report number of function evaluations

* Update setup.cfg

Remove python 3.9

* Update ci.yml

Remove python 3.9 and update to 3.11

* Update deploy.yml

Change python 3.9 by 3.11

* Update ci.yml

Add python 3.10

* Update setup.cfg

Add python 3.12

* Update ci.yml

Update from python 3.11 to 3.12

* Update deploy.yml

Update from python 3.11 to 3.12

* Skip aesara tests on case of python >= 3.12 and use 3.11 for building the documentation.

* Update test/base/test_objective.py

Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>

* Update .github/workflows/ci.yml

---------

Co-authored-by: PaulJonasJost <jost.pauljonas@gmail.com>
Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>

Prevents unclear timeouts that occur with `macos-14-arm64` (see #1383).

So far, most stats from different SacessOptimizer workers have only been available from the logs.
Now they are also available via `SacessOptimizer.worker_results`.
Additionally, the total number of objective evaluations across all workers ís logged.

For `nfft<=3`, this function computed `k` by converting infinity to int64.

For the last [4 years](fdbacb4), this seemed to not cause any trouble, because `np.float64("inf").astype(int)` yielded -9223372036854775808 and the loop did not run. However, with the current `macos-14` GitHub runner, this yields 9223372036854775807, and accordingly, rather long computation times (this is what caused #1383).

I am not really sure what caused this change. macos-12 vs macos-14, arm64 vs x86, ...? all should be IEEE 754-compliant. Maybe some different integer type? 🤷‍♂️

With this change `macos-14` runners yield the same results as the other runners. However, I am not familiar with computing power spectral density and I don't know if this is indeed what should happen. It would be great if somebody more familiar with that topic could double-check. (Also, can't we use [scipy.signal.welch](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.welch.html#scipy.signal.welch) here?)

* switch to `macos-14`
  previous issues with macos-14 were a mix of #1388 and caching with seemingly random switching between macos-12 and macos-14 runners when using macos-latest
* include arch in cache keys; use uniform cache keys
* change some imports in `test/sample/test_sample.py` to be able to run subsets of tests despite missing optional dependencies
* adjust resource limits to avoid [random](https://github.com/ICB-DCM/pyPESTO/actions/runs/8945258779/job/24573887731) [failures](https://github.com/ICB-DCM/pyPESTO/actions/runs/8945870387/job/24575587898?pr=1387) such as:
```
/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/_pytest/main.py:339: PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: _cov, Hook: pytest_runtestloop
DataError: Couldn't use data file '/Users/runner/work/pyPESTO/pyPESTO/.coverage.Mac-1714771409828.local.9672.XvRVVmRx': unable to open database file
For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
  config.hook.pytest_runtestloop(session=session)
..FFFF                                                                   [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqlitedb.py", line 52, in _connect
INTERNALERROR> sqlite3.OperationalError: unable to open database file
INTERNALERROR> 
INTERNALERROR> The above exception was the direct cause of the following exception:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/_pytest/main.py", line 285, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/_pytest/main.py", line 339, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pluggy/_callers.py", line 156, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pytest_cov/plugin.py", line 339, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pytest_cov/engine.py", line 46, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/pytest_cov/engine.py", line 256, in finish
INTERNALERROR>     self.cov.save()
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/control.py", line 785, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>            ^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/control.py", line 865, in get_data
INTERNALERROR>     if self._collector.flush_data():
INTERNALERROR>        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/collector.py", line 535, in flush_data
INTERNALERROR>     self.covdata.add_lines(self.mapped_file_dict(line_data))
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 124, in _wrapped
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 495, in add_lines
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 564, in _choose_lines_or_arcs
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 344, in _connect
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 287, in _open_db
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqldata.py", line 291, in _read_db
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqlitedb.py", line 88, in __enter__
INTERNALERROR>   File "/Users/runner/work/pyPESTO/pyPESTO/.tox/base/lib/python3.12/site-packages/coverage/sqlitedb.py", line 54, in _connect
INTERNALERROR> coverage.exceptions.DataError: Couldn't use data file '/Users/runner/work/pyPESTO/pyPESTO/.coverage.Mac-1714771409828.local.9672.XvRVVmRx': unable to open database file
```

* add load method to Hdf5AmiciHistory

Removed a spacebar preventing header to be seen as such (#1394)

* remove random.see in favor of np.random.seed

* added import numpy as np import

* update readme links
* Add relative to READ_ME.md

---------

Co-authored-by: Doresic <domagoj.doresic@gmail.com>

* Made get_full_vector more intuitive

* Corrected he usage of get_full vector. Removed it for lb and ub in favor of lb_full and ub_full.

* Apply suggestions from code review

Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>

---------

Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>

Papers using pypesto

* As a first test change the mac version

* Add a remark to revert the runs-on

* Set Runs-on to macos-13

* Fixed doc example from generic call (that did not exists) to apropriate call

* Marked code as codeblocks

* fixing call of prior in sampling

* check if priors not None

* remove deepcopy

* Lowered samples, removed comparison of two different amount of chains

* Removed even argument, due to scipy wanting to go for "simpson" and depracating "even"

* Removed warning for `test_thermodynamic_integration`

---------

Co-authored-by: Jonas Arruda <69197639+arrjon@users.noreply.github.com>
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>

* variational inference fit

* remove variational from sample

* make pymc object accessible

* save as McmcPtResult

* tests added

* add warning in write_result()

* Saves pypesto and python version to the problem.

* Update pypesto/problem/base.py

Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>

---------

Co-authored-by: Daniel Weindl <dweindl@users.noreply.github.com>

* Add pre_post_processor.setter and base_objective wrapper functions

* fix docstring & simplify implementation

* extend tests

* fix tests

* add returns to docstring

* fix max sensi orders, add test

---------

Co-authored-by: Paul Jonas Jost <70631928+PaulJonasJost@users.noreply.github.com>
Co-authored-by: Fabian Fröhlich <fabian@schaluck.com>
Co-authored-by: Fabian Fröhlich <fabian.frohlich@crick.ac.uk>

* Update Changelog and Version

---------

Co-authored-by: Fabian Fröhlich <fabian.frohlich@crick.ac.uk>
Co-authored-by: Maren Philipps <55318391+m-philipps@users.noreply.github.com>