Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump gurobipy verison #425

Merged
merged 1 commit into from
Oct 30, 2023
Merged

Bump gurobipy verison #425

merged 1 commit into from
Oct 30, 2023

Conversation

Jacob-Stevens-Haas
Copy link
Collaborator

@Jacob-Stevens-Haas Jacob-Stevens-Haas commented Oct 30, 2023

Unfortunately, Gurobi dropped support for 9.5.2, and because it depends upon a license file, gurobipy 9.5.2 is officially broken without a manually added license. Previously, we had been prevented from upgrading to 10.x because of a bug noted in #266. I've recreated that bug with gurobipy 10.0.0, but tests pass in 10.0.1 to 10.0.3. I've also requested gurobi release a 9.5.3 with an updated limited-size license.

This PR will also fix #303

Support email to gurobi

As noted here, the size-limited license with gurobipy 9.5.2 distributed via pip expired this year, disabling that release.

Our lab maintains an open-source modeling package that includes the option to use gurobipy as a backed for a step. However, gurobipy 10.x was released shortly thereafter, and is backwards-incompatible with some of the functions they used. A researcher PR'd that capability in, but updating their code every time gurobi releases a new major version, which apparently moves quickly, is too much of a maintenance burden long-term.

In addition, this makes our library (some of the gurobi features) unavailable on python 3.11, since 9.5.2 doesn't have cp311 wheels.

The technical solution on gurobi's end is simple: update the license in the 9.x branch, release an update, and build wheels for python 3.11. Gurobi enforcing upgrades couples our development cycle to yours unneccessarily.

Alternatively, we could walk all of our developers through the license process (complicated by the fact that some are no longer in academia) in order to be able to support gurobipy 9.x. That seems like an option with a lot of email exchanges every time someone wants to debug an argument, run tests, or reproduce a low-dimensional experiment.

Recreated bug when running tests:
./test/test_optimizers.py::test_fit[optimizer11] Failed: [undefined]AttributeError: 'gurobipy.MVar' object has no attribute '__cindex__'
data_derivative_1d = (array([  2.,   4.,   6.,   8.,  10.,  12.,  14.,  16.,  18.,  20.,  22.,
        24.,  26.,  28.,  30.,  32.,  34.,  ...., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
       2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.]))
optimizer = MIOSR()

    @pytest.mark.parametrize(
        "optimizer",
        [
            STLSQ(),
            SSR(),
            SSR(criteria="model_residual"),
            FROLS(),
            SR3(),
            ConstrainedSR3(),
            StableLinearSR3(),
            TrappingSR3(),
            Lasso(fit_intercept=False),
            ElasticNet(fit_intercept=False),
            DummyLinearModel(),
            MIOSR(),
        ],
    )
    def test_fit(data_derivative_1d, optimizer):
        x, x_dot = data_derivative_1d
        if len(x.shape) == 1:
            x = x.reshape(-1, 1)
        opt = WrappedOptimizer(optimizer, unbias=False)
>       opt.fit(x, x_dot)

test/test_optimizers.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pysindy/optimizers/base.py:190: in fit
    self._reduce(x_normed, y, **reduce_kws)
pysindy/optimizers/wrapped_optimizer.py:38: in _reduce
    self.optimizer.fit(x, y)
env/lib/python3.10/site-packages/sklearn/multioutput.py:216: in fit
    self.estimators_ = Parallel(n_jobs=self.n_jobs)(
env/lib/python3.10/site-packages/sklearn/utils/parallel.py:63: in __call__
    return super().__call__(iterable_with_config)
env/lib/python3.10/site-packages/joblib/parallel.py:1085: in __call__
    if self.dispatch_one_batch(iterator):
env/lib/python3.10/site-packages/joblib/parallel.py:901: in dispatch_one_batch
    self._dispatch(tasks)
env/lib/python3.10/site-packages/joblib/parallel.py:819: in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
env/lib/python3.10/site-packages/joblib/_parallel_backends.py:208: in apply_async
    result = ImmediateResult(func)
env/lib/python3.10/site-packages/joblib/_parallel_backends.py:597: in __init__
    self.results = batch()
env/lib/python3.10/site-packages/joblib/parallel.py:288: in __call__
    return [func(*args, **kwargs)
env/lib/python3.10/site-packages/joblib/parallel.py:288: in <listcomp>
    return [func(*args, **kwargs)
env/lib/python3.10/site-packages/sklearn/utils/parallel.py:123: in __call__
    return self.function(*args, **kwargs)
env/lib/python3.10/site-packages/sklearn/multioutput.py:49: in _fit_estimator
    estimator.fit(X, y, **fit_params)
pysindy/optimizers/base.py:190: in fit
    self._reduce(x_normed, y, **reduce_kws)
pysindy/optimizers/miosr.py:248: in _reduce
    coefs = self._regress(x, y, self.target_sparsity, self.initial_guess)
pysindy/optimizers/miosr.py:228: in _regress
    m, coeff_var = self._make_model(X, y, k, warm_start)
pysindy/optimizers/miosr.py:168: in _make_model
    model.addSOS(gp.GRB.SOS_TYPE1, [coeff_var[i], iszero[i]])
src/gurobipy/model.pxi:4292: in gurobipy.Model.addSOS
    ???
src/gurobipy/mvar.pxi:291: in gurobipy.MVar.__getattr__
    ???
src/gurobipy/mvar.pxi:546: in gurobipy.MVar.getAttr
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   AttributeError: 'gurobipy.MVar' object has no attribute '__cindex__'

src/gurobipy/attrutil.pxi:23: AttributeError

Recreated bug described in #266, which only arises in gurobipy 10.0.0.
Verified tests pass in 10.0.1 to 10.0.3
@codecov
Copy link

codecov bot commented Oct 30, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (cddc1e7) 93.87% compared to head (3d1b035) 93.87%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #425   +/-   ##
=======================================
  Coverage   93.87%   93.87%           
=======================================
  Files          37       37           
  Lines        3626     3626           
=======================================
  Hits         3404     3404           
  Misses        222      222           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jacob-Stevens-Haas Jacob-Stevens-Haas merged commit 13af971 into master Oct 30, 2023
6 checks passed
@Jacob-Stevens-Haas Jacob-Stevens-Haas deleted the bump-gurobipy branch October 30, 2023 14:45
jpcurbelo pushed a commit to jpcurbelo/pysindy_fork that referenced this pull request Apr 30, 2024
Fixes dynamicslab#303 
Demonstrated that bug described in dynamicslab#266 only arises in gurobipy 10.0.0.
Verified tests pass in 10.0.1 to 10.0.3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Gurobipy requirement prevents Python 3.11 support
1 participant