Error installing scikit-learn on top of scipy+numpy

[gusy]

For a requirements.txt file just like

numpy==1.8.1
scipy==0.14.0
scikit-learn==0.14.1

Current version gives me

ImportError: liblapack.so.3gf:

I had to rollback to commit 1a9ab58 to make it work. I believe the change from export to set-env might be behind the error (so libraries dont get added to LDPATH)

[branliu0]

Could you clarify when you get this ImportError? Is this upon installation or at runtime? If you could paste the relevant section of the logs, that would be helpful.

Your analysis might be correct, as set-env adds the export command to $PROFILE_PATH, but does not set the environment variable in the current shell. If you're installing scikit-learn from pip, then you would need the env variables to be set in the immediate shell. Could you try amending the set-env command to be:

set-env() {
  export "$1=$2"
  echo "export $1=$2" >> $PROFILE_PATH
}

(https://github.com/thenovices/heroku-buildpack-scipy/blob/master/bin/utils#L31)

and see if that works on the master branch?

[gusy]

Now it goes further, finishes compiling everything (kudos), but then I get

/tmp/buildpack_dabc63fb8470fef2cde88e2a413a2ed9/bin/compile: line 193: bpwatch: command not found

Maybe overwriting PATH so bpwatch cannot be found?

[branliu0]

Ah, I found the problem. Replacing that first line of set-env with:

export "$1=$(eval echo $2)

That should fix this problem.

[gusy]

Hey there!

Thanks for the fix, additionally I had to move the set-env lines after compilation (lines 182-189 in bin/compile) to the end of the file, because otherwise the bpwatch could not exec (could not find basic modules as os)

I will send you the pull request

[branliu0]

I will look into this when I have a chance... I would prefer a solution that works without having to move the set-env lines. Simply moving those lines just fixes the symptom, without resolving the underlying issue.

[branliu0]

I merged it in. HOWEVER: Please note that sklearn (0.14.1) did not pass all tests for me, when run with numpy 1.9.0 and scipy 0.13.3.

Use with your own risk...

======================================================================
ERROR: sklearn.cluster.bicluster.tests.test_utils.test_get_submatrix
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/app/.heroku/python/lib/python2.7/site-packages/sklearn/cluster/bicluster/tests/test_utils.py", line 43, in test_get_submatrix
    assert_true(np.all(X != -1))
  File "/app/.heroku/python/lib/python2.7/unittest/case.py", line 420, in assertTrue
    if not expr:
  File "/app/.heroku/python/lib/python2.7/site-packages/scipy/sparse/base.py", line 183, in __bool__
    raise ValueError("The truth value of an array with more than one "
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().

======================================================================
ERROR: sklearn.tests.test_common.test_regressors_train
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/app/.heroku/python/lib/python2.7/site-packages/sklearn/tests/test_common.py", line 798, in test_regressors_train
    assert_raises(ValueError, regressor.fit, X, y[:-1])
  File "/app/.heroku/python/lib/python2.7/unittest/case.py", line 473, in assertRaises
    callableObj(*args, **kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/sklearn/linear_model/least_angle.py", line 937, in fit
    for train, test in cv)
  File "/app/.heroku/python/lib/python2.7/site-packages/sklearn/externals/joblib/parallel.py", line 516, in __call__
    for function, args, kwargs in iterable:
  File "/app/.heroku/python/lib/python2.7/site-packages/sklearn/linear_model/least_angle.py", line 937, in <genexpr>
    for train, test in cv)
IndexError: index 199 is out of bounds for axis 1 with size 199

======================================================================
FAIL: sklearn.utils.tests.test_extmath.test_random_weights
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/app/.heroku/python/lib/python2.7/site-packages/sklearn/utils/tests/test_extmath.py", line 68, in test_random_weights
    assert_true(np.all(score.ravel() == w[:, :5].sum(1)))
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1715 tests in 104.590s

FAILED (SKIP=15, errors=2, failures=1)