Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SPARK-1426: Make MLlib work with NumPy versions older than 1.7 #391

Closed
wants to merge 1 commit into from
Closed

SPARK-1426: Make MLlib work with NumPy versions older than 1.7 #391

wants to merge 1 commit into from

Conversation

techaddict
Copy link
Contributor

Currently it requires NumPy 1.7 due to using the copyto method (http://docs.scipy.org/doc/numpy/reference/generated/numpy.copyto.html) for extracting data out of an array.
Replace it with a fallback

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@mengxr
Copy link
Contributor

mengxr commented Apr 11, 2014

Jenkins, test this please.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14053/

@mengxr
Copy link
Contributor

mengxr commented Apr 11, 2014

@techaddict Is it the only place that requires numpy 1.7? If so, please also remove the check in /Users/meng/src/spark-mengxr/python/pyspark/mllib/__init__.py and run python/run-tests with numpy 1.6 to verify.

@techaddict
Copy link
Contributor Author

@mengxr yup, tests passed with 1.6
Updated the Docs too

@pwendell
Copy link
Contributor

Jenkins, test this please.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14070/

@mateiz
Copy link
Contributor

mateiz commented Apr 13, 2014

Is 1.6 the oldest version it works with now, or could it also work with 1.5 or older?

@techaddict
Copy link
Contributor Author

Tested, up until 1.4 all tests pass

@techaddict
SPARK-1426: Make MLlib work with NumPy versions older than 1.7
d365962 
Currently it requires NumPy 1.7 due to using the copyto method (http://docs.scipy.org/doc/numpy/reference/generated/numpy.copyto.html) for extracting data out of an array.
Replace it with a fallback
@mengxr
Copy link
Contributor

mengxr commented Apr 13, 2014

Just verified with numpy 1.6.2. All tests passed.

@techaddict
Copy link
Contributor Author

@mengxr yupp works with 1.4, 1.5 too 👍

@techaddict
Copy link
Contributor Author

@mateiz anymore changes ? or this is good ?

@mateiz
Copy link
Contributor

mateiz commented Apr 15, 2014

Alright, great! That's a good set of versions to support. I'm going to merge this then.

@mateiz
Copy link
Contributor

mateiz commented Apr 15, 2014

Jenkins, test this please

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14139/

@mateiz
Copy link
Contributor

mateiz commented Apr 15, 2014

Thanks Sandeep! I've merged this in.

@asfgit asfgit closed this in df36091 Apr 15, 2014
asfgit pushed a commit that referenced this pull request Apr 15, 2014
@techaddict @mateiz
SPARK-1426: Make MLlib work with NumPy versions older than 1.7
1491b2a 
Currently it requires NumPy 1.7 due to using the copyto method (http://docs.scipy.org/doc/numpy/reference/generated/numpy.copyto.html) for extracting data out of an array.
Replace it with a fallback

Author: Sandeep <sandeep@techaddict.me>

Closes #391 from techaddict/1426 and squashes the following commits:

d365962 [Sandeep] SPARK-1426: Make MLlib work with NumPy versions older than 1.7 Currently it requires NumPy 1.7 due to using the copyto method (http://docs.scipy.org/doc/numpy/reference/generated/numpy.copyto.html) for extracting data out of an array. Replace it with a fallback

(cherry picked from commit df36091)
Signed-off-by: Matei Zaharia <matei@databricks.com>
pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
@techaddict @pdeyhim
SPARK-1426: Make MLlib work with NumPy versions older than 1.7
76bdb3d 
Currently it requires NumPy 1.7 due to using the copyto method (http://docs.scipy.org/doc/numpy/reference/generated/numpy.copyto.html) for extracting data out of an array.
Replace it with a fallback

Author: Sandeep <sandeep@techaddict.me>

Closes apache#391 from techaddict/1426 and squashes the following commits:

d365962 [Sandeep] SPARK-1426: Make MLlib work with NumPy versions older than 1.7 Currently it requires NumPy 1.7 due to using the copyto method (http://docs.scipy.org/doc/numpy/reference/generated/numpy.copyto.html) for extracting data out of an array. Replace it with a fallback
mccheah pushed a commit to mccheah/spark that referenced this pull request Nov 28, 2018
@robert3005
Merge pull request apache#391 from palantir/os/merge-upstream
df34e2d
bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
@stmcginnis
Merge pull request apache#391 from theopenlab/revert-390-kubernetes/c…
9c8e11d 
…loud-provider-openstack#400

Revert "Do not run jobs for changes to documentation"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@techaddict @AmplabJenkins @mengxr @pwendell @mateiz