Skip to content

Commit

Permalink
[SPARK-47481][INFRA][3.4] Pin matplotlib<3.3.0 to fix Python linter…
Browse files Browse the repository at this point in the history
… failure

### What changes were proposed in this pull request?
The pr aims to fix `python linter issue` on branch-3.4 through pinning `matplotlib<3.3.0`

### Why are the changes needed?
- Through this PR apache#45600, we found that the version of `matplotlib` in our Docker image was `3.8.2`, which clearly did not meet the original requirements for `branch-3.4`.
  https://github.com/panbingkun/spark/actions/runs/8354370179/job/22869580038
  <img width="1072" alt="image" src="https://github.com/apache/spark/assets/15246973/dd425bfb-ce5f-4a99-a487-a462d6ebbbb9">
  https://github.com/apache/spark/blob/branch-3.4/dev/requirements.txt#L12
  <img width="973" alt="image" src="https://github.com/apache/spark/assets/15246973/70485648-b886-4218-bb21-c41a85d5eecf">

- Fix as follows:
<img width="989" alt="image" src="https://github.com/apache/spark/assets/15246973/db31d8fb-0b6c-4925-95e1-0ca0247bb9f5">

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Pass GA.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#45608 from panbingkun/branch_3.4_pin_matplotlib.

Authored-by: panbingkun <panbingkun@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
panbingkun authored and kazuyukitanimura committed Mar 20, 2024
1 parent 1daf5e2 commit 2933efc
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions dev/infra/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ RUN add-apt-repository ppa:pypy/ppa
RUN apt update
RUN $APT_INSTALL gfortran libopenblas-dev liblapack-dev
RUN $APT_INSTALL build-essential
RUN $APT_INSTALL python3-matplotlib

RUN mkdir -p /usr/local/pypy/pypy3.7 && \
curl -sqL https://downloads.python.org/pypy/pypy3.7-v7.3.7-linux64.tar.bz2 | tar xjf - -C /usr/local/pypy/pypy3.7 --strip-components=1 && \
Expand Down Expand Up @@ -64,8 +65,8 @@ RUN Rscript -e "devtools::install_version('roxygen2', version='7.2.0', repos='ht
# See more in SPARK-39735
ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"

RUN pypy3 -m pip install numpy 'pandas<=1.5.3' scipy coverage matplotlib
RUN python3.9 -m pip install 'numpy==1.23.5' 'pyarrow==12.0.1' 'pandas<=1.5.3' scipy unittest-xml-reporting plotly>=4.8 scikit-learn 'mlflow>=1.0' coverage matplotlib openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*'
RUN pypy3 -m pip install numpy 'pandas<=1.5.3' scipy coverage 'matplotlib<3.3.0'
RUN python3.9 -m pip install 'numpy==1.23.5' 'pyarrow==12.0.1' 'pandas<=1.5.3' scipy unittest-xml-reporting plotly>=4.8 scikit-learn 'mlflow>=1.0' coverage 'matplotlib<3.3.0' openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*'

# Add Python deps for Spark Connect.
RUN python3.9 -m pip install grpcio protobuf googleapis-common-protos grpcio-status
Expand Down

0 comments on commit 2933efc

Please sign in to comment.