macOS-12/20231029.1 fails to install CMAKE and arrow

[PBalsdon]

Description

We have a daily GHA that runs on macos-12.

As of Sunday, it started using the 20231029 image, and hasn't worked.

In particular the error is to do with CMAKE being used to install arrow.

            CMake Error at CMakeLists.txt:63 (find_package):
              By not providing "FindArrow.cmake" in CMAKE_MODULE_PATH this project has
              asked CMake to find a package configuration file provided by "Arrow", but
              CMake did not find one.
      
              Could not find a package configuration file provided by "Arrow" with any of
              the following names:
      
                ArrowConfig.cmake
                arrow-config.cmake
      
              Add the installation prefix of "Arrow" to CMAKE_PREFIX_PATH or set
              "Arrow_DIR" to a directory containing one of the above files.  If "Arrow"
              provides a separate development package or SDK, be sure it has been
              installed.
      
      
            -- Configuring incomplete, errors occurred!
            error: command '/usr/local/bin/cmake' failed with exit code 1
            [end of output]

We're running Python 3.11.1 and installing pyarrow==10.0.1

This all runs locally on a macos machine, but the error has started since the image update to 20231029.

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 20.04
• Ubuntu 22.04
• macOS 11
• macOS 12
• macOS 13
• Windows Server 2019
• Windows Server 2022

Image version and build link

Version: 20231029.1

Is it regression?

Version: 20230921.1

Expected behavior

pip to euccesfully install python packages.

notably, with python 3.11.1, to install pyarrow==10.0.1.

Actual behavior

pip exit with exit code 1
See overview for error log

Repro steps

Install run pip install pyarrow==10.0.01 on macos-12 runner.

[shamil-mubarakshin]

Hello @PBalsdon,
I couldn't reproduce your issue by provided repro steps https://github.com/shamil-mubarakshin/tests-repository/actions/runs/6771729689/job/18402755993.
Could you share more info about your workflow?

[PBalsdon]

Hello @shamil-mubarakshin , thanks for looking at this.

This is the content of the file I've been using for the actions - it fails on 20231029.1, but passes on 2023-0921.1

name: Github Workflow

on:
  pull_request:
    branches:
      - main

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.11.1]
        os: ['macos-12']
        try: ['0', '1', '2']

    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: setup venv
        run: python3 -m venv venv && venv/bin/pip install --upgrade pip
      
      - name: install pyarrow
        run: venv/bin/pip install pyarrow==10.0.1

Thanks for giving the idea of a matrix to test the different images - I was struggling to reproduce when it was giving me a random runner!

For context, we want this to run in a venv to keep the local dev experience close to the github actions. (All of our setup/run commands are actually makefile commands)

[shamil-mubarakshin]

@PBalsdon, the reason might be in incompatibility of pyarrow and python3.12 . I tried various version/OS with little success.
Here is also some info on enabling wheels.

For build consistency, it is recommended to use setup-python action in your workflow to select specific python version:

- uses: actions/setup-python@v4
  with:
    python-version: '3.11'

Or you can try going with venv/bin/pip install pyarrow==14.0.0, which seems to work with python 3.12.

[PBalsdon]

Hi @shamil-mubarakshin
Thanks so much for your help on this. This has closed the issue.

Yes, it looks like the underlying python version changing is what caused the issues - I was specifiying in the matrix but never actually setting it up!

Thanks so much for your help, I would have been debugging a lot longer without your help!