windows-2016 Azure Pipelines environment is deprecated
#4748
Comments
|
Thanks for noticing and documenting this.
Do you think that between now and March 15, 2022 we should try to move generation of these artifacts into Windows containers? So that future changes in these VM images will be less impactful? (linking relevant comment #3469 (comment)) |
|
That would be great! |
|
Example of Windows containers with LightGBM installation from PyPI: |
|
!!!! Awesome. Glad our friends over there already did so much of the hard work 😛 |
|
I just received an email from GitHub. To alert projects about this deprecation, they are going to perform two "brown outs" (temporary planned outages).
|
|
I think this notice explains why we just started observing some failures in the e.g. https://github.com/microsoft/LightGBM/runs/4384271108?check_suite_focus=true
It's currently 5:11p UTC, so for the next 5 hours this project's CI will be blocked by this 😬 |
|
The main problem I see with Windows containers is Visual Studio Build Tools. We need it to compile LightGBM. Yesterday I played with Windows Docker for GitHub Actions and learned some interesting things. Quotes for strings in env file are part of a value.
There should be In DOTNET methods current working directory doesn't match one set by PowerShell commandlet. There are a lot of restrictions on Host and client versions of Windows: GitHub Actions doesn't support containers out of the box on Windows and macOS runners. And there are no plans for adding support. There are no official images with installed Visual Studio:
I'd like to post my current progress here - maybe it would be helpful to someone. diff --git a/.ci/test_r_package_windows.ps1 b/.ci/test_r_package_windows.ps1
index 62562b47..1915d203 100644
--- a/.ci/test_r_package_windows.ps1
+++ b/.ci/test_r_package_windows.ps1
@@ -65,6 +65,7 @@ Remove-Item C:\rtools40 -Force -Recurse -ErrorAction Ignore
# NOTES:
# * some paths and file names are different on R4.0
$env:R_MAJOR_VERSION = $env:R_VERSION.split('.')[0]
+$env:PANDOC_VERSION = "2.16.2"
if ($env:R_MAJOR_VERSION -eq "3") {
# Rtools 3.x has to be installed at C:\Rtools\
# * https://stackoverflow.com/a/46619260/3986677
@@ -86,7 +87,7 @@ if ($env:R_MAJOR_VERSION -eq "3") {
$env:R_LIB_PATH = "$env:BUILD_SOURCESDIRECTORY/RLibrary" -replace '[\\]', '/'
$env:R_LIBS = "$env:R_LIB_PATH"
-$env:PATH = "$env:RTOOLS_BIN;" + "$env:RTOOLS_MINGW_BIN;" + "$env:R_LIB_PATH/R/bin/x64;" + "$env:R_LIB_PATH/miktex/texmfs/install/miktex/bin/x64;" + $env:PATH
+$env:PATH = "$env:RTOOLS_BIN;" + "$env:RTOOLS_MINGW_BIN;" + "$env:R_LIB_PATH/R/bin/x64;" + "$env:R_LIB_PATH/miktex/texmfs/install/miktex/bin/x64;" + "$env:BUILD_SOURCESDIRECTORY/pandoc/pandoc-$env:PANDOC_VERSION;" + $env:PATH
$env:CRAN_MIRROR = "https://cloud.r-project.org/"
$env:CTAN_MIRROR = "https://ctan.math.illinois.edu/systems/win32/miktex"
$env:CTAN_PACKAGE_ARCHIVE = "$env:CTAN_MIRROR/tm/packages/"
@@ -102,7 +103,6 @@ if (($env:COMPILER -eq "MINGW") -and ($env:R_BUILD_TYPE -eq "cmake")) {
$env:CXX = "$env:RTOOLS_MINGW_BIN/g++.exe"
$env:CC = "$env:RTOOLS_MINGW_BIN/gcc.exe"
}
-
cd $env:BUILD_SOURCESDIRECTORY
tzutil /s "GMT Standard Time"
[Void][System.IO.Directory]::CreateDirectory($env:R_LIB_PATH)
@@ -114,11 +114,11 @@ Download-File-With-Retries -url "https://github.com/microsoft/LightGBM/releases/
# Install R
Write-Output "Installing R"
-Start-Process -FilePath R-win.exe -NoNewWindow -Wait -ArgumentList "/VERYSILENT /DIR=$env:R_LIB_PATH/R /COMPONENTS=main,x64,i386" ; Check-Output $?
+Start-Process -FilePath ./R-win.exe -NoNewWindow -Wait -ArgumentList "/VERYSILENT /DIR=$env:R_LIB_PATH/R /COMPONENTS=main,x64,i386" ; Check-Output $?
Write-Output "Done installing R"
Write-Output "Installing Rtools"
-Start-Process -FilePath Rtools.exe -NoNewWindow -Wait -ArgumentList "/VERYSILENT /SUPPRESSMSGBOXES /DIR=$RTOOLS_INSTALL_PATH" ; Check-Output $?
+Start-Process -FilePath ./Rtools.exe -NoNewWindow -Wait -ArgumentList "/VERYSILENT /SUPPRESSMSGBOXES /DIR=$RTOOLS_INSTALL_PATH" ; Check-Output $?
Write-Output "Done installing Rtools"
Write-Output "Installing dependencies"
@@ -132,7 +132,7 @@ Run-R-Code-Redirect-Stderr "options(install.packages.check.source = 'no'); insta
if (($env:COMPILER -eq "MINGW") -or ($env:R_BUILD_TYPE -eq "cran")) {
Download-File-With-Retries "https://github.com/microsoft/LightGBM/releases/download/v2.0.12/miktexsetup-4.0-x64.zip" -destfile "miktexsetup-x64.zip"
Add-Type -AssemblyName System.IO.Compression.FileSystem
- [System.IO.Compression.ZipFile]::ExtractToDirectory("miktexsetup-x64.zip", "miktex")
+ [System.IO.Compression.ZipFile]::ExtractToDirectory("$env:BUILD_SOURCESDIRECTORY/miktexsetup-x64.zip", "$env:BUILD_SOURCESDIRECTORY/miktex")
Write-Output "Setting up MiKTeX"
.\miktex\miktexsetup.exe --remote-package-repository="$env:CTAN_PACKAGE_ARCHIVE" --local-package-repository=./miktex/download --package-set=essential --quiet download ; Check-Output $?
Write-Output "Installing MiKTeX"
@@ -140,6 +140,9 @@ if (($env:COMPILER -eq "MINGW") -or ($env:R_BUILD_TYPE -eq "cran")) {
Write-Output "Done installing MiKTeX"
Run-R-Code-Redirect-Stderr "result <- processx::run(command = 'initexmf', args = c('--set-config-value', '[MPM]AutoInstall=1'), echo = TRUE, windows_verbatim_args = TRUE, error_on_status = TRUE)" ; Check-Output $?
+
+ Download-File-With-Retries -url "https://github.com/jgm/pandoc/releases/download/$env:PANDOC_VERSION/pandoc-$env:PANDOC_VERSION-windows-x86_64.zip" -destfile "pandoc-windows-x86_64.zip"
+ [System.IO.Compression.ZipFile]::ExtractToDirectory("$env:BUILD_SOURCESDIRECTORY/pandoc-windows-x86_64.zip", "$env:BUILD_SOURCESDIRECTORY/pandoc")
}
Write-Output "Building R package"
diff --git a/.github/workflows/r_package.yml b/.github/workflows/r_package.yml
index b2e16261..20ae9c6e 100644
--- a/.github/workflows/r_package.yml
+++ b/.github/workflows/r_package.yml
@@ -3,7 +3,7 @@ name: R-package
on:
push:
branches:
- - master
+ - win_docker
pull_request:
branches:
- master
@@ -32,52 +32,6 @@ jobs:
################
# CMake builds #
################
- - os: ubuntu-latest
- task: r-package
- compiler: gcc
- r_version: 3.6
- build_type: cmake
- - os: ubuntu-latest
- task: r-package
- compiler: gcc
- r_version: 4.0
- build_type: cmake
- - os: ubuntu-latest
- task: r-package
- compiler: clang
- r_version: 3.6
- build_type: cmake
- - os: ubuntu-latest
- task: r-package
- compiler: clang
- r_version: 4.0
- build_type: cmake
- - os: macOS-latest
- task: r-package
- compiler: gcc
- r_version: 3.6
- build_type: cmake
- - os: macOS-latest
- task: r-package
- compiler: gcc
- r_version: 4.0
- build_type: cmake
- - os: macOS-latest
- task: r-package
- compiler: clang
- r_version: 3.6
- build_type: cmake
- - os: macOS-latest
- task: r-package
- compiler: clang
- r_version: 4.0
- build_type: cmake
- - os: windows-latest
- task: r-package
- compiler: MINGW
- toolchain: MINGW
- r_version: 3.6
- build_type: cmake
- os: windows-latest
task: r-package
compiler: MINGW
@@ -85,26 +39,12 @@ jobs:
r_version: 4.0
build_type: cmake
# Visual Studio 2017
- - os: windows-2016
+ - os: windows-latest
task: r-package
compiler: MSVC
toolchain: MSVC
r_version: 3.6
build_type: cmake
- # Visual Studio 2019
- - os: windows-2019
- task: r-package
- compiler: MSVC
- toolchain: MSVC
- r_version: 4.0
- build_type: cmake
- # Visual Studio 2022
- - os: windows-2022
- task: r-package
- compiler: MSVC
- toolchain: MSVC
- r_version: 4.0
- build_type: cmake
###############
# CRAN builds #
###############
@@ -114,30 +54,6 @@ jobs:
toolchain: MINGW
r_version: 3.6
build_type: cran
- - os: windows-latest
- task: r-package
- compiler: MINGW
- toolchain: MSYS
- r_version: 4.0
- build_type: cran
- - os: ubuntu-latest
- task: r-package
- compiler: gcc
- r_version: 4.0
- build_type: cran
- - os: macOS-latest
- task: r-package
- compiler: clang
- r_version: 4.0
- build_type: cran
- ################
- # Other checks #
- ################
- - os: ubuntu-latest
- task: r-rchk
- compiler: gcc
- r_version: 4.0
- build_type: cran
steps:
- name: Prevent conversion of line endings on Windows
if: startsWith(matrix.os, 'windows')
@@ -150,99 +66,28 @@ jobs:
submodules: true
- name: Install pandoc
uses: r-lib/actions/setup-pandoc@v1
- - name: Setup and run tests on Linux and macOS
- if: matrix.os == 'macOS-latest' || matrix.os == 'ubuntu-latest'
- shell: bash
- run: |
- export TASK="${{ matrix.task }}"
- export COMPILER="${{ matrix.compiler }}"
- export GITHUB_ACTIONS="true"
- if [[ "${{ matrix.os }}" == "macOS-latest" ]]; then
- export OS_NAME="macos"
- elif [[ "${{ matrix.os }}" == "ubuntu-latest" ]]; then
- export OS_NAME="linux"
- fi
- export BUILD_DIRECTORY="$GITHUB_WORKSPACE"
- export R_VERSION="${{ matrix.r_version }}"
- export R_BUILD_TYPE="${{ matrix.build_type }}"
- $GITHUB_WORKSPACE/.ci/setup.sh
- $GITHUB_WORKSPACE/.ci/test.sh
- name: Setup and run tests on Windows
- if: startsWith(matrix.os, 'windows')
- shell: pwsh -command ". {0}"
run: |
- $env:BUILD_SOURCESDIRECTORY = $env:GITHUB_WORKSPACE
- $env:TOOLCHAIN = "${{ matrix.toolchain }}"
- $env:R_VERSION = "${{ matrix.r_version }}"
- $env:R_BUILD_TYPE = "${{ matrix.build_type }}"
- $env:COMPILER = "${{ matrix.compiler }}"
- $env:GITHUB_ACTIONS = "true"
- $env:TASK = "${{ matrix.task }}"
- & "$env:GITHUB_WORKSPACE/.ci/test_windows.ps1"
- test-r-sanitizers:
- name: r-sanitizers (ubuntu-latest, R-devel, ${{ matrix.compiler }} ASAN/UBSAN)
- timeout-minutes: 60
- runs-on: ubuntu-latest
- container: wch1/r-debug
- strategy:
- fail-fast: false
- matrix:
- include:
- - r_customization: san
- compiler: gcc
- - r_customization: csan
- compiler: clang
- steps:
- - name: Checkout repository
- uses: actions/checkout@v2.4.0
- with:
- fetch-depth: 5
- submodules: true
- - name: Install packages
- shell: bash
- run: |
- RDscript${{ matrix.r_customization }} -e "install.packages(c('R6', 'data.table', 'jsonlite', 'knitr', 'Matrix', 'rmarkdown', 'testthat'), repos = 'https://cran.r-project.org', Ncpus = parallel::detectCores())"
- sh build-cran-package.sh --r-executable=RD${{ matrix.r_customization }}
- RD${{ matrix.r_customization }} CMD INSTALL lightgbm_*.tar.gz || exit -1
- - name: Run tests with sanitizers
- shell: bash
- run: |
- cd R-package/tests
- exit_code=0
- RDscript${{ matrix.r_customization }} testthat.R >> tests.log 2>&1 || exit_code=-1
- cat ./tests.log
- exit ${exit_code}
- test-r-debian-clang:
- name: r-package (debian, R-devel, clang)
- timeout-minutes: 60
- runs-on: ubuntu-latest
- container: rhub/debian-clang-devel
- steps:
- - name: Install Git before checkout
- shell: bash
- run: |
- apt-get update --allow-releaseinfo-change
- apt-get install --no-install-recommends -y git
- - name: Checkout repository
- uses: actions/checkout@v2.4.0
- with:
- fetch-depth: 5
- submodules: true
- - name: Install packages and run tests
- shell: bash
- run: |
- export PATH=/opt/R-devel/bin/:${PATH}
- Rscript -e "install.packages(c('R6', 'data.table', 'jsonlite', 'knitr', 'Matrix', 'rmarkdown', 'testthat'), repos = 'https://cran.r-project.org', Ncpus = parallel::detectCores())"
- sh build-cran-package.sh
- R CMD check --as-cran --run-donttest lightgbm_*.tar.gz || exit -1
- if grep -q -E "NOTE|WARNING|ERROR" lightgbm.Rcheck/00check.log; then
- echo "NOTEs, WARNINGs, or ERRORs have been found by R CMD check"
- exit -1
- fi
- all-successful:
- # https://github.community/t/is-it-possible-to-require-all-github-actions-tasks-to-pass-without-enumerating-them/117957/4?u=graingert
- runs-on: ubuntu-latest
- needs: [test, test-r-sanitizers, test-r-debian-clang]
- steps:
- - name: Note that all tests succeeded
- run: echo "🎉"
+ $ROOT_DOCKER_FOLDER = "C:/LightGBM"
+ @"
+ BUILD_SOURCESDIRECTORY=$ROOT_DOCKER_FOLDER
+ TOOLCHAIN=${{ matrix.toolchain }}
+ R_VERSION=${{ matrix.r_version }}
+ R_BUILD_TYPE=${{ matrix.build_type }}
+ COMPILER=${{ matrix.compiler }}
+ GITHUB_ACTIONS=true
+ TASK=${{ matrix.task }}
+ _R_CHECK_SYSTEM_CLOCK_=0
+ _R_CHECK_CRAN_INCOMING_REMOTE_=0
+ _R_CHECK_PKG_SIZES_THRESHOLD_=100
+ "@ | Out-File -FilePath docker.env
+ docker run `
+ --rm `
+ --env-file docker.env `
+ -v "${{ github.workspace }}:$ROOT_DOCKER_FOLDER" `
+ "mcr.microsoft.com/windows/servercore:ltsc2019-amd64" `
+ powershell `
+ -NoProfile `
+ -NonInteractive `
+ -ExecutionPolicy Bypass `
+ -File "$ROOT_DOCKER_FOLDER/.ci/test_windows.ps1"Visual Studio Build Tools installation time and resulting image size is quite big. I guess our only option is to prebuild Docker images we need (for R, for Python, etc.) and publish them, let say, on GitHub Packages.
For example, I can find some images with VS Tools at Docker Hub: |
|
@StrikerRUS thanks for this great investigation! @shiyu1994 is there any opportunity for someone from Microsoft to help us with this issue of I also want to note that @StrikerRUS if you or anyone else comes back to trying to make Windows containers work for this, since it seems like this might be a very large effort, I think Python jobs should probably be prioritized over the R ones. Unlike with the wheels published to PyPI, for the R package we don't publish any precompiled artifacts for R. |
Absolutely agree! I picked up R jobs just because we already install all necessary R stuff on the fly in our scripts (particularly, for CRAN ones). In Python jobs, we rely, for example, on pre-installed conda and CMake. |
|
Just a reminder: one month left. |
Sure. I'll search to see if there's anyone can help within the company tomorrow. Thanks for you reminder! |
Is there any progress with it? I guess we have to simply bump |
|
@StrikerRUS Thanks for fixing this! Great work! |
|
There won't be official support for VS 2017 from GitHub team:
|
|
😬 that puts LightGBM into a difficult situation, given that Microsoft has committed to supporting VS At https://docs.microsoft.com/en-us/visualstudio/releases/2017/vs2017-servicingarchive-vs I see the following
See https://docs.microsoft.com/en-us/lifecycle/policies/fixed#lifecycle-phases-for-products-under-the-fixed-lifecycle-policy for "the Microsoft Product Lifecycle Policy". I guess we'll have to rely on AppVeyor to get some coverage of VS https://www.appveyor.com/docs/windows-images-software/#visual-studio-2017 |
Sorry, I'm confused here. VS Code is a code editor developed by Microsoft. How is it related to the VS Studio? Right now we are using even VS 2015 at Appveyor: Line 3 in b462d0a So, I guess everything is OK and no action should be taken. |
|
oh sorry, I meant "VS" not "VS Code". You're absolutely right. |
|
Microsoft confused me by giving those two things such similar names. I make this mistake often 😂 |
|
This issue has been automatically locked since there has not been any recent activity since it was closed. |
We use this environment here
LightGBM/.vsts-ci.yml
Line 260 in 96ecab6
The main problem is that that environment is used right now to create release artifacts: LightGBM executable, dll, SWIG jar and
win_amd64Python wheel file.LightGBM/.vsts-ci.yml
Lines 287 to 288 in 96ecab6
So we should be careful during migration to not accidentally change LightGBM installation requirements (refer to #1708, #2889 as an examples of problems we may face).
The text was updated successfully, but these errors were encountered: