Windows: add MSYS2 install scripts #355

MSP-Greg · 2020-02-03T15:02:22Z

Install script contains a ps1 script to install & update base MSYS2, then installs the following:

MSYS2 - base-devel compression

MinGW32 & MinGW64 - clang cmake gcc ragel

This installs all packages for what I believe to be common uses for MSYS2 and will allow minimal refresh times for most of the Actions workflows using it.

Packages/groups can always be added or subtracted at a later date.

Questions:

What to do in Validate-MSYS2.ps1? How many checks to do?

Add-SoftwareDetailsToMarkdown - What to include. Do we list the install date, the pacman version, compiler versions, etc, etc?

MSP-Greg · 2020-02-03T15:33:27Z

Script running in Actions:
https://github.com/MSP-Greg/msys2-action/runs/417780327?check_suite_focus=true#step:3:1

AndreyMaslennikov · 2020-02-04T10:33:53Z

images/win/scripts/Installers/Install-MSYS2.ps1

+$msy2_uri  = "http://repo.msys2.org/distrib/x86_64/msys2-base-x86_64-20190524.tar.xz"
+$msy2_sha1 = "cfe5035b1b81b43469d16bfc23be8006b9a44455".ToUpper()
+
+$msy2_file = "C:/msys2.tar.xz"


We have the environment variable AGENT_TEMPDIRECTORY in the image with the path to temp files/folders. Can we use it?

Done. Changed to:

$msy2_file = "$env:AGENT_TEMPDIRECTORY/msys2.tar.xz"

AndreyMaslennikov · 2020-02-04T10:37:26Z

images/win/scripts/Installers/Install-MSYS2.ps1

+$tar      = "$git_path\usr\bin\tar.exe"
+
+# extract tar.xz to C:\
+&$tar -Jxf $msy2_file_u -C /c/


And here it probably makes sense to store MSYS2 in the C:\tools folder. Thoughts?

AndreyMaslennikov · 2020-02-04T10:46:48Z

images/win/scripts/Installers/Install-MSYS2.ps1

+
+Write-Host Finished extraction
+
+$env:PATH = "C:\msys64\mingw64\bin;C:\msys64\usr\bin;$orig_path"


Am I right, that here you want to update PATH for the whole machine, and not for the current session?
If so, you can import the ImageHelpers module and use the Add-MachinePathItem method to update PATH.

@AndreyMaslennikov

Am I right, that here you want to update PATH for the whole machine, and not for the current session?

Sorry, wasn't clear. Just for the current session.

AndreyMaslennikov · 2020-02-04T10:49:41Z

images/win/scripts/Installers/Install-MSYS2.ps1

+$ErrorActionPreference = "Continue"
+
+Write-Host " bash.exe -c `"pacman-key --init`"".PadLeft($wid, $dash)
+bash.exe -c `"pacman-key --init`" 2> $null


There will be collisions with the bash from the WSL. We need to decide which one we will use by default.

This script temporarily changes PATH so things will run:

$env:PATH = "C:\msys64\mingw64\bin;C:\msys64\usr\bin;$orig_path"

As mentioned below, I'm not recommending any permanent system wide PATH changes, as there are already too many conflicts.

MSP-Greg · 2020-02-04T14:31:48Z

@AndreyMaslennikov

Path and location are somewhat linked, so replied here...

And here it probably makes sense to store MSYS2 in the C:\tools folder. Thoughts?

A standard MSYS2 install is done to C:/msys64, and I know there is code that looks in that location to adjust PATH.

Am I right, that here you want to update PATH for the whole machine, and not for the current session?

No, I adjusted PATH so pacman would run for the package listing (which could be done in Validate-MSYS2.ps1).

As to adding MSYS2 to PATH, I'd really prefer not to. Below is the result of running 'where' on a few files.

There's a lot of code that is tested on multiple platforms. Let's say something goes wrong with your CI, and the 'link' to your preferred OpenSSL dll breaks. Your code will load another copy, and it may be incompatible with the intended one. If you're a macOS or *nix type, that is not expected.

JFYI, some of the available OpenSSL dll's are built with msvc, some are built with MSYS2/MinGW. Also, the default location of the system cert file/dir is specified in the dll's, so that can also mix up CI...

———————————————————————————————————————————————————————————— libcrypto-1_1-x64.dll
C:\Windows\System32\libcrypto-1_1-x64.dll
C:\Program Files\Git\mingw64\bin\libcrypto-1_1-x64.dll
c:\tools\php\libcrypto-1_1-x64.dll
C:\Program Files\OpenSSL\bin\libcrypto-1_1-x64.dll
———————————————————————————————————————————————————————————— libssl-1_1-x64.dll
C:\Windows\System32\libssl-1_1-x64.dll
C:\Program Files\Git\mingw64\bin\libssl-1_1-x64.dll
c:\tools\php\libssl-1_1-x64.dll
C:\Program Files\OpenSSL\bin\libssl-1_1-x64.dll

———————————————————————————————————————————————————————————— make
C:\ProgramData\Chocolatey\bin\make.exe

———————————————————————————————————————————————————————————— gcc
C:\ProgramData\Chocolatey\bin\gcc.exe
C:\Strawberry\c\bin\gcc.exe

MSP-Greg · 2020-02-10T16:29:25Z

@al-cheb

Just checking in to see the status of MSYS2 & Windows images. This issue has been around a while, see Issue #30.

So, has a decision been made to include MSYS2?

If above is true, is there anything that all the people contributing code and discussing the issue can do to help speed the process along?

Do you/GH/MSFT have any specific questions?

Generalizing, MSYS2 is a) shell tools, b) compile tools, c) general packages. I think discussion seems to centered on a & b.

Based on discussions re MSYS2, I just updated the compile tools packages/groups to:
clang cmake llvm toolchain ragel

'toolchain' is a package group which installs all the gcc related tools. Note that what is included in this install script is a much smaller subset of what is currently available on AppVeyor.

miketimofeev · 2020-02-17T16:05:56Z

@MSP-Greg we're still investigating the best and safe way to add MSYS2 and get rid of other mingw installations(choco mingw installation, git mingw) and not to break anything

MSP-Greg · 2020-02-17T16:31:56Z

we're still investigating the best and safe way to add MSYS2.

I don't think there is. People using MSYS2 in CI have been thru it on other CI platforms, and should know what things need to be checked/changed, especially PATH issues. They know that multiple similar toolsets may be installed and they adjust their build scripts accordingly.

My suggestion is to:

Install it in the default location, C:\msys64
For now, make no changes to PATH, let people change their PATH as needed.

I work with Ruby, which uses MSYS2/MinGW/gcc for building the most popular Windows packages, and we're integrating the needed changes into our custom actions. We also run CI with msvc.

miketimofeev · 2020-02-19T16:02:25Z

@MSP-Greg Thanks for suggestions. And what do you think of getting rid of choco mingw installation?
https://github.com/actions/virtual-environments/blob/master/images/win/scripts/Installers/Install-MinGW.ps1

MSP-Greg · 2020-02-19T16:20:14Z

And what do you think of getting rid of choco mingw installation?

Everything I work with uses MSYS2 or VS/msvc, so I don't feel qualified to give an opinion on that.

Hate to duck and run...

MSP-Greg · 2020-02-29T18:12:03Z

@miketimofeev

Re the whole 'Path' issue on Window, it's messy.

I work with Ruby, and CI is running against Ubuntu, macOS, and Windows. Additionally, on Windows, two builds are available, one built with MSYS2/MinGW, and one built with MSVC.

We've got a custom multi-platform action for installing Rubies (https://github.com/ruby/setup-ruby), but now we're trying to create a cross-platform action to install build dependencies. Currently, build dependencies require either conditional step code or script code that many users may have problems making cross-platform.

Anyway, given the work I've done on Actions, I might suggest pruning down 'Path' on Windows.

As to what tools should be on the images (or whether to remove one), I can't answer.

Finally, how's MSYS2 moving along? GitHub Desktop for Windows uses it...

MSP-Greg · 2020-03-04T23:44:13Z

So MSYS2, the build tools used for GitHub Desktop for Windows, are now having service issues.

Maybe the 13 or 14 GB of packages they host could be moved to GitHub, considering you use the packages in your Windows desktop client? See https://packages.msys2.org/repos

That site is having issues, very possibly because whomever can't seem to move forward with an MSYS2 install here, and http://repo.msys2.org is getting clobbered with all the traffic updating the Actions MSYS2 install(s)...

maxim-lobanov · 2020-05-14T05:29:34Z

I am closing this PR because MSYS2 was added to the image in scope of #632
Please let me know if you have any concerns

MSP-Greg mentioned this pull request Feb 3, 2020

Install WSL and MSYS2 on Windows Images #342

Closed

AndreyMaslennikov reviewed Feb 4, 2020

View reviewed changes

MSP-Greg force-pushed the msys2 branch from 6488f29 to 8e91631 Compare February 4, 2020 14:06

al-cheb mentioned this pull request Feb 10, 2020

Add C++ support (GCC and Clang) on all platforms #104

Closed

Windows: add MSYS2 install scripts

6b0878c

MSP-Greg force-pushed the msys2 branch from 8e91631 to 6b0878c Compare February 10, 2020 16:25

vsafonkin mentioned this pull request Mar 20, 2020

Install MSYS2 to Windows. #585

Merged

maxim-lobanov closed this May 14, 2020

MSP-Greg deleted the msys2 branch May 14, 2020 13:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows: add MSYS2 install scripts #355

Windows: add MSYS2 install scripts #355

MSP-Greg commented Feb 3, 2020

MSP-Greg commented Feb 3, 2020

AndreyMaslennikov Feb 4, 2020

MSP-Greg Feb 4, 2020

AndreyMaslennikov Feb 4, 2020

MSP-Greg Feb 4, 2020

AndreyMaslennikov Feb 4, 2020

MSP-Greg Feb 4, 2020

MSP-Greg Feb 26, 2020

AndreyMaslennikov Feb 4, 2020

MSP-Greg Feb 4, 2020

MSP-Greg Feb 17, 2020

MSP-Greg commented Feb 4, 2020

MSP-Greg commented Feb 10, 2020

miketimofeev commented Feb 17, 2020

MSP-Greg commented Feb 17, 2020 •

edited

Loading

miketimofeev commented Feb 19, 2020

MSP-Greg commented Feb 19, 2020

MSP-Greg commented Feb 29, 2020

MSP-Greg commented Mar 4, 2020

maxim-lobanov commented May 14, 2020


		Write-Host Finished extraction

		$env:PATH = "C:\msys64\mingw64\bin;C:\msys64\usr\bin;$orig_path"

Windows: add MSYS2 install scripts #355

Windows: add MSYS2 install scripts #355

Conversation

MSP-Greg commented Feb 3, 2020

MSP-Greg commented Feb 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MSP-Greg commented Feb 4, 2020

MSP-Greg commented Feb 10, 2020

miketimofeev commented Feb 17, 2020

MSP-Greg commented Feb 17, 2020 • edited Loading

miketimofeev commented Feb 19, 2020

MSP-Greg commented Feb 19, 2020

MSP-Greg commented Feb 29, 2020

MSP-Greg commented Mar 4, 2020

maxim-lobanov commented May 14, 2020

MSP-Greg commented Feb 17, 2020 •

edited

Loading