Create a script to execute msys2 and maintain PATH and PKG_CONFIG_PATH #104
Comments
|
@eine I tried a small modification in setup-msys2: async function writeWrapper(msysRootDir, pathtype, destDir, name) {
let wrap = [
`@echo off`,
`setlocal`,
`echo OMG`,
`set`,
`echo %GITHUB_SCRIPT%`,
`type %GITHUB_SCRIPT%`,
`echo done`,Here's what it looks like in my action when it runs:
That extra line at the top and the bottom of my script are not from my CI workflow. I don't understand how the bash interpreter is even able to deal with those! I can work around it but I would like to understand why it's there. |
|
I think that PATH is one of the few variables which you don't want to add using a custom procedure. There is already an option which can be With regard to the extra lines shown in the script, I cannot tell. I would recommend asking about it in https://github.com/actions/virtual-environments or https://github.com/actions/runner/. Maybe the runner removes those first and last lines before executing the script. |
|
I don't know if minimal or inherit will do the right thing. The way to modify the path is to write lines to the $GITHUB_PATH file. And then github will add them to the path of future steps. But it adds it to the windows path, not the path in msys2. So if you inherit, you'll get all those but also all the windows paths, which you don't need. But if you don't inherit, you won't get any of them. Also, it's possible to set the PATH by writing to $GITHUB_ENV. And that, too, will get set outside the msys but not inside. It's not clear to me how all those will interact with the inherit option. Does inherit convert paths from windows to unix format even? My solution is kind of a mess that combines some windows scripting with some unix scripting but it works. I don't know if it would be possible to made a solution that would only be a modification to msys2.cmd because I still need a script inside the msys2 unix. |
|
Looks like it's described here: https://github.com/actions/runner/blob/feafd3e1d79a627b18e63705d1021165d978ba46/docs/adrs/0277-run-action-shell-options.md#L82 But I'm using a custom command and getting the text added anyway. Weird. |
|
I think you are correct about the limited additional paths added to GITHUB_PATH or the envvars added to GITHUB_ENV. Moreover, for consistency, you might want writting to /cc @al-cheb @maxim-lobanov @alepauly
I do think so.
That might be something to be reported. I tried printing |
|
I found the cause of those extra lines and filed a bug. I think that it would be fixed pretty easily. |
|
Here's the fix, we'll see if GitHub likes it: actions/runner#923 |
|
So, how do you add stuff to msys2s PATH? I tried adding stuff to |
|
@hasufell Like this: |
That's what I did before using this action. Using this with does not work. |
|
If you want to write a CI script that works for both unix and window then you need to do this: Your I'm able to write a single script that is unix-based and have it work both with unix, mac, and windows. |
|
If you only need Windows and not Unix then you can search for a different solution but you will still need to find a way to convert paths because the GITHUB_ENV is processed in windows and makes windows paths, like |
|
what I'm saying is that |
|
Your suggestions didn't work either: https://github.com/vmchale/libarchive/runs/3475297119?check_suite_focus=true I've probably spent more than an hour on this action and it doesn't work well. |
|
Looks like your doing some sort of combination of windows and unix commands. https://github.com/vmchale/libarchive/blob/master/.github/workflows/haskell.yml#L25 Mixing powershell and msys2 together is going to cause a lot of headaches. I did all my work inside msys2, so it's entirely unix, no windows. If it makes you feel better, I spent probably 10-20 hours to figure it out. It's difficult. If you have CI that works from unix, you might be better off starting over with the shell as I use it and write unix-only commands, like I did. Your CI is just 163 lines so it might not be too much work. Mine is 471 lines and works well on all platforms. These lines here: It should just say this instead: No check for windows vs non-windows is necessary because everything needs to happen inside of msys2. There is no such thing as If you simplify your script to work for ubuntu only and then use the shell as I suggested then it'll probably work and your CI might be down to around 100 lines instead. You can look at my CI to see examples of how I'm able to install code from sources, from package repos, run tests, etc. |
No, some of the stuff needs to run in powershell. Those are different scripts that do different things. There's no duplication in the current workflow file, it already uses the same logic for most steps. defaults:
run:
shell: msys2 {0}Doesn't work, because PATH is ignored, you can see it here: https://github.com/vmchale/libarchive/runs/3475991363?check_suite_focus=true Then you suggested to use this abomination: shell: |
powershell -command "[System.Environment]::SetEnvironmentVariable('GITHUB_SCRIPT', ('{0}' -replace '\\','\\')); [System.Environment]::SetEnvironmentVariable('PKG_CONFIG_PATH_MSYS', $Env:PKG_CONFIG_PATH); [System.Environment]::SetEnvironmentVariable('PATH_MSYS', $Env:Path);"
msys2 -c 'if [[ -v MSYSTEM ]]; then sed -i 1d $(cygpath $GITHUB_SCRIPT); sed -i \$d $(cygpath $GITHUB_SCRIPT); if [[ -v PKG_CONFIG_PATH_MSYS ]]; then export PKG_CONFIG_PATH=$PKG_CONFIG_PATH_MSYS; fi; if [[ -v PATH_MSYS && ! ( $PATH_MSYS =~ ^D:\\\\a\\\\_temp\\\\msys\\;C:\\\\Users ) ]]; then export PATH=$(echo $PATH_MSYS | sed s/D:\\\\\\\\a\\\\\\\\_temp\\\\\\\\msys\;// | sed s/\;/:/g); fi; fi; source $(cygpath $GITHUB_SCRIPT)'
That also doesn't work, PATH is still ignored, see here: https://github.com/vmchale/libarchive/runs/3476772657?check_suite_focus=true#step:8:10 |
|
Maybe you want https://github.com/msys2/setup-msys2#path-type ? |
https://github.com/vmchale/libarchive/runs/3476859370?check_suite_focus=true#step:8:9 This is all broken. |
|
That's interference from the "abomination" - try taking that out and just using shell: msys {0} Also, msys wrapper launches a login shell, so I'd expect |
Okay, then my technique can't help you. You'll have to do it the hard way. Why do you insist that some stuff needs to be in powershell? I've got a CI that is a lot more complex than yours, with no powershell. What is the thing that needs to be done in powershell that can't be done in bash? |
Doesn't workWorksBut pollutes the PATH with the windows ones: So it seems it's impossible to simply add something to PATH with this action without These are all with |
As explained above, it's really orthogonal. |
|
For curiosity, try 3 with |
|
@hasufell, as @jeremyd2019 suggested, you should use bash_profile. See: name: MSYS2-Path
on: [push]
jobs:
msys2:
runs-on: windows-latest
defaults:
run:
shell: msys2 {0}
steps:
- uses: msys2/setup-msys2@v2
with:
update: true
- run: MSYS2_PATH_TYPE=inherit echo "export PATH=\$PATH:/c/ghcup/env" >> ~/.bash_profile
- run: echo $PATHNonetheless, this issue is unrelated to the Action. That's how MSYS2 works. |
Tried that, didn't work: |
|
According to https://github.com/vmchale/libarchive/runs/3477401792?check_suite_focus=true#step:5:20, the location might be |
|
|
I recommend that you reduce the complexity of your testing to a MWE, so that you can properly understand what's going on. Then, slowly add features to it. The env file is sourced after adding content to bash_profile (https://github.com/vmchale/libarchive/actions/runs/1187661602/workflow#L86-L87), maybe that's the problem. Anyway, the solution explained above does work: https://github.com/eine/actions/runs/3477651227?check_suite_focus=true |
No. The test -f ~/.bashrc && . ~/.bashrc
This seems minimal enough. |
|
My CI for pcb2gcode is, frankly, far more involved than what you're trying to do. Your CI is pretty trivial. If you started with the pcb2gcode CI file, deleted away all the parts that you didn't need, and added what you did, I think that it would work. I worked on the problem of having a single CI that does both windows and unix and I worked on it for a long time. I also tried the approach that you're attempting. It led me to the abomination. Perhaps if you put in many more hours you will eventually come up with a working solution that will be the similar to what I already completed. If you have need for native windows commands then my technique won't help you but in that case, maybe msys2 isn't a good fit in the first place. I'm interested to see if your hard work ends up with something much different then where mine went. Best of luck with it! |
My CI already works exactly as I want, except for the PATH that is vigorously ignored by msys2 or this action. Anywhere else I try it, it works.
I've already done that and it was trivial: https://github.com/vmchale/libarchive/pull/29/files |
Yes, indeed, that is the tricky part. That's what the "abomination" fixes. The rest of it is easy. I wanted something that works consistently for both Unix and Windows and uses the GITHUB_ENV so I wrote commands that will copy the original windows path into a temporary variable. Then, when msys starts, it uses All those steps are done in my script like this: Another thing that you might want to do it also be careful about using the Anyway, I'm excited to see if you can come up with a solution that works and is nicer than mine. |
@hasufell, use some file other than name: MSYS2-Path
on: [push]
jobs:
msys2:
runs-on: windows-latest
defaults:
run:
shell: msys2 {0}
steps:
- uses: msys2/setup-msys2@v2
with:
update: true
- run: |
mkdir -p /c/ghcup/bin
echo '#!/usr/bin/env bash' > /c/ghcup/bin/mytool
echo 'echo HELLO' >> /c/ghcup/bin/mytool
echo 'export PATH="$HOME/.cabal/bin:/c/ghcup/bin:$PATH"' >> /c/ghcup/env
echo '. /c/ghcup/env' >> ~/.myenv
echo 'test -f ~/.myenv && . ~/.myenv' >> ~/.bash_profile
- run: echo $PATH
- run: which mytool || echo "NOT FOUND!"
- run: mytoolhttps://github.com/eine/actions/runs/3477994273?check_suite_focus=true |
Could you elaborate why? |
|
Probably because .bashrc exits/returns early if certain conditions are not met; hence, the commands you put at the end are never executed. You either need to understand the content of .bashrc because you add there, or you can use a clean file so you are not affected by previous code. |
|
Excellent... this should be added to the documentation. |
|
Feel free to propose a PR in https://github.com/msys2/msys2.github.io for enhancing https://www.msys2.org/docs/ci/ (https://github.com/msys2/msys2.github.io/edit/source/web/docs/ci.md). |
|
Does that work for both windows and ubuntu? It's convenient to have a solution that works for both... |
|
The specific example above should work both on Windows and on Ubuntu. However, in the projects I co-maintain we rarely use the same job for both. Typically, we have different matrices for them (see https://github.com/ghdl/ghdl/actions/runs/1194048862). Therefore, I cannot confirm. |
|
See the continuation of this issue in #171. |
@eyal0 you might want to extend this Action with those two lines, in order to make them easier to maintain and reuse. I would suggest the following:
npm run install. Either get NodeJs, use anodecontainer, or use WSL.npm run pkg.index.js, commit and push.eyal0/setup-msys2@feature-branch-nameand change theshell: yourcustomentrypoint {0}. Commit and push.Originally posted by @eine in #98 (comment)
The text was updated successfully, but these errors were encountered: