Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] enable usage on pkg-config on MSYS2 (Windows) #3653

Closed
franko opened this issue Jun 1, 2018 · 14 comments
Closed

[feature request] enable usage on pkg-config on MSYS2 (Windows) #3653

franko opened this issue Jun 1, 2018 · 14 comments

Comments

@franko
Copy link

franko commented Jun 1, 2018

Hi all,

I stumbled on a problem using meson on q MSYS2 system. On this system one can use unix-like path like /usr/lib on the MSYS2 bash shell.

If you install pkg-config using pacman, the standard packaging system, you gen a functional pkg-config. The problem arise when you invoke Meson and what happens is that Meson is not able to recognize and use the MSYS2-provided pkg-config.

By doing some investigations the problem seems to be the path:

> which pkg-config
/usr/bin/pkg-config

Otherwise the pkg-config is a native windows executable that can be ran without any problem.

I did some investigations and I think that Meson is confused by the fact that pkg-config is located in /usr/bin and this is not a Windows-like PATH.

So the paradox is that you have a working pkg-config from your shell, it is a native windows executable, but Meson is unable to use it.

I confess that I don't what could be the best approach to fix this problem but I use and I like very much Meson so I would love to have this case, usage inside MSYS2 bash, fixed.

Thank you in advance for any help.

Francesco

@lazka
Copy link
Contributor

lazka commented Jun 2, 2018

Do you run meson with mingw Python or the cygwin one?

If the mingw one you need to install the mingw pkg-config: pacman -S mingw-w64-i686-pkg-config

@nirbheek
Copy link
Member

nirbheek commented Jun 2, 2018

The problem arise when you invoke Meson and what happens is that Meson is not able to recognize and use the MSYS2-provided pkg-config.

What's the return value of shutil.which('pkg-config') inside a python 3 REPL?

Like @lazka said, mingw32/mingw-w64-i686-pkg-config and mingw64/mingw-w64-x86_64-pkg-config in MSYS2 work fine. Are you sure you're not accidentally using the msys/pkg-config package instead? Confusingly, that's not supposed to be used for building with mingw ;)

@franko
Copy link
Author

franko commented Jun 2, 2018

Hi all,

actually I was using the pkg-config from msys but I tried using the one from mingw64 as suggested and I have the same problem as before.

Now, with the package from mingw64, I get from python3's shutil the following:

>>> shutil.which('pkg-config')
'/mingw64/bin/pkg-config'

and from the bash shell:

> which pkg-confi
/mingw64/bin/pkg-config

and still Meson obstinately refuse to use it! :-)

Here the output:

$ meson --buildtype=debug build-debug
The Meson build system
Version: 0.46.1
Source dir: /home/utente/dev/libcanvas
Build dir: /home/utente/dev/libcanvas/build-debug
Build type: native build
Project name: libcavas
Native C compiler: cc (gcc 7.3.0 "cc.exe (Rev2, Built by MSYS2 project) 7.3.0")
Native C++ compiler: c++ (gcc 7.3.0 "c++.exe (Rev2, Built by MSYS2 project) 7.3.0")
Build machine cpu family: x86_64
Build machine cpu: x86_64
Found Pkg-config: NO

src/meson.build:1:0: ERROR:  Pkg-config not found.

In addition I may mention that I installed Meson from msys using python's pip installer.

@nirbheek
Copy link
Member

nirbheek commented Jun 2, 2018

Which python3 are you using? The same story as pkg-config is also for python3. Also MSYS2 ships with meson, so you don't need to use pip.

@franko
Copy link
Author

franko commented Jun 2, 2018

Python 3.6.2 from msys. I am just realizing now that meson is available as a msys2 package.

I tried the meson install with pacman and now it works, thank you!

Yet I am wondering if there isn't something that should be fixed anyway because the pip install was otherwise working and it is legitimate to install meson using pip.

@nirbheek
Copy link
Member

nirbheek commented Jun 3, 2018

I'm guessing that you are using msys/python instead of mingw64/mingw-w64-x86_64-python3.

I used mingw64/mingw-w64-x86_64-python3-pip and pip3 install meson, and the resulting meson worked fine.

I'm going to put a warning about that in meson.

nirbheek added a commit that referenced this issue Jun 3, 2018
This mistake seems to be a very common hiccup for people trying to use
Meson with MSYS2 on Windows from git or with pip.

msys/python uses POSIX paths with '/' as the root instead of a drive
like `C:/`, and also does not identify the platform as Windows.

This means that configure checks will be wrong, and many build tools
will be unable to parse the paths that are returned by functions in
Python such as shutil.which.

Closes #3653
nirbheek added a commit that referenced this issue Jun 3, 2018
This mistake seems to be a very common hiccup for people trying to use
Meson with MSYS2 on Windows from git or with pip.

msys/python uses POSIX paths with '/' as the root instead of a drive
like `C:/`, and also does not identify the platform as Windows.

This means that configure checks will be wrong, and many build tools
will be unable to parse the paths that are returned by functions in
Python such as shutil.which.

Closes #3653
nirbheek added a commit that referenced this issue Jun 3, 2018
This mistake seems to be a very common hiccup for people trying to use
Meson with MSYS2 on Windows from git or with pip.

msys/python uses POSIX paths with '/' as the root instead of a drive
like `C:/`, and also does not identify the platform as Windows.

This means that configure checks will be wrong, and many build tools
will be unable to parse the paths that are returned by functions in
Python such as shutil.which.

Closes #3653
nirbheek added a commit that referenced this issue Jun 3, 2018
This mistake seems to be a very common hiccup for people trying to use
Meson with MSYS2 on Windows from git or with pip.

msys/python uses POSIX paths with '/' as the root instead of a drive
like `C:/`, and also does not identify the platform as Windows.

This means that configure checks will be wrong, and many build tools
will be unable to parse the paths that are returned by functions in
Python such as shutil.which.

Closes #3653
@franko
Copy link
Author

franko commented Jun 3, 2018

Ok, got it, thank you very much for looking at that and for the good work you are doing on Meson.

bonzini pushed a commit to bonzini/meson that referenced this issue Jun 5, 2018
This mistake seems to be a very common hiccup for people trying to use
Meson with MSYS2 on Windows from git or with pip.

msys/python uses POSIX paths with '/' as the root instead of a drive
like `C:/`, and also does not identify the platform as Windows.

This means that configure checks will be wrong, and many build tools
will be unable to parse the paths that are returned by functions in
Python such as shutil.which.

Closes mesonbuild#3653
@LRN
Copy link

LRN commented Mar 18, 2019

Please amend this patch to detect where meson is installed before issuing that error. If meson is installed in /usr, then it's a MSYS meson, and it should use MSYS Python, nothing wrong with that. Otherwise (or if it's installed in /mingw*, whichever lets you sleep at night) it's a Windows meson, and it needs MinGW Python or W32 Python.

@nirbheek
Copy link
Member

Please amend this patch to detect where meson is installed before issuing that error. If meson is installed in /usr, then it's a MSYS meson, and it should use MSYS Python, nothing wrong with that

The point is that MSYS Python is completely broken for a build system's purposes, so it should never be used. If MSYS2 is shipping meson in its non-mingw distribution, that's wrong and it should stop doing that.

@LRN
Copy link

LRN commented Apr 7, 2019

How would you propose to build msys2-glib (or, even better, cygwin-glib) then?

@nirbheek
Copy link
Member

nirbheek commented Apr 7, 2019

I don't know, how is msys2-glib built at all? msys2-python3 can't even correctly resolve PATH afaict, how does that work in that case?

@LRN
Copy link

LRN commented Apr 7, 2019

It's built just fine. The only thing i had to patch out of meson to make it work on MSYS-MSYS2 is the This python3 seems to be msys/python on MSYS2 Windows, which is known to have path semantics incompatible with Meson error. I'm sure it's well-intended, but a bit misguided, since it's not just the platform you should be looking at, but the MSYSTEM envvar as well. If it's MINGW*, then the error is warranted. If it's MSYS, then it isn't.

After i patched that out, meson worked just fine, and i was able to compile msys2-glib (naturally, i had to also build msys2-ninja), and the only patches that needed were the ones that Cygwin applies anyway.

No idea what you mean by PATH resolution. And it's very surprising that this topic even comes up at all. MSYS2 is Cygwin with serial numbers filed off with a few patches on top. If python3 is broken on Cygwin, that would be huge news for Cygwin maintainers.

@nirbheek
Copy link
Member

nirbheek commented Apr 8, 2019

No idea what you mean by PATH resolution

I can't check right now because MSYS2 completely broke for me at some point (refuses to update saying error: msys: signature from "Alexey Pavlov (Alexpux) <alexpux@gmail.com>" is invalid, and a fresh installation fails in the same way). However, from what I remember, msys/python cannot resolve windows-style paths in PATH, so it can't find pkg-config at all. Perhaps that's just because it's not meant to be used from the MINGW64 prompt at all (which sounds like an MSYS2 bug?).

I'd be happy to review and merge a PR that disables the error when $MSYSTEM is MSYS.

@FrankHB
Copy link

FrankHB commented Dec 19, 2019

The incompatibility also comes from other MSYS2 tools like sed, which is not relevant to PATH. For example, MSYS tools will convert directory paths based on /etc/fstab configurations. This can mess up things totally. Perhaps there should be more warnings when MSYS2 tools are used.

lazka added a commit to lazka/meson that referenced this issue Apr 1, 2022
msys/python in MSYS2 pretends to be cygwin in all cases for some time
now, so this check was impossible to hit.

The underlying confusion it tried to prevent is still there, namely trying
to build with mingw but wrongly using a msys/cygwin python/meson.

We can use the MSYSTEM env var to detect if we are in a mingw shell, and
in case the Python doesn't match we suggest installing mingw variants of both
python and meson.

Using msys/python + meson in a MSYS environment works fine on the other hand,
so no need to error out in that case.

Fixes mesonbuild#8726

Also addresses the concern raised in
mesonbuild#3653 (comment)
xclaesse pushed a commit that referenced this issue Apr 1, 2022
msys/python in MSYS2 pretends to be cygwin in all cases for some time
now, so this check was impossible to hit.

The underlying confusion it tried to prevent is still there, namely trying
to build with mingw but wrongly using a msys/cygwin python/meson.

We can use the MSYSTEM env var to detect if we are in a mingw shell, and
in case the Python doesn't match we suggest installing mingw variants of both
python and meson.

Using msys/python + meson in a MSYS environment works fine on the other hand,
so no need to error out in that case.

Fixes #8726

Also addresses the concern raised in
#3653 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants