Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

utf-8 decode error when loading hdf5 dependency #12740

Closed
EigenDev opened this issue Jan 14, 2024 · 6 comments
Closed

utf-8 decode error when loading hdf5 dependency #12740

EigenDev opened this issue Jan 14, 2024 · 6 comments

Comments

@EigenDev
Copy link

Describe the bug
Whenever I try to load the hdf5 dependency, I get the following:

The Meson build system
Version: 1.3.1
Source dir: /home/mdupont/research/code_test/meson
Build dir: /home/mdupont/research/code_test/meson/builddir
Build type: native build
Project name: test
Project version: undefined
C++ compiler for the host machine: c++ (gcc 13.2.1 "c++ (GCC) 13.2.1 20230801")
C++ linker for the host machine: c++ ld.bfd 2.41.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: YES (/usr/bin/pkg-config) 2.1.0
Traceback (most recent call last):
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/mesonmain.py", line 194, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/msetup.py", line 358, in run
    app.generate()
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/msetup.py", line 181, in generate
    return self._generate(env, capture, vslite_ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/msetup.py", line 226, in _generate
    intr.run()
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreter/interpreter.py", line 3004, in run
    super().run()
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 172, in run
    self.evaluate_codeblock(self.ast, start=1)
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 198, in evaluate_codeblock
    raise e
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 208, in evaluate_statement
    self.assignment(cur)
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 644, in assignment
    value = self.evaluate_statement(node.value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 204, in evaluate_statement
    return self.function_call(cur)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 530, in function_call
    res = func(node, func_args, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/decorators.py", line 809, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/decorators.py", line 809, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/decorators.py", line 809, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 4 more times]
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/decorators.py", line 108, in wrapped
    ret = f(*wrapped_args, **wrapped_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/decorators.py", line 126, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/decorators.py", line 260, in wrapper
    return f(*nargs, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreterbase/decorators.py", line 579, in wrapper
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreter/interpreter.py", line 1794, in func_dependency
    d = df.lookup(kwargs)
        ^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreter/dependencyfallbacks.py", line 355, in lookup
    dep = func(kwargs, func_args, func_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/interpreter/dependencyfallbacks.py", line 87, in _do_dependency
    dep = dependencies.find_external_dependency(name, self.environment, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/dependencies/detect.py", line 115, in find_external_dependency
    candidates = _build_external_dependency_list(name, env, for_machine, kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/dependencies/detect.py", line 200, in _build_external_dependency_list
    dep = entry2(env, for_machine, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/dependencies/factory.py", line 152, in wrapped
    return func(env, for_machine, kwargs, process_method_kw(methods, kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/dependencies/hdf5.py", line 161, in hdf5_factory
    for mod in pkg.list_all():
               ^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/dependencies/pkgconfig.py", line 201, in list_all
    ret, out, err = self._call_pkgbin(['--list-all'])
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/dependencies/pkgconfig.py", line 280, in _call_pkgbin
    p, out, err = Popen_safe_logged(cmd, env=env)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/utils/universal.py", line 1572, in Popen_safe_logged
    p, o, e = Popen_safe(args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/site-packages/mesonbuild/utils/universal.py", line 1530, in Popen_safe
    o, e = p.communicate(write)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/subprocess.py", line 1209, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/subprocess.py", line 2146, in _communicate
    stdout = self._translate_newlines(stdout,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mdupont/miniconda3/lib/python3.11/subprocess.py", line 1086, in _translate_newlines
    data = data.decode(encoding, errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 104972: invalid start byte

meson.build:2:10: ERROR: Unhandled python exception

    This is a Meson bug and should be reported!

To Reproduce
meson.build:

project('test', 'cpp')
hdf5 = dependency('hdf5', language: 'cpp', required: true)

Expected behavior
meson builds successfully

system parameters

  • Is this a cross build or just a plain native build (for the same computer)?
    native build.
  • what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.)
    arch linux.
  • what Python version are you using e.g. 3.8.0
    3.11.6
  • what meson --version
    1.3.1
  • what ninja --version if it's a Ninja build
    1.11.1.git.kitware.jobserver-1
@bruchar1
Copy link
Member

I don't think this is directly related to hdf5. It seems to be the encoding on Popen that is not set, or wrongly set. What is the locale in your terminal?

@EigenDev
Copy link
Author

EigenDev commented Jan 17, 2024 •
edited
Loading

Thanks for your reply.
locale -a returns:

C
C.utf8
en_US
en_US.iso88591
en_US.utf8
POSIX

and python -c "import subprocess; subprocess.Popen(['locale', '-a'])" outputs:

C
C.utf8
en_US
en_US.iso88591
en_US.utf8
POSIX

locale outputs:

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

and python -c "import subprocess; subprocess.Popen(['locale'])" outputs:

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

@bruchar1
Copy link
Member

If you set the environment variable PYTHONIOENCODING=UTF-8, does it changes something?

If you call pkg-config --list-all, does the output contain non utf-8 characters?

@EigenDev
Copy link
Author

doing export PYTHONIOENCODING=UTF-8 doesn't change anything and pkg-config --list-all lists many packages, and I do not immediately see any non utf-8 characters there.

@EigenDev
Copy link
Author

EigenDev commented Jan 20, 2024 •
edited
Loading

@bruchar1 after taking another look and doing grep -axv '.*' pkg_dump.txt where pkg_dump.txt is just the output of pkg-config --list-all, I am seeing:

vpl                            Intel� Video Processing Library - Accelerated video decode, encode, and frame processing capabilities on Intel� GPUs`

I don't exactly know how to fix this.

@EigenDev
Copy link
Author

Nevermind, I got it fixed. The issue is the introduction of the (R) symbol by Intel when going from openvpl to libvpl. I fixed it by following this solution. Thanks for the hint, @bruchar1 !!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bruchar1 @EigenDev