Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Unregistered SAT variable name' #202

Closed
jamescrowley opened this issue Jun 28, 2022 · 7 comments
Closed

'Unregistered SAT variable name' #202

jamescrowley opened this issue Jun 28, 2022 · 7 comments

Comments

@jamescrowley
Copy link

jamescrowley commented Jun 28, 2022

running 'conda-lock -f environment.yml' on the following yaml fails:

name: dataprocessing_dev
channels:
  - conda-forge
dependencies:
  - python=3.9
  - geopandas=0.10.2 # or <=
platforms:
  - osx-arm64
  - linux-64

but the following works:

name: dataprocessing_dev
channels:
  - conda-forge
dependencies:
  - python=3.9
  - geopandas>=0.10.2
platforms:
  - osx-arm64
  - linux-64

with

conda 4.13.0 py39hca03da5_0
conda-lock 1.0.5 pyhd8ed1ab_1 conda-forge
conda-package-handling 1.8.1 py39h1a28f6b_0

I get the following output

/Users/userx/miniconda3/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
Locking dependencies for ['linux-64', 'osx-arm64']...
INFO:conda_lock.conda_solver:linux-64 using specs ['python 3.9.*', 'geopandas 0.10.2.*']
Message key not found in json! returning the full json text
Could not lock the environment for platform linux-64
{'command': '/Users/userx/miniconda3/bin/conda create --prefix /var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs/prefix --dry-run --json --override-channels --channel conda-forge --channel file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm python 3.9.* geopandas 0.10.2.*', 'conda_info': {'GID': 20, 'UID': 501, 'active_prefix': '/Users/userx/miniconda3', 'active_prefix_name': 'base', 'av_data_dir': '/Users/userx/miniconda3/etc/conda', 'av_metadata_url_base': None, 'channels': ['https://conda.anaconda.org/conda-forge/linux-64', 'https://conda.anaconda.org/conda-forge/noarch', 'file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64', 'file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/noarch'], 'conda_build_version': 'not installed', 'conda_env_version': '4.13.0', 'conda_location': '/Users/userx/miniconda3/lib/python3.9/site-packages/conda', 'conda_prefix': '/Users/userx/miniconda3', 'conda_private': False, 'conda_shlvl': 1, 'conda_version': '4.13.0', 'config_files': ['/Users/userx/.condarc'], 'default_prefix': '/Users/userx/miniconda3', 'env_vars': {'CIO_TEST': '<not set>', 'CONDA_ADD_PIP_AS_PYTHON_DEPENDENCY': 'False', 'CONDA_DEFAULT_ENV': 'base', 'CONDA_EXE': '/Users/userx/miniconda3/bin/conda', 'CONDA_OVERRIDE_ARCHSPEC': '', 'CONDA_OVERRIDE_CUDA': '', 'CONDA_OVERRIDE_GLIBC': '', 'CONDA_OVERRIDE_LINUX': '', 'CONDA_OVERRIDE_OSX': '', 'CONDA_OVERRIDE_UNIX': '', 'CONDA_OVERRIDE_WIN': '', 'CONDA_PKGS_DIRS': '/var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs', 'CONDA_PREFIX': '/Users/userx/miniconda3', 'CONDA_PROMPT_MODIFIER': '(base) ', 'CONDA_PYTHON_EXE': '/Users/userx/miniconda3/bin/python', 'CONDA_ROOT': '/Users/userx/miniconda3', 'CONDA_SHLVL': '1', 'CONDA_SUBDIR': 'linux-64', 'CONDA_UNSATISFIABLE_HINTS_CHECK_DEPTH': '0', 'CURL_CA_BUNDLE': '<not set>', 'INFOPATH': '/opt/homebrew/share/info:', 'MANPATH': '/Users/userx/.nvm/versions/node/v18.2.0/share/man:/opt/homebrew/share/man::', 'PATH': '/Users/userx/miniconda3/bin:/Users/userx/.poetry/bin:/Users/userx/miniconda3/condabin:/Users/userx/.rbenv/shims:/Users/userx/.poetry/bin:/Users/userx/.nvm/versions/node/v18.2.0/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin', 'REQUESTS_CA_BUNDLE': '<not set>', 'SSL_CERT_FILE': '<not set>'}, 'envs': [], 'envs_dirs': ['/Users/userx/miniconda3/envs', '/Users/userx/.conda/envs'], 'netrc_file': None, 'offline': False, 'pkgs_dirs': ['/var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs'], 'platform': 'linux-64', 'python_version': '3.9.11.final.0', 'rc_path': '/Users/userx/.condarc', 'requests_version': '2.27.1', 'root_prefix': '/Users/userx/miniconda3', 'root_writable': True, 'site_dirs': [], 'sys.executable': '/Users/userx/miniconda3/bin/python', 'sys.prefix': '/Users/userx/miniconda3', 'sys.version': '3.9.11 (main, Mar 29 2022, 14:04:34) \n[Clang 12.0.0 ]', 'sys_rc_path': '/Users/userx/miniconda3/.condarc', 'user_agent': 'conda/4.13.0 requests/2.27.1 CPython/3.9.11 Darwin/21.4.0 OSX/12.3.1', 'user_rc_path': '/Users/userx/.condarc', 'virtual_pkgs': [['__linux', '0', '0'], ['__unix', '0', '0']]}, 'error': "ValueError('Unregistered SAT variable name: file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64::__linux-5.10-0')", 'exception_name': 'ValueError', 'exception_type': "<class 'ValueError'>", 'traceback': 'Traceback (most recent call last):\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py", line 125, in _convert\n    return self.names[name]\nKeyError: \'file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64::__linux-5.10-0\'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/exceptions.py", line 1114, in __call__\n    return func(*args, **kwargs)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/main.py", line 86, in main_subshell\n    exit_code = do_call(args, p)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 90, in do_call\n    return getattr(module, func_name)(args, parser)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/main_create.py", line 41, in execute\n    install(args, parser, \'create\')\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/install.py", line 259, in install\n    unlink_link_transaction = solver.solve_for_transaction(\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 152, in solve_for_transaction\n    unlink_precs, link_precs = self.solve_for_diff(update_modifier, deps_modifier,\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 195, in solve_for_diff\n    final_precs = self.solve_final_state(update_modifier, deps_modifier, prune, ignore_pinned,\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 313, in solve_final_state\n    ssc = self._add_specs(ssc)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py", line 707, in _add_specs\n    conflicts = ssc.r.get_conflicting_specs(tuple(MatchSpec(_)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/resolve.py", line 1109, in get_conflicting_specs\n    C = r2.gen_clauses()\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/io.py", line 86, in decorated\n    return f(*args, **kwds)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/resolve.py", line 911, in gen_clauses\n    nkey = C.Not(self.to_sat_name(prec))\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py", line 144, in Not\n    return self._eval(self._clauses.Not, (x,), (), polarity, name)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py", line 130, in _eval\n    args = self._convert(args)\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py", line 120, in _convert\n    return type(x)(map(self._convert, x))\n  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py", line 127, in _convert\n    raise ValueError("Unregistered SAT variable name: {}".format(name))\nValueError: Unregistered SAT variable name: file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64::__linux-5.10-0\n'}
    Command: ['/Users/userx/miniconda3/bin/conda', 'create', '--prefix', '/var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs/prefix', '--dry-run', '--json', '--override-channels', '--channel', 'conda-forge', '--channel', 'file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm', 'python 3.9.*', 'geopandas 0.10.2.*']
    STDOUT:
{
  "command": "/Users/userx/miniconda3/bin/conda create --prefix /var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs/prefix --dry-run --json --override-channels --channel conda-forge --channel file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm python 3.9.* geopandas 0.10.2.*",
  "conda_info": {
    "GID": 20,
    "UID": 501,
    "active_prefix": "/Users/userx/miniconda3",
    "active_prefix_name": "base",
    "av_data_dir": "/Users/userx/miniconda3/etc/conda",
    "av_metadata_url_base": null,
    "channels": [
      "https://conda.anaconda.org/conda-forge/linux-64",
      "https://conda.anaconda.org/conda-forge/noarch",
      "file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64",
      "file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/noarch"
    ],
    "conda_build_version": "not installed",
    "conda_env_version": "4.13.0",
    "conda_location": "/Users/userx/miniconda3/lib/python3.9/site-packages/conda",
    "conda_prefix": "/Users/userx/miniconda3",
    "conda_private": false,
    "conda_shlvl": 1,
    "conda_version": "4.13.0",
    "config_files": [
      "/Users/userx/.condarc"
    ],
    "default_prefix": "/Users/userx/miniconda3",
    "env_vars": {
      "CIO_TEST": "<not set>",
      "CONDA_ADD_PIP_AS_PYTHON_DEPENDENCY": "False",
      "CONDA_DEFAULT_ENV": "base",
      "CONDA_EXE": "/Users/userx/miniconda3/bin/conda",
      "CONDA_OVERRIDE_ARCHSPEC": "",
      "CONDA_OVERRIDE_CUDA": "",
      "CONDA_OVERRIDE_GLIBC": "",
      "CONDA_OVERRIDE_LINUX": "",
      "CONDA_OVERRIDE_OSX": "",
      "CONDA_OVERRIDE_UNIX": "",
      "CONDA_OVERRIDE_WIN": "",
      "CONDA_PKGS_DIRS": "/var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs",
      "CONDA_PREFIX": "/Users/userx/miniconda3",
      "CONDA_PROMPT_MODIFIER": "(base) ",
      "CONDA_PYTHON_EXE": "/Users/userx/miniconda3/bin/python",
      "CONDA_ROOT": "/Users/userx/miniconda3",
      "CONDA_SHLVL": "1",
      "CONDA_SUBDIR": "linux-64",
      "CONDA_UNSATISFIABLE_HINTS_CHECK_DEPTH": "0",
      "CURL_CA_BUNDLE": "<not set>",
      "INFOPATH": "/opt/homebrew/share/info:",
      "MANPATH": "/Users/userx/.nvm/versions/node/v18.2.0/share/man:/opt/homebrew/share/man::",
      "PATH": "/Users/userx/miniconda3/bin:/Users/userx/.poetry/bin:/Users/userx/miniconda3/condabin:/Users/userx/.rbenv/shims:/Users/userx/.poetry/bin:/Users/userx/.nvm/versions/node/v18.2.0/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin",
      "REQUESTS_CA_BUNDLE": "<not set>",
      "SSL_CERT_FILE": "<not set>"
    },
    "envs": [],
    "envs_dirs": [
      "/Users/userx/miniconda3/envs",
      "/Users/userx/.conda/envs"
    ],
    "netrc_file": null,
    "offline": false,
    "pkgs_dirs": [
      "/var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs"
    ],
    "platform": "linux-64",
    "python_version": "3.9.11.final.0",
    "rc_path": "/Users/userx/.condarc",
    "requests_version": "2.27.1",
    "root_prefix": "/Users/userx/miniconda3",
    "root_writable": true,
    "site_dirs": [],
    "sys.executable": "/Users/userx/miniconda3/bin/python",
    "sys.prefix": "/Users/userx/miniconda3",
    "sys.version": "3.9.11 (main, Mar 29 2022, 14:04:34) \n[Clang 12.0.0 ]",
    "sys_rc_path": "/Users/userx/miniconda3/.condarc",
    "user_agent": "conda/4.13.0 requests/2.27.1 CPython/3.9.11 Darwin/21.4.0 OSX/12.3.1",
    "user_rc_path": "/Users/userx/.condarc",
    "virtual_pkgs": [
      [
        "__linux",
        "0",
        "0"
      ],
      [
        "__unix",
        "0",
        "0"
      ]
    ]
  },
  "error": "ValueError('Unregistered SAT variable name: file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64::__linux-5.10-0')",
  "exception_name": "ValueError",
  "exception_type": "<class 'ValueError'>",
  "traceback": "Traceback (most recent call last):\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py\", line 125, in _convert\n    return self.names[name]\nKeyError: 'file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64::__linux-5.10-0'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/exceptions.py\", line 1114, in __call__\n    return func(*args, **kwargs)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/main.py\", line 86, in main_subshell\n    exit_code = do_call(args, p)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/conda_argparse.py\", line 90, in do_call\n    return getattr(module, func_name)(args, parser)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/main_create.py\", line 41, in execute\n    install(args, parser, 'create')\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/cli/install.py\", line 259, in install\n    unlink_link_transaction = solver.solve_for_transaction(\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py\", line 152, in solve_for_transaction\n    unlink_precs, link_precs = self.solve_for_diff(update_modifier, deps_modifier,\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py\", line 195, in solve_for_diff\n    final_precs = self.solve_final_state(update_modifier, deps_modifier, prune, ignore_pinned,\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py\", line 313, in solve_final_state\n    ssc = self._add_specs(ssc)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/core/solve.py\", line 707, in _add_specs\n    conflicts = ssc.r.get_conflicting_specs(tuple(MatchSpec(_)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/resolve.py\", line 1109, in get_conflicting_specs\n    C = r2.gen_clauses()\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/io.py\", line 86, in decorated\n    return f(*args, **kwds)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/resolve.py\", line 911, in gen_clauses\n    nkey = C.Not(self.to_sat_name(prec))\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py\", line 144, in Not\n    return self._eval(self._clauses.Not, (x,), (), polarity, name)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py\", line 130, in _eval\n    args = self._convert(args)\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py\", line 120, in _convert\n    return type(x)(map(self._convert, x))\n  File \"/Users/userx/miniconda3/lib/python3.9/site-packages/conda/common/logic.py\", line 127, in _convert\n    raise ValueError(\"Unregistered SAT variable name: {}\".format(name))\nValueError: Unregistered SAT variable name: file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm/linux-64::__linux-5.10-0\n"
}

Traceback (most recent call last):
  File "/Users/userx/miniconda3/bin/conda-lock", line 10, in <module>
    sys.exit(main())
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 1170, in lock
    lock_func(
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 940, in run_lock
    make_lock_files(
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 388, in make_lock_files
    lock_content = lock_content | create_lockfile_from_spec(
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 726, in create_lockfile_from_spec
    deps = _solve_for_arch(
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 678, in _solve_for_arch
    conda_deps = solve_conda(
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda_lock/conda_solver.py", line 159, in solve_conda
    dry_run_install = solve_specs_for_arch(
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/conda_lock/conda_solver.py", line 346, in solve_specs_for_arch
    proc.check_returncode()
  File "/Users/userx/miniconda3/lib/python3.9/site-packages/poetry/utils/_compat.py", line 168, in check_returncode
    raise CalledProcessError(
poetry.utils._compat.CalledProcessError: Command '['/Users/userx/miniconda3/bin/conda', 'create', '--prefix', '/var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs/prefix', '--dry-run', '--json', '--override-channels', '--channel', 'conda-forge', '--channel', 'file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm', 'python 3.9.*', 'geopandas 0.10.2.*']' returned non-zero exit status 1.
@mariusvniekerk
Copy link
Collaborator

Which versions of conda/ mamba are you using in conjunction with conda-lock?

@mariusvniekerk
Copy link
Collaborator

For complex dependencies like geopandas I would strongly recommend using mamba

@gabe-microsoft
Copy link

The problem is not just geopandas, as I've been seeing the same issue for about a month now. even for every simple environments (e.g., python>=3.8 and poetry>=1.1).

Yes, updating conda and/or switching from conda to mamba often does resolve the issue, but there's still a conda-lock bug here that should be explored.

Looking closely at the error message to identify the command that failed, we see:

/Users/userx/miniconda3/bin/conda create --prefix /var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpaly4pubs/prefix --dry-run --json --override-channels --channel conda-forge --channel file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm python 3.9.* geopandas 0.10.2.*

The unexpected thing here is --channel file:///var/folders/0g/hkd2l_zs4lb9yv3rtwrmv_000000gn/T/tmpf3_1j4bm. I don't know why conda-lock is using a local temp dir as a channel, but this is the root of the problem. The "Unregistered SAT variable name" error references a path within this temp dir. By removing this channel, conda will be able to solve the environment without problems.

So, how and why is conda-lock using a local temp dir as a channel, and what needs to be fixed in the conda-lock implementation to prevent this "Unregistered SAT variable name" within this temp dir?

@maresb
Copy link
Contributor

maresb commented Sep 17, 2022

I have seen this error in various complicated configurations, but never got around to tracking it down.

I'm trying to get the CI working again in the context of #240, and this is occurring in the CI pytest runs.

@maresb
Copy link
Contributor

maresb commented Sep 17, 2022

Looking into the source code, from what I can gather, the "virtual package channel" which @gabe-microsoft indicates is being used by conda-lock to inject virtual packages. These mechanisms are completely new to me, so I'm hoping that @mariusvniekerk can shed some light here, and hopefully suggest a way to resolve it.

The temp directory contains a subdirectory for each platform:

/tmp/tmp09t0cpji $ ls
linux-64  linux-aarch64  linux-ppc64le  noarch  osx-64  osx-arm64  win-64

Each subdirectory contains a single repodata.json file. Relevant for me is:

linux-64/repodata.json
{
  "info": {
    "subdir": "linux-64"
  },
  "packages": {
    "__archspec-1-x86_64_0.tar.bz2": {
      "build": "x86_64_0",
      "build_number": 0,
      "build_string": "x86_64",
      "depends": [],
      "name": "__archspec",
      "noarch": "",
      "subdir": "linux-64",
      "timestamp": 1577854800000,
      "version": "1"
    },
    "__cuda-11.4-0.tar.bz2": {
      "build": "0",
      "build_number": 0,
      "build_string": "",
      "depends": [],
      "name": "__cuda",
      "noarch": "",
      "subdir": "linux-64",
      "timestamp": 1577854800000,
      "version": "11.4"
    },
    "__glibc-2.17-0.tar.bz2": {
      "build": "0",
      "build_number": 0,
      "build_string": "",
      "depends": [],
      "name": "__glibc",
      "noarch": "",
      "subdir": "linux-64",
      "timestamp": 1577854800000,
      "version": "2.17"
    },
    "__linux-5.10-0.tar.bz2": {
      "build": "0",
      "build_number": 0,
      "build_string": "",
      "depends": [],
      "name": "__linux",
      "noarch": "",
      "subdir": "linux-64",
      "timestamp": 1577854800000,
      "version": "5.10"
    },
    "__unix-0-0.tar.bz2": {
      "build": "0",
      "build_number": 0,
      "build_string": "",
      "depends": [],
      "name": "__unix",
      "noarch": "",
      "subdir": "linux-64",
      "timestamp": 1577854800000,
      "version": "0"
    }
  }
}

The error message I'm getting is:

ValueError: Unregistered SAT variable name: file:///tmp/tmp09t0cpji/linux-64::__unix-0-0

which corresponds to the last listed virtual package. I'm not sure yet why this is causing conda to choke...

@maresb
Copy link
Contributor

maresb commented Sep 18, 2022

I have spent some time with the debugger tracing what leads up to this error. I think I more-or-less understand what's causing it. Here are my findings:

The linux-64/repodata.json file included above, which represents the so-called "virtual packages", is parsed and turned into a PackageRecord object here. This is a dataclass-like object which is created from the info dictionary:

PackageRecord(**(info := {
 'arch': None,
 'build': '0',
 'build_number': 0,
 'build_string': '',
 'channel': Channel("tmpologha1x/linux-64"),
 'depends': [],
 'fn': '__unix-0.0-0.tar.bz2',
 'name': '__unix',
 'noarch': '',
 'platform': None,
 'schannel': 'file:///tmp/tmpologha1x',
 'subdir': 'linux-64',
 'timestamp': 1577854800000,
 'url': 'file:///tmp/tmpologha1x/linux-64/__unix-0.0-0.tar.bz2',
 'version': '0.0'}))

Conda itself injects its own virtual packages named __unix, __linux, __glibc, and __archspec here, with the PackageRecord definition

name = "__unix"
PackageRecord(
  package_type=PackageType.VIRTUAL_SYSTEM,
  name=name,
  version=version or '0',
  build_string=build_string,
  channel='@',
  subdir=context.subdir,
  md5="12345678901234567890123456789012",
  build_number=0,
  fn=name,
)

Note that the package_type of conda-lock's virtual package is unset, while it is set for conda's virtual package.

This Resolve object plays a key role leading up to the error. The constructor of the Resolve object takes index as a parameter. This is a dict of package records where each key == value. (I'm not so sure what's the point of using a dict here.) These are grouped by name into self.groups. So groups is a dictionary which contains the key __unix and value being the two-element list consisting of both corresponding PackageRecords. But then the values of groups are filtered, so that if a group contains "unmanageable" package records, then only the unmanageable records are kept and fed to the solver. And "unmanageable" is defined based on the value of the package_type property. Remember that package_type for conda-lock's virtual package is None, so it's not "unmanageable" and gets filtered out from groups["__unix"].

For whatever reason it seems that the SAT problem is unsatisfiable, and then I end up here calling gen_clauses() on a Resolve object. This creates a new Clauses object and registers all the PackageRecords from the resolver's groups. We subsequently iterate over the resolver's index, and in the process we invoke C.Not('file:///tmp/tmpologha1x/linux-64::__unix-0-0').

Now the problem is clear (if you implausibly actually took the time to understand all of the above...)

We have filtered out conda-lock's virtual package from the resolver's groups["__unix"] because it was not "unmanageable" since package_type wasn't set. But when creating SAT clauses, we iterate not over the groups, but over the resolver's unfiltered index. And for this reason, there is no registered SAT variable for the PackageRecord of conda-lock's virtual __unix package.

😅

We might be able to solve this by adding "package_type": "virtual_system" to the repodata.json. I need to do some tests...

@maresb
Copy link
Contributor

maresb commented Sep 19, 2022

I think this is now fixed in main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants