TypeError: descriptor '__call__' for 'type' objects doesn't apply to a 'property' object

[degrainger]

Describe the bug
I am running the SCENICplus Snakemake pipeline but get the error described below. When I paste the error into ChatGPT it says "due to a compatibility issue between different versions of the libraries being used. Specifically, the error suggests a problem with the dask library, which is a dependency for SCENIC+".

Please could you help me identify where this error stems from? Thank you.

Error output

Assuming unrestricted shared filesystem usage for local execution.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 20
Rules claiming more threads will be scaled down.
Job stats:
job count

---

AUCell_direct 1
AUCell_extended 1
all 1
eGRN_direct 1
eGRN_extended 1
region_to_gene 1
scplus_mudata 1
tf_to_gene 1
total 8

Select jobs to execute...
Execute 1 jobs...

[Wed Jun 26 20:55:46 2024]
localrule region_to_gene:
input: ACC_GEX.h5mu, search_space.tsv
output: region_to_gene_adj.tsv
jobid: 10
reason: Missing output files: region_to_gene_adj.tsv
threads: 20
resources: tmpdir=/var/scratch/ostone/1800216

2024-06-26 20:56:49,359 SCENIC+ INFO Reading multiome MuData.
/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/anndata/_core/anndata.py:522: FutureWarning: The dtype argument is deprecated and will be removed in late 2024.
warnings.warn(
/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/anndata/_core/anndata.py:522: FutureWarning: The dtype argument is deprecated and will be removed in late 2024.
warnings.warn(
2024-06-26 20:56:56,896 SCENIC+ INFO Reading search space
2024-06-26 20:56:57,586 R2G INFO Calculating region to gene importances, using GBM method

Running using 20 cores: 0%| | 0/16994 [00:00<?, ?it/s]
Running using 20 cores: 0%| | 1/16994 [00:00<37:41, 7.51it/s]
Running using 20 cores: 0%| | 12/16994 [00:00<04:59, 56.76it/s]
Running using 20 cores: 0%| | 20/16994 [00:02<47:54, 5.91it/s]
Running using 20 cores: 0%| | 34/16994 [00:02<22:47, 12.40it/s]
Running using 20 cores: 0%| | 34/16994 [00:19<22:47, 12.40it/s]joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
r = call_item()
^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 291, in call
return self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 589, in call
return [func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 589, in
return [func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/scenicplus/enhancer_to_gene.py", line 140, in _score_regions_to_single_gene
from arboreto import core as arboreto_core
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/arboreto/core.py", line 12, in
from dask.dataframe import from_delayed
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/init.py", line 100, in
from dask.dataframe import backends, dispatch, rolling
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/backends.py", line 15, in
from dask.dataframe.core import DataFrame, Index, Scalar, Series, _Frame
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/core.py", line 36, in
from dask.dataframe import methods
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/methods.py", line 34, in
from dask.dataframe.utils import is_dataframe_like, is_index_like, is_series_like
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/utils.py", line 20, in
from dask.dataframe import ( # noqa: F401 register pandas extension types
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/_dtypes.py", line 9, in
from dask.dataframe.extensions import make_array_nonempty, make_scalar
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/extensions.py", line 8, in
from dask.dataframe.accessor import (
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 126, in
class DatetimeAccessor(Accessor):
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 81, in init_subclass
_bind_property(cls, pd_cls, attr, min_version)
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/dataframe/accessor.py", line 35, in _bind_property
setattr(cls, attr, property(derived_from(pd_cls, version=min_version)(func)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/utils.py", line 987, in wrapper
method.doc = _derived_from(
^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/utils.py", line 940, in _derived_from
method_args = get_named_args(method)
^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/dask/utils.py", line 701, in get_named_args
s = inspect.signature(func)
^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/inspect.py", line 3263, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/inspect.py", line 3011, in from_callable
return _signature_from_callable(obj, sigcls=cls,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/inspect.py", line 2599, in _signature_from_callable
call = _descriptor_get(call, obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/inspect.py", line 2432, in _descriptor_get
return get(descriptor, obj, type(obj))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: descriptor 'call' for 'type' objects doesn't apply to a 'property' object
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/bin/scenicplus", line 8, in
sys.exit(main())
^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/scenicplus/cli/scenicplus.py", line 1137, in main
args.func(args)
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/scenicplus/cli/scenicplus.py", line 330, in TF_to_gene
infer_region_to_gene(
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/scenicplus/cli/commands.py", line 739, in infer_region_to_gene
adj = calculate_regions_to_genes_relationships(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/scenicplus/enhancer_to_gene.py", line 261, in calculate_regions_to_genes_relationships
region_to_gene_importances = _score_regions_to_genes(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/scenicplus/enhancer_to_gene.py", line 219, in _score_regions_to_genes
joblib.Parallel(
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 1952, in call
return output if self.return_generator else list(output)
^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 1595, in _get_outputs
yield from self._retrieve()
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 1699, in _retrieve
self._raise_error_fast()
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 1734, in _raise_error_fast
error_job.get_result(self.timeout)
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 736, in get_result
return self._return_or_raise()
^^^^^^^^^^^^^^^^^^^^^^^
File "/ceph/project/Stone_multiome/ostone/conda_local/envs/scenicplus/lib/python3.11/site-packages/joblib/parallel.py", line 754, in _return_or_raise
raise self._result
TypeError: descriptor 'call' for 'type' objects doesn't apply to a 'property' object
[Wed Jun 26 20:57:24 2024]
Error in rule region_to_gene:
jobid: 10
input: ACC_GEX.h5mu, search_space.tsv
output: region_to_gene_adj.tsv
shell:

    scenicplus grn_inference region_to_gene             --multiome_mudata_fname ACC_GEX.h5mu             --search_space_fname search_space.tsv             --temp_dir /ceph/project/Stone_multiome/ostone/SCENICplus/scplus_pipeline/tmp             --out_region_to_gene_adjacencies region_to_gene_adj.tsv             --importance_scoring_method GBM             --correlation_scoring_method SR             --n_cpu 20
    
    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-06-26T205536.041583.snakemake.log
WorkflowError:
At least one job did not complete successfully.

Version (please complete the following information):
Python: 3.11.9
arboreto 0.1.6
dask 2024.2.1
joblib 1.3.2
scenicplus 1.0a1

[MatthewTCManion]

I ran into this issue and it seems to be an incompatibility between dask 2024.2.1 and Python 3.11.9. I was able to resolve this particular issue by rolling back to Python 3.11.8, but I haven't confirmed that the rest of the pipeline will work after this change.

[MatthewTCManion]

I ran into this issue and it seems to be an incompatibility between dask 2024.2.1 and Python 3.11.9. I was able to resolve this particular issue by rolling back to Python 3.11.8, but I haven't confirmed that the rest of the pipeline will work after this change.

On second thought, I am seeing other people resolved this by updating dask to 2024.5.0, which gives a warning, but it sounds like it works fine after that. #386

[degrainger]

Thanks very much @MatthewTCManion! Rolling back to Python 3.11.8 solved my problem and allowed the pipeline to run to the end 😄