PicklingError: Can't pickle <class 'genclass.genclass.<locals>.Test'>: it's not found as genclass.genclass.<locals>.Test

[ghost]

I have a Jupyter notebook with the following code:

import dill
from genclass import *

dill.dumps(genclass())

Plus a file genclass.py in the same folder:

def genclass():
  class Test:
    pass
  return Test

This results in the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/usr/lib/python3.6/pickle.py in save_global(self, obj, name)
    917             module = sys.modules[module_name]
--> 918             obj2, parent = _getattribute(module, name)
    919         except (ImportError, KeyError, AttributeError):

/usr/lib/python3.6/pickle.py in _getattribute(obj, name)
    265             raise AttributeError("Can't get local attribute {!r} on {!r}"
--> 266                                  .format(name, obj))
    267         try:

AttributeError: Can't get local attribute 'genclass.<locals>.Test' on <function genclass at 0x7f4f6c694510>

During handling of the above exception, another exception occurred:

PicklingError                             Traceback (most recent call last)
<ipython-input-8-31411f25147e> in <module>
      2 from genclass import *
      3 
----> 4 dill.dumps(genclass())

~/.local/share/virtualenvs/w6BVVrpR/lib/python3.6/site-packages/dill/_dill.py in dumps(obj, protocol, byref, fmode, recurse)
    292     """pickle an object to a string"""
    293     file = StringIO()
--> 294     dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
    295     return file.getvalue()
    296 

~/.local/share/virtualenvs/w6BVVrpR/lib/python3.6/site-packages/dill/_dill.py in dump(obj, file, protocol, byref, fmode, recurse)
    285         raise PicklingError(msg)
    286     else:
--> 287         pik.dump(obj)
    288     stack.clear()  # clear record of 'recursion-sensitive' pickled objects
    289     return

/usr/lib/python3.6/pickle.py in dump(self, obj)
    407         if self.proto >= 4:
    408             self.framer.start_framing()
--> 409         self.save(obj)
    410         self.write(STOP)
    411         self.framer.end_framing()

/usr/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    474         f = self.dispatch.get(t)
    475         if f is not None:
--> 476             f(self, obj) # Call unbound method with explicit self
    477             return
    478 

~/.local/share/virtualenvs/w6BVVrpR/lib/python3.6/site-packages/dill/_dill.py in save_type(pickler, obj)
   1328        #print ("%s\n%s" % (type(obj), obj.__name__))
   1329        #print ("%s\n%s" % (obj.__bases__, obj.__dict__))
-> 1330         StockPickler.save_global(pickler, obj)
   1331         log.info("# T4")
   1332     return

/usr/lib/python3.6/pickle.py in save_global(self, obj, name)
    920             raise PicklingError(
    921                 "Can't pickle %r: it's not found as %s.%s" %
--> 922                 (obj, module_name, name))
    923         else:
    924             if obj2 is not obj:

PicklingError: Can't pickle <class 'genclass.genclass.<locals>.Test'>: it's not found as genclass.genclass.<locals>.Test

Interestingly, the following code works just fine:

import dill

def genclass():
  class Test:
    pass
  return Test


dill.dumps(genclass())

There are some issues with similar error messages, but at least the ones I have seen are all fixed. So this is most likely something else.

Versions:

• Python: 3.6
• Dill: 0.29

[mmckerns]

Not sure I'm understanding you correctly... in the first case, you have have the class in a local file that you import -- importantly, not in an installed module. In the second case, you define the class locally. Is that correct? If so, this may be a duplicate of #123, #128, or #56.

[ghost]

The Python file is in the same folder as the Jupyter notebook. There are no other modules involved. In the second case (which works) the class is in the Jupyter notebook.

[mmckerns]

Ok, thanks for the clarification. So, yes, this is a duplicate issue. dill imports the class differently if it's from __main__ versus if it's from another module. If it's from another module, it tries to import it by reference... and if the module is not installed (and thus can't be read by reference) then, it will fail. You can workaround this by artificially changing the class's module attribute to be __main__.

I'm closing this issue as duplicate. However, I think I have a fix in mind, and I'll work on this issue as part of the others listed above.