Tab completion inside a shell

[nmounet]

Within a verdi shell, the following command is perfectly valid (and works):
from aiida.orm.calculation.inline import InlineCalculation

BUT, if one tries to obtain it using TAB completion, it works only up to aiida.orm.calculation and then it
does not work at all (e.g., typing aiida.orm.calculation.<TAB> does not give any list of choices, nor does e.g. aiida.orm.calculation.inl<TAB> or anything of that kind).

Worse: if one then goes manually until reaching the final import and then press TAB there, e.g.:

from aiida.orm.calculation.inline import Inline<TAB>

then everything begins to collapse: first an error is raised

Traceback (most recent call last):                                             
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner       
    self.run()                                                                 
  File "/usr/lib/python2.7/threading.py", line 763, in run                     
    self.__target(*self.__args, **self.__kwargs)
  File "/home/aiida/aiidapy/local/lib/python2.7/site-packages/prompt_toolkit/interface.py", line 865, in run
    completions = list(buffer.completer.get_completions(document, complete_event))
  File "/home/aiida/aiidapy/local/lib/python2.7/site-packages/IPython/terminal/ptutils.py", line 51, in get_completions
    cursor_pos=document.cursor_position_col
  File "/home/aiida/aiidapy/local/lib/python2.7/site-packages/IPython/core/completer.py", line 1169, in complete
    custom_res = self.dispatch_custom_completer(text)
  File "/home/aiida/aiidapy/local/lib/python2.7/site-packages/IPython/core/completer.py", line 1085, in dispatch_custom_completer
    res = c(event)
  File "/home/aiida/aiidapy/local/lib/python2.7/site-packages/IPython/core/completerlib.py", line 259, in module_completer
    return module_completion(event.line)
  File "/home/aiida/aiidapy/local/lib/python2.7/site-packages/IPython/core/completerlib.py", line 242, in module_completion
    return try_import(mod)
  File "/home/aiida/aiidapy/local/lib/python2.7/site-packages/IPython/core/completerlib.py", line 169, in try_import
    m = getattr(m, module)
AttributeError: 'module' object has no attribute 'inline'

and the shell cannot <TAB> complete anything anymore whatsoever (one has to restart it from scratch), i.e. even import aii<TAB> or import nump<TAB> will not autocomplete.

I presume this is related to the way the two possible backends (django and SQLalchemy) are implemented.

[DropD]

I can reproduce this using the same steps.

[DropD]

The reason seems to be the following:
aiida.orm.__init__ does

from aiida.orm.implementation import *
from aiida.orm.implementation.calculation import *

so once aiida.orm is imported, aiida.orm.calculation points to aiida.orm.implementation.calculation, which has no submodule inline, since it is just a module file. It can also not import aiida.orm.calculation.inline, because it is imported from there, which would lead to circular imports.

The Ipython completer however tries to import modules (using __import__) up to where you have been typing when you hit tab to give you suggestions. So when you type from aiida.orm.calculation import , you will actually see suggestions for aiida.orm.implementation.calculation.

If the IPython completer was using importlib.import_module, it probably would not get confused, as

In[1]: import_module('aiida.orm.calculation')
Out[1]: <module 'aiida.orm.calculation' from '/home/hauselmann/Documents/aiida-fork/aiida/orm/calculation/__init__.pyc'>

In [2]: import_module('aiida.orm')
Out[2]: <module 'aiida.orm.implementation.calculation' from '/home/hauselmann/Documents/aiida-fork/aiida/orm/implementation/calculation.pyc'>

However, likely IPython is not using this because it does not exist in older python2 versions.

Conclusion: If we want this to work, we need to help out the IPython interpreter to recognize what is going on by avoiding to overwrite submodule names with module members in aiida.orm.__init__.

Suggested solution: instead of importing everything in orm.__init__, leave the necessary imports to orm.calculation and import that instead.

[DropD]

This is exactly the reason why from ... import * is strongly discouraged inside libraries by anyone with a bit of experience! Unless of course the module you import from has __all__ defined.