python: import statement not generated when using KSY imports in a certain way

[armijnhemel]

When generating Python code with Kaitai 0.9 for the .wav some import statements that should be generated are not generated.

The wav.ksy file has an import:

  imports:
    - /common/riff

The code that is generated does not import the riff module. When adding a dummy type like this and regenerating, then it is:

  dummy:
    seq:
      - id: dummy_id
        type: riff

The difference in generated code:

]$ diff -u wav.py.orig wav.py
--- wav.py.orig	2021-04-22 20:23:00.731716225 +0200
+++ wav.py	2021-04-22 20:23:14.630908638 +0200
@@ -9,6 +9,7 @@
 if parse_version(kaitaistruct.__version__) < parse_version('0.9'):
     raise Exception("Incompatible Kaitai Struct Python API: 0.9 or later is required, but you have %s" % (kaitaistruct.__version__))
 
+import riff
 class Wav(KaitaiStruct):
     """The WAVE file format is a subset of Microsoft's RIFF specification for the
     storage of multimedia files. A RIFF file starts out with a file header
@@ -456,6 +457,17 @@
             self.samples = self._io.read_u4le()
 
 
+    class Dummy(KaitaiStruct):
+        def __init__(self, _io, _parent=None, _root=None):
+            self._io = _io
+            self._parent = _parent
+            self._root = _root if _root else self
+            self._read()
+
+        def _read(self):
+            self.dummy_id = riff.Riff(self._io)
+
+
     class ChannelMaskAndSubformatType(KaitaiStruct):
         def __init__(self, _io, _parent=None, _root=None):
             self._io = _io

Another issue is that the class Riff, which is used in the code, is not imported either.

$ grep Riff wav.py.orig 
        self.chunk = Riff.Chunk(self._io, self, self._root)
            self.chunk = Riff.Chunk(self._io, self, self._root)
            self.parent_chunk_data = Riff.ParentChunkData(self._io, self, self._root)
            self.chunk = Riff.Chunk(self._io, self, self._root)
            self._m_parent_chunk_data = Riff.ParentChunkData(io, self, self._root)

Normally I would have expected something like from riff import Riff for it to work.

My guess is that referencing an enum in an imported file is where code generation fails:

seq:
  - id: chunk
    type: 'riff::chunk'

I don't know where in the compiler to fix this.

[generalmimon]

Last time I investigated this a bit (more specifically, while looking into #295 and soon realized that there isn't a quick fix but rather an architectonic change), I found that there isn't anything in the compiler code like iterating the /meta/imports and adding the imports one by one to the generated code. When the compiler generates the parser module for the top-level spec, the imports are treated exactly the same as opaque types, except that the presence of the type name in /meta/imports "legalizes" the usage of the specific type - otherwise the compiler would complain about encountering an unknown type (because the /meta/ks-opaque-types option is not enabled). So /meta/imports is kind of like /meta/ks-opaque-types: true except that the allowed types are enumerated.

Note that this applies only in the part of KSC handling the actual code generation - other parts require the imported spec to be present, because the type derivations and checks depend on it.

[Mimickal]

Just so I understand, are imports entirely unusable at the moment?
I'm a little confused because of this line:

The code that is generated does not import the riff module. When adding a dummy type like this and regenerating, then it is

This seems to imply there is some combination of tricks that cause imports to be generated, but I have not found one that generates the expected import statement in any language.

[generalmimon]

This issue looks as a duplicate of #703, so I'll close this one too. We're not doing a great job at identifying duplicate issues 😐

---

@Mimickal:

Just so I understand, are imports entirely unusable at the moment?

I don't think they're unusable, except that a module does not get imported just because you reference its inner type/enum. Importing other modules as a whole works fine, AFAIK.

Referencing the top-level type of the imported module from some "dummy" type will add the import for sure, but I'm not sure whether this alone is enough to allow referencing nested types/enums in the imported spec - I wouldn't be surprised if it generates an invalid path to the imported type/enum for some languages.

[generalmimon]

Duplicate of #703