Import / Export without empty directories

[PhilippRue]

For some nodes AiiDA generates empty directories in the repository.
In my case I removed some of these to reduce the number of inodes I have. The database export and import now poses a problem since it runs into these (false positive) exception:

aiida-core/aiida/tools/importexport/dbexport/__init__.py

Line 500 in 12f9641

raise exceptions.ArchiveExportError(

and

aiida-core/aiida/tools/importexport/dbimport/backends/django/__init__.py

Line 392 in 12f9641

if not subfolder.exists():

It would probably be good to not require empty directories to be included in import/export files.

[ltalirz]

@sphuber Let's discuss whether it perhaps makes sense to - until the repo migration - add a "lazy" node folder creation that only creates node folders when something should be put inside.

[CasperWA]

I agree - and it should be a fairly easy(TM) fix in the end. I can try and knock it out.

[ltalirz]

One comment by @sphuber : Since we are currently not tracking the content of folders, this change would mean that we are no longer able to detect whether the folder was (e.g. accidentally) deleted.

I.e. we should weigh whether the reduction in inodes is worth giving up this consistency check (which is of course quite basic - e.g. missing files are anyhow not detected).

Perhaps @giovannipizzi can comment

[ltalirz]

This will be addressed by the new archive format aiidateam/AEP#21

[chrisjsewell]

Heya, I've marked this a being closed by: #3445.
This is because with this new repository system there will actually no longer be a folder system on disk; files are stored in a compressed format (indexed by hash), with the node's database entry containing a virtual mapping of the actual file/folder structure.

Let's discuss whether it perhaps makes sense to - until the repo migration - add a "lazy" node folder creation that only creates node folders when something should be put inside.

I very much doubt anyone's going to have time to do this now, it'll just be wasted effort