Matfile Creation for IMOS and potetnially others

[lachlan194924]

saving imos subfolders seperately works.
however when combined there seems to be issues.

In my experimentiation. Where i deleted everything in the dataware house except having one IMOS subfolder at a time i found that the "agency" copy of the dataset was bigger than the "global" copy, which would make sense if there were other things in the dataware house, but there wasnt.

I explored these two structures and couldnt see any difference.

I read through the code and found that the "agency" copy of the data is saved with a generic save command if its smaller than 1700Mb, however anything larger gets saved as a version 7.3 file.

The global one is always saved like this.
So I took the global one and saved it with a generic statement and it was the same as the agency one.

Not sure why imos "crashes" the pipeline it appears to freeze. Because there are no errors, but top and gnome system monitor dont detect any active processes, so while technically it could just be creating an enormous .mat it doesnt appear to be.