AFP and sample track: allow relative sample paths like ./sample.flac

[unfa]

I'd like to be able to place my project file and samples it uses in the same directory, then ZIP the whole thing and send t to a friend, so he can load the project and play it with the samples in place.
But it looks like LMMS saves the absolute paths to samples and there's no way to choose otherwise, so the samples never work when I move the files to another machine.

Am I missing something?

[tresf]

But it looks like LMMS saves the absolute paths to samples and there's no way to choose otherwise, so the samples never work when I move the files to another machine.

Am I missing something?

Yes you are. Just set your home directory properly and save everything within that home directory and you should be all set. I've had the best results placing samples in samples, projects in projects, etc.

My collaboration team uses a cloud storage service that syncs locally and we all have different places where LMMS is installed (some on different OSs) and if the home directory is setup properly in LMMS, we can all open the projects.

Over the years there have been a few relative path bugs (such as VSTs), but 1.1 seems to have that fixed. 👍

related: #1257

[unfa]

Ok, I don't like placing samples in samples folder and project files elsewhere, because I want to sort files by projects (tracks, songs) - when I move a project, I move one directory and everything else that belongs to the projects goes with it (project files, lyrics, samples, ardour sessions, graphic designs etc). Keeping samples in /samples directory makes it looks like I'm installing a unix program (binaries in /usr/bin, data in /usr/share/ etc.) I don't like that personallly. Would there be a way to use relative paths and allow samples to be in the same folder as the project file and load correctly regardless of the LMMS home directory configuration?

[tresf]

Keeping samples in /samples directory makes it looks like I'm installing a unix program

With a large enough sample collection, this approach is usually preferred. It doesn't work well for the track-specific samples though as they tend to never be re-used.

because I want to sort files by projects (tracks, songs) - when I move a project, I move one directory and everything else that belongs to the projects goes with it

In this case, I would say the "export" feature is really what you want to allow archiving.

Would there be a way to use relative paths and allow samples to be in the same folder as the project file and load correctly regardless of the LMMS home directory configuration?

Well, for starters we should treat everything in the home directory as relative (rather than the current method of looking for samples in "samples", etc. That way you can have a folder test project containing test.ogg and test.mmpz. Currently, placing any samples in the project folder stores them as absolute paths, i.e. E:/My LMMS Stuff/projects/test.ogg which should be fixed.

In regards to the relative-ness of projects and samples, I can see that potentially causing backwards compat issues, but still might be workable. :)

[Wallacoloo]

Long-term, I'd like to see a solution where the audio sample is hashed and
we store that hash as an identifier (maybe alongside the filename). Then,
if the sample or project folder ever moves, or the user reorganizes their
samples directory, we can build a list of all the samples on the user's
system and figure out the new location by matching hashes.

Since audio sample could be modified outside of lmms, we would probably
need to retain the current path handling and just use hash lookups as a
fallback for when we can't find the sample.

[tresf]

Long-term, I'd like to see a solution where the audio sample is hashed and
we store that hash as an identifier (maybe alongside the filename). Then,
if the sample or project folder ever moves, or the user reorganizes their
samples directory, we can build a list of all the samples on the user's
system and figure out the new location by matching hashes.

That is quite a complex proposal for a few moved sample files, don't you think? I guess it could be nice though for scalability in renaming sample files that ship with the software as that would help backwards compat, but we'll be running an SQLite instance by the time we're done. :)

[Wallacoloo]

@tresf I don't think it'd be THAT complex, though there is a reason that I said "long term". Whenever an audio clip is loaded, calculate its hash & store that as a property on the clip. Then when the project is saved, also store the hash. If you ever try to open a project and a file cannot be found, a dialog would pop up and say "x cannot be found at y. It may have been moved. Have lmms try to locate it?" If user answers yes, we go through all the audio files in the sample directories and project directories, hash them & compare against our stored hash. To reduce processing time, we could hash & compare only files with the same name before checking the rest. That sounds like a 300 line change, or maybe a tad less, plus dropping in some pre-written hash function/library. The only reason it would become more complex is if you're concerned about processing time and want to cache the hashes for the user's sample library on disk. But since the lookups only happen on very rare occasions, you could probably get away without doing any caching.

[tresf]

if you're concerned about processing time and want to cache
the hashes for the user's sample library on disk. But since the lookups
only happen on very rare occasions, you could probably get away without
doing any caching.

Well, for a single sample perhaps. Take a project where 20 samples have been moved in a library of 1,000 samples times the average time for the hash algorithm (larger files calls for a larger hash usually, right?) and this could quickly turn into a lengthy process.

And albeit obvious... Multiple instances of the same hash would have to have all identical hash instances in the mmpz automatically replaced after finding the first match just in case someone used the same sample1.ogg file on several different tracks. :)

[Wallacoloo]

Actually, my thinking was that it becomes more justifiable when you're searching for more files. It's ok to cache the hash:file mappings in ram since the files aren't likely to chance on disk during the brief period of time that's spent searching, so on each successive lookup the number of hashes you're likely to need to calculate decreases. But yes, I don't see a good way around dealing with duplicated samples that reside in multiple locations. Would probably need to prompt the user to choose one. Edge-cases - Gah. Ok, so maybe just changing behavior so that samples located in the same folder as the project (or a deeper folder nested within it) are stored relative to the project file is best?

[unfa]

+1 for relative sample paths. Maybe for backwards compatibility relative paths should always start with a dot:

Linux:

./sample.flac
../vocal samples/hello world.flac

Windows:

.\sample.flac
..\vocal samples\hello world.flac

This way the old paths can be recognized as absolute ( I think users should be able to force any of these anyway), because they don't start with a dot:

Linux and Windows:

/home/lmms/samples/vocal/me/whoa.flac
C:\Documents and Setings\lmms user\lmms\samples\blah.flac

Another way to distinguish absolute paths would be to see if the path starts with root dir "/" or Windows drive "C:" etc.

One more could be simply a flag in XML like:

<path_relative>true</path_relative>

Adds some extra bytes, but saves on program logic I guess...

[tresf]

Ok, so maybe just changing behavior so that samples located in the same
folder as the project (or a deeper folder nested within it) are stored
relative to the project file is best?

Yeah that's probably best. The relative path "dot" rule unfa mentioned will likely work as I don't believe dots work currently in any practical fashion (they didn't work in my testing, as they probably look for the file relative to the lmms executable, which IMO we'd never use). This just might do the trick... :)

[unfa]

Hmm, and what if I want to have project files like:

album/tracks/project.mmpz
album/samples/sample.flac

The relative path from project.mmpz to sample.flac would be

../samples/sample/flac

This would be impossible with the "same or sub directory" approach. In such cases I'd prefer a checkbox "use relative path" and not relay on any automatic, that won't understand my vision.

[tresf]

Right... or just stop putting your samples in the sample directory and let the relative logic work its magic, which was the point of this to begin with (right?).

[tresf]

Actually your example would still be fine with the proposed logic so as long as "album" isn't your LMMS home directory.

[unfa]

No but "album" migth be under LMMS home directory. Sometimes it is, sometimes it isn't.
I'd really like a switch to manually decide whether use relative or absolute paths. It would save a ton of headache. However I see that the GUI change is the most difficult one here. Maybe it could be in the file selection dialog?

[Wallacoloo]

Rather than trying to figure out when to store the path as relative and when to use an absolute one, could we just store BOTH the absolute and relative path, and then when the project loads, try relative first and fall back onto the absolute path if the relative path no longer points to a valid file?

[unfa]

That sounds like the most rugged solution! And it will always work automagically without user interaction.

[softrabbit]

Would there be a way to use relative paths and allow samples to be in the same folder as the project file and load correctly regardless of the LMMS home directory configuration?

Right now you should be able to work around this if you create a directory under ~/lmms (add /projects to that if you like) and keep all the files the project needs there. Tried that some time back when someone on IRC was asking about collaboration through git. NB. if you rename the project directory the paths in the file will be invalid.

Ok, so maybe just changing behavior so that samples located in the same folder as the project (or a deeper folder nested within it) are stored relative to the project file is best?

Absolutely. And I'd go as far as having all data files the project needs in the project directory (defined as the directory containing the .mmp(z) file. If you add a stock sample or anything from the outside, it gets copied in. So, only relative paths, and forget about anything up the tree from the project.

• Something like this is mandated by NSM (Add NSM support #525), if that is ever to be supported
• Want to archive a project? Just zip its directory! At some point, LMMS could even read those zipped projects straight from the file...
• Collaboration through whatever you choose gets easier: git, a shared network directory, USB stick.

try relative first and fall back onto the absolute path if the relative path no longer points to a valid file?

In my preferred model, that's unlikely to happen unless the user goes poking around manually where it shouldn't.

[unfa]

Ok, but why not allows paths to folders up the tree (../../../)?

[musikBear]

am i missing something? i thought it was agreed that the exporting should include a 'export.to-zipped-folder' option, where lmms would collect all and store it in a 'ready to go' zip folder
Anyone else remember that discussion?
could that have been a dream ..naaa :p

[unfa]

Ok. Exporting to a ZIP sounds nice.

I use Syncthing for collaboration (a bit like Dropbox, more like Bittorent Sync, only opensource and cooler) and relative paths with ../ would be cool.

[softrabbit]

I use Syncthing for collaboration (a bit like Dropbox, more like Bittorent Sync, only opensource and cooler) and relative paths with ../ would be cool.

Don't your collaborators get at least a bit annoyed if you use a file that's outside the shared folders?

[unfa]

We'd share the whole tree.

A is the shared folder - the root of the collaboartion tree:

A/tracks/1.mmpz
A/tracks/2.mmpz
A/tracks/3.mmpz
A/samples/1.flac
A/samples/2.flac
A/samples/3.flac

[tresf]

Duplicate of #2982.