RFC: improving glob matching

[istudyatuni]

Current limitations

Tested on rustic v0.7.0

1. You should specify nested folders separately, e.g.

   [[backup.sources]]
   source = "/dir"
   glob = [
       "!*",
       "/dir/a", # without this line it won't work
       "/dir/a/**/*",
   ]

2. File matching is affected by order of the glob patterns in config, but should be based on globs precedence. For example, this excludes a/b.txt:

   [[backup.sources]]
   source = "/dir"
   glob = [
       "!*",
       "/dir/a",
       "/dir/a/**/*",
       "!/dir/a/b.txt",
   ]

   but this don't:

   [[backup.sources]]
   source = "/dir"
   glob = [
       "!*",
       "/dir/a",
       "!/dir/a/b.txt",
       "/dir/a/**/*",
   ]

3. Relative paths don't match files, only folders, e.g. this won't backup files, only folders:

   [[backup.sources]]
   source = "/dir"
   glob = [
       "!*",
       "a",
       "a/**/*",
   ]

4. Specifying folder without /**/* backups only this folder and not any of its content, but this is useless and inconsistent with the fact that when glob isn't specified, the source is backed up recursively.

5. More?

Requirements of the new implementation

1. Relative paths works properly.

2. You don't need to specify nested folders separately.

3. When specifying folder, it's content included recursively (but you can specify this explicitly, so "a" == "a/**/*").

4. If a file/subfolder is specified separately, it takes precedence over the specified parent folder, regardless of its position in the list.

5. It's possible to include only files, but not subfolders, like

   glob = [
       "a/*",
       "!a/**/*"
   ]

   or

   glob = [
       "a/*",
       "!a/**/"
   ]

   or even just

   glob = [
       "a/*"
   ]

6. More?

Why new implementation?

I'm not sure that walkdir can give such flexibility.

Backward compatibility

TODO

[istudyatuni]

I wrote a workaround for this for nix configuration. Link to lines where main work done. Basic idea:

• For files and directories add additional nesting directories, like:
  For source /home/user and file /home/user/.config/rustic/rustic.toml (specified as (dir ".config/rustic/rustic.toml")) also add directories

  • /home/user/.config
  • /home/user/.config/rustic

• For folders in include additionally append /**/* to full directory

• Sort, so order will be

  • /home/user/.config
  • /home/user/.config/rustic
  • /home/user/.config/rustic/rustic.toml
  • !/home/user/.config/rustic/other.toml

• Deduplicate