Chained metapackages

[taldcroft]

@jeanconn - Yea or Nea? This will make a big difference in overall planning, e.g. #16 etc.

[jeanconn]

My first test of this broke and I don't know why (trouble searching for the first package), but the second test worked. So this is probably fine. My opinion is that it is more awkward for maintenance but if you like it for the one-package cmd line, that's fine.

So the update process for a package in ska3-core would be:

• edit ska3-core/meta.yaml with updated individual package version and updated ska3-core version
• build ska3-core
• edit ska3-flight/meta.yaml with updated ska3-core version and updated ska3-flight version
• build ska3-flight

I don't think it makes a huge difference for overall planning. If we end up with trouble with the package solver for chained metapackage, we'd just move from

conda install ska3-flight

to

conda install ska3-core ska3-flight

[taldcroft]

The update process you described seems not at all awkward and nicely rigorous and versioned to me. Basically you make a change to the core packages and commit that with an easily identified new version tag (e.g. version=2019.06.22), and then update ska3-flight with ska3-core =2019.06.22. This two-step dance happens once, maybe twice a year, and we get nice commits and visibility. The process will be documented and it's just a matter of turning the crank.

I would be very hesitant not have the explicit ska3-core package requirement in ska3-flight, because then one could easily (mistakenly) do conda install ska3-flight (without ska3-core) and get a non-flight environment. So without chained packages, we would need to glop everything into ska3-flight, which would really be annoying.

[jeanconn]

Disagreeing just a little bit in that you can mistakenly do a lot of things, but if we had to adopt the convention that to have a flight environment you'd install with the process of ska3-core ska3-flight that convention also doesn't seem particularly onerous.

The only flaw I really noticed in the process was that conda build would let me build ska3-flight even if ska3-core did not exist. It isn't checking meta packages on build the way that it does with real packages. No big deal when you go to install.

There's probably also an issue that I don't think the metapackage currently allows you to specify which channel provides a package. This is basically fine for us.

With regard to the checking.. that process does mean that I was wrong about unversioned packages actually getting versions specified in the metapackage (for ska-dev). I thought that real versions were fetched from the available packages and assigned in the built metapackage, but it looks like you just end up with a package list, and I think you are going to get the versions available in your channels when you install in your new environment. I'm not sure what the update process would/could look like in that case.

[taldcroft]

The convention is not onerous, but I think mistake prone and definitely more complicated than required (hopefully!).

I don't know what you mean by "unversioned packages ...", and I'm entirely lost on that whole paragraph. But if it deals with ska-dev, then it probably is not important, we can live without that.

[jeanconn]

And yes, that was ska-dev related. From "It is like ska3-flight but does not include the ska3-flight-core dependence nor any version numbers." in #16 I thought we would have a metapackage with "unversioned packages".

I thought they would be assigned versions at build time, but it looks like you just get what you get at installtime.