Support optional merge function during copy

[dazuma]

Feature request for synthtool. During file copy, allows the result to be a "merge" of the contents of the source and the existing destination file. The merge function itself is provided as an optional argument to transforms.move.

One use case supported by this feature the ruby gemspec. This file specifies, among other things, the gem version, and the library dependencies. The former is updated during the release process (i.e. GAPIC does not know what version the gem currently is, so it cannot generate that part of the file correctly). The latter should be controlled by GAPIC (e.g. only GAPIC knows what version of google-gax its generated code depends on). Thus, the final gemspec output by synth must include the current version from the existing gemspec, and the dependency information from the freshly gapic-generated gemspec.

Open question: currently the merge function takes two arguments: the contents of the source (i.e. from-gapic) file, and the contents of the existing destination file. This is sufficient for my current use case, in which I am issuing a transforms.copy for a single file. However, maybe the merge function should also be passed the file path, so it has more information if it is invoked during a directory copy that includes multiple files? (And if so, should it be passed the source file path or the destination file path, or both?)

[theacodes]

However, maybe the merge function should also be passed the file path, so it has more information if it is invoked during a directory copy that includes multiple files? (And if so, should it be passed the source file path or the destination file path, or both?)

The path should be the same relative path, right?

[dazuma]

Yes, it should be the same relative path. Although I'm not entirely clear how to get the "common" relative path.

[theacodes]

I think you can use the _tracked_paths wizardry: https://github.com/GoogleCloudPlatform/synthtool/pull/21/files#diff-cb9e5e9cfb56e43da6d7c82738c747c2L116

Another option would be to compute all of the common parts between the src and dst (.parts works well for this).

[dazuma]

I decided to pass the (canonical) destination path as-is to the merge function, since it is the actual file being operated on. By default it's already a relative path anyway.

[theacodes]

@dazuma I made a few changes and I think everything still works, can you confirm?

[dazuma]

@theacodes Yes, my synth using the new feature still works. Thanks for the fixes. Sometime can you point me to how to run the linter/typechecker/whatever tools you're using? I totally missed a bunch of those type issues.

[theacodes]

nox -s lint, you'll need to pip install nox-automation first.

[JustinBeckwith]

👋 Sorry for getting to this late. @dazuma can you make sure this is documented in the README?