Add an 'out_dir' option to jsonnet_to_json() (#184)

If jsonnet_to_json() yields a large number of output files, or yields a
set of paths that is not known up front, it would be preferable to store
these in a directory artifact (i.e., created using
ctx.actions.declare_directory()).

This change adds a new 'out_dir' option that causes the interpreter to
be invoked with the -m flag. The -m flag will point to a single
directory.

docs/jsonnet.md

@@ -97,7 +97,7 @@ Example:
 <pre>
 jsonnet_to_json(<a href="#jsonnet_to_json-name">name</a>, <a href="#jsonnet_to_json-deps">deps</a>, <a href="#jsonnet_to_json-src">src</a>, <a href="#jsonnet_to_json-data">data</a>, <a href="#jsonnet_to_json-outs">outs</a>, <a href="#jsonnet_to_json-code_vars">code_vars</a>, <a href="#jsonnet_to_json-ext_code">ext_code</a>, <a href="#jsonnet_to_json-ext_code_envs">ext_code_envs</a>, <a href="#jsonnet_to_json-ext_code_file_vars">ext_code_file_vars</a>,
                 <a href="#jsonnet_to_json-ext_code_files">ext_code_files</a>, <a href="#jsonnet_to_json-ext_str_envs">ext_str_envs</a>, <a href="#jsonnet_to_json-ext_str_file_vars">ext_str_file_vars</a>, <a href="#jsonnet_to_json-ext_str_files">ext_str_files</a>, <a href="#jsonnet_to_json-ext_strs">ext_strs</a>, <a href="#jsonnet_to_json-extra_args">extra_args</a>,
-                <a href="#jsonnet_to_json-imports">imports</a>, <a href="#jsonnet_to_json-jsonnet">jsonnet</a>, <a href="#jsonnet_to_json-multiple_outputs">multiple_outputs</a>, <a href="#jsonnet_to_json-stamp_keys">stamp_keys</a>, <a href="#jsonnet_to_json-tla_code">tla_code</a>, <a href="#jsonnet_to_json-tla_code_envs">tla_code_envs</a>,
+                <a href="#jsonnet_to_json-imports">imports</a>, <a href="#jsonnet_to_json-jsonnet">jsonnet</a>, <a href="#jsonnet_to_json-multiple_outputs">multiple_outputs</a>, <a href="#jsonnet_to_json-out_dir">out_dir</a>, <a href="#jsonnet_to_json-stamp_keys">stamp_keys</a>, <a href="#jsonnet_to_json-tla_code">tla_code</a>, <a href="#jsonnet_to_json-tla_code_envs">tla_code_envs</a>,
                 <a href="#jsonnet_to_json-tla_code_files">tla_code_files</a>, <a href="#jsonnet_to_json-tla_str_envs">tla_str_envs</a>, <a href="#jsonnet_to_json-tla_str_files">tla_str_files</a>, <a href="#jsonnet_to_json-tla_strs">tla_strs</a>, <a href="#jsonnet_to_json-vars">vars</a>, <a href="#jsonnet_to_json-yaml_stream">yaml_stream</a>)
 </pre>
 
@@ -215,7 +215,7 @@ Example:
 | <a id="jsonnet_to_json-deps"></a>deps |  List of targets that are required by the `srcs` Jsonnet files.   | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional |  `[]`  |
 | <a id="jsonnet_to_json-src"></a>src |  The `.jsonnet` file to convert to JSON.   | <a href="https://bazel.build/concepts/labels">Label</a> | optional |  `None`  |
 | <a id="jsonnet_to_json-data"></a>data |  -   | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional |  `[]`  |
-| <a id="jsonnet_to_json-outs"></a>outs |  Names of the output `.json` files to be generated by this rule.<br><br>If you are generating only a single JSON file and are not using jsonnet multiple output files, then this attribute should only contain the file name of the JSON file you are generating.<br><br>If you are generating multiple JSON files using jsonnet multiple file output (`jsonnet -m`), then list the file names of all the JSON files to be generated. The file names specified here must match the file names specified in your `src` Jsonnet file.<br><br>For the case where multiple file output is used but only for generating one output file, set the `multiple_outputs` attribute to 1 to explicitly enable the `-m` flag for multiple file output.   | List of labels | required |  |
+| <a id="jsonnet_to_json-outs"></a>outs |  Names of the output `.json` files to be generated by this rule.<br><br>If you are generating only a single JSON file and are not using jsonnet multiple output files, then this attribute should only contain the file name of the JSON file you are generating.<br><br>If you are generating multiple JSON files using jsonnet multiple file output (`jsonnet -m`), then list the file names of all the JSON files to be generated. The file names specified here must match the file names specified in your `src` Jsonnet file.<br><br>For the case where multiple file output is used but only for generating one output file, set the `multiple_outputs` attribute to 1 to explicitly enable the `-m` flag for multiple file output.<br><br>This attribute is incompatible with `out_dir`.   | List of labels | optional |  `[]`  |
 | <a id="jsonnet_to_json-code_vars"></a>code_vars |  Deprecated (use 'ext_code').   | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional |  `{}`  |
 | <a id="jsonnet_to_json-ext_code"></a>ext_code |  -   | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional |  `{}`  |
 | <a id="jsonnet_to_json-ext_code_envs"></a>ext_code_envs |  -   | List of strings | optional |  `[]`  |
@@ -228,7 +228,8 @@ Example:
 | <a id="jsonnet_to_json-extra_args"></a>extra_args |  -   | List of strings | optional |  `[]`  |
 | <a id="jsonnet_to_json-imports"></a>imports |  List of import `-J` flags to be passed to the `jsonnet` compiler.   | List of strings | optional |  `[]`  |
 | <a id="jsonnet_to_json-jsonnet"></a>jsonnet |  A jsonnet binary   | <a href="https://bazel.build/concepts/labels">Label</a> | optional |  `"@rules_jsonnet//jsonnet:jsonnet_tool"`  |
-| <a id="jsonnet_to_json-multiple_outputs"></a>multiple_outputs |  Set to `True` to explicitly enable multiple file output via the `jsonnet -m` flag.<br><br>This is used for the case where multiple file output is used but only for generating a single output file. For example:<br><br><pre><code>local foo = import "foo.jsonnet";&#10;&#10;{&#10;    "foo.json": foo,&#10;}</code></pre>   | Boolean | optional |  `False`  |
+| <a id="jsonnet_to_json-multiple_outputs"></a>multiple_outputs |  Set to `True` to explicitly enable multiple file output via the `jsonnet -m` flag.<br><br>This is used for the case where multiple file output is used but only for generating a single output file. For example:<br><br><pre><code>local foo = import "foo.jsonnet";&#10;&#10;{&#10;    "foo.json": foo,&#10;}</code></pre><br><br>This attribute is incompatible with `out_dir`.   | Boolean | optional |  `False`  |
+| <a id="jsonnet_to_json-out_dir"></a>out_dir |  Name of the directory where output files are stored.<br><br>If the names of output files are not known up front, this option can be used to write all output files to a single directory artifact. Files in this directory cannot be referenced individually.<br><br>This attribute is incompatible with `outs` and `multiple_outputs`.   | String | optional |  `""`  |
 | <a id="jsonnet_to_json-stamp_keys"></a>stamp_keys |  -   | List of strings | optional |  `[]`  |
 | <a id="jsonnet_to_json-tla_code"></a>tla_code |  -   | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional |  `{}`  |
 | <a id="jsonnet_to_json-tla_code_envs"></a>tla_code_envs |  -   | List of strings | optional |  `[]`  |

examples/BUILD

@@ -267,3 +267,9 @@ jsonnet_to_json_test(
         "@other_module//:world",
     ],
 )
+
+jsonnet_to_json(
+    name = "out_dir",
+    src = "out_dir.jsonnet",
+    out_dir = "out_dir.output",
+)

examples/out_dir.jsonnet

@@ -0,0 +1,4 @@
+{
+  'hello.txt': 'Hello, Bazel!',
+  'goodbye.txt': 'Goodbye, Bazel!',
+}

jsonnet/jsonnet.bzl

@@ -251,10 +251,20 @@ def _jsonnet_to_json_impl(ctx):
 
     outputs = []
 
-    # If multiple_outputs is set to true, then jsonnet will be invoked with the
-    # -m flag for multiple outputs. Otherwise, jsonnet will write the resulting
-    # JSON to stdout, which is redirected into a single JSON output file.
-    if len(ctx.attr.outs) > 1 or ctx.attr.multiple_outputs:
+    if (ctx.attr.outs or ctx.attr.multiple_outputs) and ctx.attr.out_dir:
+        fail("The \"outs\" and \"multiple_outputs\" attributes are " +
+             "incompatible with \"out_dir\".")
+
+    # If multiple_outputs or out_dir is set, then jsonnet will be
+    # invoked with the -m flag for multiple outputs. Otherwise, jsonnet
+    # will write the resulting JSON to stdout, which is redirected into
+    # a single JSON output file.
+    if ctx.attr.out_dir:
+        output_manifest = ctx.actions.declare_file("_%s_outs.mf" % ctx.label.name)
+        out_dir = ctx.actions.declare_directory(ctx.attr.out_dir)
+        outputs += [out_dir, output_manifest]
+        command += [ctx.file.src.path, "-c", "-m", out_dir.path, "-o", output_manifest.path]
+    elif len(ctx.attr.outs) > 1 or ctx.attr.multiple_outputs:
         output_manifest = ctx.actions.declare_file("_%s_outs.mf" % ctx.label.name)
         outputs += ctx.outputs.outs + [output_manifest]
         command += ["-m", ctx.outputs.outs[0].dirname, ctx.file.src.path, "-o", output_manifest.path]
@@ -300,6 +310,12 @@ def _jsonnet_to_json_impl(ctx):
         progress_message = "Compiling Jsonnet to JSON for " + ctx.label.name,
     )
 
+    if ctx.attr.out_dir:
+        return [DefaultInfo(
+            files = depset([out_dir]),
+            runfiles = ctx.runfiles(files = [out_dir]),
+        )]
+
 _EXIT_CODE_COMPARE_COMMAND = """
 EXIT_CODE=$?
 EXPECTED_EXIT_CODE=%d
@@ -594,8 +610,9 @@ specified in your `src` Jsonnet file.
 For the case where multiple file output is used but only for generating one
 output file, set the `multiple_outputs` attribute to 1 to explicitly enable
 the `-m` flag for multiple file output.
+
+This attribute is incompatible with `out_dir`.
 """,
-        mandatory = True,
     ),
     "multiple_outputs": attr.bool(
         doc = """\
@@ -611,6 +628,19 @@ local foo = import "foo.jsonnet";
     "foo.json": foo,
 }
 ```
+
+This attribute is incompatible with `out_dir`.
+""",
+    ),
+    "out_dir": attr.string(
+        doc = """\
+Name of the directory where output files are stored.
+
+If the names of output files are not known up front, this option can be
+used to write all output files to a single directory artifact. Files in
+this directory cannot be referenced individually.
+
+This attribute is incompatible with `outs` and `multiple_outputs`.
 """,
     ),
 }