-
Notifications
You must be signed in to change notification settings - Fork 989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add json output to info command (both console and file) #4359
Changes from 7 commits
45ca72b
8af3862
46d2a93
102566e
7827f29
432cf2d
9c8c785
cf3ed89
325f12d
c5a9084
50bd677
8c17b8e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -475,8 +475,7 @@ def info(self, *args): | |
"specify both install-folder and any setting/option " | ||
"it will raise an error.") | ||
parser.add_argument("-j", "--json", nargs='?', const="1", type=str, | ||
help='Only with --build-order option, return the information in a json.' | ||
' e.g --json=/path/to/filename.json or --json to output the json') | ||
help='Path to a json file where the information will be written') | ||
jgsogo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
parser.add_argument("-n", "--only", nargs=1, action=Extender, | ||
help="Show only the specified fields: %s. '--paths' information can " | ||
"also be filtered with options %s. Use '--only None' to show only " | ||
|
@@ -485,8 +484,8 @@ def info(self, *args): | |
help='Print information only for packages that match the filter pattern' | ||
' e.g., MyPackage/1.2@user/channel or MyPackage*') | ||
|
||
dry_build_help = ("Apply the --build argument to output the information, as it would be done " | ||
"by the install command") | ||
dry_build_help = ("Apply the --build argument to output the information, as it would be done" | ||
" by the install command") | ||
parser.add_argument("-db", "--dry-build", action=Extender, nargs="?", help=dry_build_help) | ||
build_help = ("Given a build policy, return an ordered list of packages that would be built" | ||
" from sources during the install command") | ||
|
@@ -526,7 +525,11 @@ def info(self, *args): | |
remote_name=args.remote, | ||
check_updates=args.update, | ||
install_folder=args.install_folder) | ||
self._outputer.nodes_to_build(nodes) | ||
if args.json: | ||
json_arg = True if args.json == "1" else args.json | ||
self._outputer.json_nodes_to_build(nodes, json_arg, get_cwd()) | ||
else: | ||
self._outputer.nodes_to_build(nodes) | ||
|
||
# INFO ABOUT DEPS OF CURRENT PROJECT OR REFERENCE | ||
else: | ||
|
@@ -553,6 +556,9 @@ def info(self, *args): | |
|
||
if args.graph: | ||
self._outputer.info_graph(args.graph, deps_graph, get_cwd()) | ||
elif args.json: | ||
jgsogo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
json_arg = True if args.json == "1" else args.json | ||
self._outputer.json_info(deps_graph, json_arg, get_cwd(), show_paths=args.paths) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we could output always the paths when json output and forbid the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For packages installed as editable access to paths raises. There are two options:
|
||
else: | ||
self._outputer.info(deps_graph, only, args.package_filter, args.paths) | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,17 @@ | ||
import json | ||
import os | ||
from collections import OrderedDict | ||
|
||
from conans.client.graph.graph import RECIPE_CONSUMER, RECIPE_VIRTUAL | ||
from conans.client.graph.graph import RECIPE_EDITABLE | ||
from conans.client.installer import build_id | ||
from conans.client.printer import Printer | ||
from conans.model.ref import ConanFileReference, PackageReference | ||
from conans.paths.simple_paths import SimplePaths | ||
from conans.search.binary_html_table import html_binary_graph | ||
from conans.unicode import get_cwd | ||
from conans.util.env_reader import get_env | ||
from conans.util.files import save | ||
from conans.client.graph.graph import RECIPE_EDITABLE | ||
|
||
|
||
class CommandOutputer(object): | ||
|
@@ -82,13 +86,120 @@ def _read_dates(self, deps_graph): | |
def nodes_to_build(self, nodes_to_build): | ||
self.user_io.out.info(", ".join(str(n) for n in nodes_to_build)) | ||
|
||
def _handle_json_output(self, data, json_output, cwd): | ||
json_str = json.dumps(data) | ||
|
||
if json_output is True: | ||
self.user_io.out.write(json_str) | ||
else: | ||
if not os.path.isabs(json_output): | ||
json_output = os.path.join(cwd, json_output) | ||
save(json_output, json.dumps(data)) | ||
self.user_io.out.writeln("") | ||
self.user_io.out.info("JSON file created at '%s'" % json_output) | ||
|
||
def json_nodes_to_build(self, nodes_to_build, json_output, cwd): | ||
data = [str(n) for n in nodes_to_build] | ||
self._handle_json_output(data, json_output, cwd) | ||
|
||
def _grab_info_data(self, deps_graph, grab_paths): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this "conversion function" doesn't belong to this module. A There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This class is created and called from outside the conan_api, the graph shouldn't have left the api. I'm sure that this json should be the output of the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree. Open the engineering issue, please. |
||
""" Convert 'deps_graph' into consumible information for json and cli """ | ||
compact_nodes = OrderedDict() | ||
for node in sorted(deps_graph.nodes): | ||
compact_nodes.setdefault((node.ref, node.conanfile.info.package_id()), []).append(node) | ||
|
||
ret = [] | ||
for (ref, package_id), list_nodes in compact_nodes.items(): | ||
node = list_nodes[0] | ||
if node.recipe == RECIPE_VIRTUAL: | ||
continue | ||
|
||
item_data = {} | ||
conanfile = node.conanfile | ||
if node.recipe == RECIPE_CONSUMER: | ||
ref = str(conanfile) | ||
|
||
item_data["reference"] = str(ref) | ||
item_data["is_ref"] = isinstance(ref, ConanFileReference) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I need this field in the JSON to perform some logic in the console output: will output There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ... if we do not consider breaking to remove this |
||
item_data["display_name"] = conanfile.display_name | ||
item_data["id"] = package_id | ||
item_data["build_id"] = build_id(conanfile) | ||
|
||
# Paths | ||
if isinstance(ref, ConanFileReference) and grab_paths: | ||
jgsogo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
item_data["export_folder"] = self.cache.export(ref) | ||
item_data["source_folder"] = self.cache.source(ref, conanfile.short_paths) | ||
if isinstance(self.cache, SimplePaths): | ||
# @todo: check if this is correct or if it must always be package_id() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You have the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👍 I can remove one line, the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The two calls to |
||
bid = build_id(conanfile) | ||
if not bid: | ||
bid = conanfile.info.package_id() | ||
pref = PackageReference(ref, bid) | ||
item_data["build_folder"] = self.cache.build(pref, conanfile.short_paths) | ||
|
||
package_id = conanfile.info.package_id() | ||
pref = PackageReference(ref, package_id) | ||
item_data["package_folder"] = self.cache.package(pref, conanfile.short_paths) | ||
|
||
try: | ||
reg_remote = self.cache.registry.refs.get(ref) | ||
if reg_remote: | ||
item_data["remote"] = {"name": reg_remote.name, "url": reg_remote.url} | ||
except: | ||
pass | ||
|
||
def _add_if_exists(attrib, as_list=False): | ||
value = getattr(conanfile, attrib, None) | ||
if value: | ||
if not as_list: | ||
item_data[attrib] = value | ||
else: | ||
item_data[attrib] = list(value) if isinstance(value, (list, tuple, set)) \ | ||
else [value, ] | ||
|
||
_add_if_exists("url") | ||
_add_if_exists("homepage") | ||
_add_if_exists("license", as_list=True) | ||
_add_if_exists("author") | ||
_add_if_exists("topics", as_list=True) | ||
|
||
if isinstance(ref, ConanFileReference): | ||
item_data["recipe"] = node.recipe | ||
|
||
if get_env("CONAN_CLIENT_REVISIONS_ENABLED", False) and node.ref.revision: | ||
item_data["revision"] = node.ref.revision | ||
|
||
item_data["binary"] = node.binary | ||
if node.binary_remote: | ||
item_data["binary_remote"] = node.binary_remote.name | ||
|
||
node_times = self._read_dates(deps_graph) | ||
if node_times and node_times.get(ref, None): | ||
item_data["creation_date"] = node_times.get(ref, None) | ||
|
||
if isinstance(ref, ConanFileReference): | ||
dependants = [n for node in list_nodes for n in node.inverse_neighbors()] | ||
required = [d.conanfile for d in dependants if d.recipe != RECIPE_VIRTUAL] | ||
if required: | ||
item_data["required_by"] = [d.display_name for d in required] | ||
|
||
depends = node.neighbors() | ||
requires = [d for d in depends if not d.build_require] | ||
build_requires = [d for d in depends if d.build_require] | ||
|
||
if requires: | ||
item_data["requires"] = [repr(d.ref) for d in requires] | ||
|
||
if build_requires: | ||
item_data["build_requires"] = [repr(d.ref) for d in build_requires] | ||
|
||
ret.append(item_data) | ||
|
||
return ret | ||
|
||
def info(self, deps_graph, only, package_filter, show_paths): | ||
registry = self.cache.registry | ||
Printer(self.user_io.out).print_info(deps_graph, | ||
only, registry, | ||
node_times=self._read_dates(deps_graph), | ||
path_resolver=self.cache, | ||
package_filter=package_filter, | ||
data = self._grab_info_data(deps_graph, grab_paths=show_paths) | ||
Printer(self.user_io.out).print_info(data, only, package_filter=package_filter, | ||
show_paths=show_paths) | ||
|
||
def info_graph(self, graph_filename, deps_graph, cwd): | ||
|
@@ -104,6 +215,10 @@ def info_graph(self, graph_filename, deps_graph, cwd): | |
graph_filename = os.path.join(cwd, graph_filename) | ||
grapher.graph_file(graph_filename) | ||
|
||
def json_info(self, deps_graph, json_output, cwd, show_paths): | ||
data = self._grab_info_data(deps_graph, grab_paths=show_paths) | ||
self._handle_json_output(data, json_output, cwd) | ||
|
||
def print_search_references(self, search_info, pattern, raw, all_remotes_search): | ||
printer = Printer(self.user_io.out) | ||
printer.print_search_recipes(search_info, pattern, raw, all_remotes_search) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've intentionally removed the mention to
--json
(without value) will output to console, I think it is something we don't want to promote. Shall I revert it?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not promote? isn't such text interface used as a base for unix command-line pipelining, e.g.
conan --json | some_tool | some_other_tool
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We cannot make this commitment right now, there are some commands that are outputting a lot of information to the console and then generating the json data (like the
info
one with the profiles information), so you cannot pipe the output to the next tool.It would be a nice-to-have, it will require to silent all the output and then just print the JSON data. Something to do, not so hard, but not yet.
More issues related to output: #4225 (what we are talking here), and two more issues also worth reading #4215, #4067
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure we can just remove this despite the fact that I don't like it as it is either. We will need to discuss it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm talking about removing it just from the
--help
output.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, it is fine for me