Docs: add the "How to explore the provenance graph" section #4491
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
giovannipizzi
requested changes
Oct 20, 2020
docs/source/howto/exploring.rst
Outdated
| Incoming and outgoing links | ||
| =========================== | ||
|
|
||
| The provenance graph in AiiDA is a :ref:`directed acyclic graph <topics:provenance:concepts>`. |
There was a problem hiding this comment.
Suggested change
| The provenance graph in AiiDA is a :ref:`directed acyclic graph <topics:provenance:concepts>`. | |
| The provenance graph in AiiDA is a :ref:`directed graph <topics:provenance:concepts>`. |
docs/source/howto/exploring.rst
Outdated
|
|
||
| The provenance graph in AiiDA is a :ref:`directed acyclic graph <topics:provenance:concepts>`. | ||
| The vertices of the graph are the *nodes* and the edges that connect them are called *links*. | ||
| Since the graph is directed, any node can have *incoming* and *outgoing* links that connects it to neighboring nodes. |
There was a problem hiding this comment.
Suggested change
| Since the graph is directed, any node can have *incoming* and *outgoing* links that connects it to neighboring nodes. | |
| Since the graph is directed, any node can have *incoming* and *outgoing* links that connect it to neighboring nodes. |
docs/source/howto/exploring.rst
Outdated
| Since the graph is directed, any node can have *incoming* and *outgoing* links that connects it to neighboring nodes. | ||
|
|
||
| To discover the neighbors of a given node, you can use the methods :meth:`~aiida.orm.nodes.node.Node.get_incoming` and :meth:`~aiida.orm.nodes.node.Node.get_outgoing`. | ||
| They have the exact same interface but will return the neighbors from which the links are incoming, or to which the links are pointing, respectively. |
There was a problem hiding this comment.
Suggested change
| They have the exact same interface but will return the neighbors from which the links are incoming, or to which the links are pointing, respectively. | |
| They have the exact same interface but will return the neighbors connected to the current node with link coming into it, or with links going out of it, respectively. |
I feel this is a bit more clear (I hope)
docs/source/howto/exploring.rst
Outdated
| The :meth:`~aiida.orm.nodes.node.Node.get_incoming` and :meth:`~aiida.orm.nodes.node.Node.get_outgoing` methods accept various arguments that allow one to filter what neighboring nodes should be matched: | ||
|
|
||
| * ``node_class``: accepts a subclass of :class:`~aiida.orm.nodes.node.Node`, only neighboring nodes with a class that matches this will be returned | ||
| * ``link_type``: accepts a vale of :class:`~aiida.common.links.LinkType`, only neighboring nodes that are linked with this link type will be returned |
There was a problem hiding this comment.
Suggested change
| * ``link_type``: accepts a vale of :class:`~aiida.common.links.LinkType`, only neighboring nodes that are linked with this link type will be returned | |
| * ``link_type``: accepts a value of :class:`~aiida.common.links.LinkType`, only neighboring nodes that are linked with this link type will be returned |
docs/source/howto/exploring.rst
Outdated
|
|
||
| * ``node_class``: accepts a subclass of :class:`~aiida.orm.nodes.node.Node`, only neighboring nodes with a class that matches this will be returned | ||
| * ``link_type``: accepts a vale of :class:`~aiida.common.links.LinkType`, only neighboring nodes that are linked with this link type will be returned | ||
| * ``link_label_filter``: accepts a string regex (with optional wildcards), only neighboring nodes that are linked with a link label that matches the regex will be returned |
There was a problem hiding this comment.
Suggested change
| * ``link_label_filter``: accepts a string regex (with optional wildcards), only neighboring nodes that are linked with a link label that matches the regex will be returned | |
| * ``link_label_filter``: accepts a string expression (with optional wildcards using the syntax of SQL `LIKE` patterns, see below), only neighboring nodes that are linked with a link label that matches the regex will be returned |
I think regex means a specific thing - we shouldn't say that these are regexes.
These are called patterns e.g. in the postgres docs
docs/source/howto/exploring.rst
Outdated
| outputs = process_node.outputs | ||
|
|
||
| These properties do not return the actual inputs and outputs directly, but instead return an instance of :class:`~aiida.orm.utils.managers.NodeLinksManager` | ||
| The reason is because through the manager, the inputs or outputs are accessible through their link label and it is tab-completed. |
There was a problem hiding this comment.
Suggested change
| The reason is because through the manager, the inputs or outputs are accessible through their link label and it is tab-completed. | |
| The reason is because through the manager, the inputs or outputs are accessible through their link label (that, for inputs and outputs of processes, is unique) and can be tab-completed. |
| .. note:: | ||
|
|
||
| The ``inputs`` and ``outputs`` properties are only defined for :class:`~aiida.orm.nodes.process.process.ProcessNode`'s. | ||
| This means that you cannot *chain* these calls, because an input or output of a process node is guaranteed to be a :class:`~aiida.orm.nodes.data.Data` node, which does not have inputs or outputs. |
There was a problem hiding this comment.
I would put this comment below, after explaining creator, to say that inputs+creator allows to go up in the provenance graph as much as you want, chaining them. Instead, going down, you cannot for the reason you mention
There was a problem hiding this comment.
Not sure where exactly you intend this to be placed. It is true that chaining creator and inputs alternatively works, but that is just one example and not every node has a creator. Instead, I kept this as is and added another note after the explanation of creator that gives the example you propose.
docs/source/howto/exploring.rst
Outdated
| * :meth:`~aiida.orm.nodes.process.process.ProcessNode.called`: defined for :class:`~aiida.orm.nodes.process.process.ProcessNode`'s and returns the list of process nodes called by this node. | ||
| If this process node did not call any other processes, this property returns an empty list. | ||
| * :meth:`~aiida.orm.nodes.process.process.ProcessNode.caller`: defined for :class:`~aiida.orm.nodes.process.process.ProcessNode`'s and returns the process node that called this node. | ||
| If this ndoe was not called by a process, this property returns ``None``. |
There was a problem hiding this comment.
Suggested change
| If this ndoe was not called by a process, this property returns ``None``. | |
| If this node was not called by a process, this property returns ``None``. |
|
|
||
| Similar to the ``inputs`` and ``outputs`` properties of process nodes, there are some more properties that make exploring the provenance graph easier: | ||
|
|
||
| * :meth:`~aiida.orm.nodes.process.process.ProcessNode.called`: defined for :class:`~aiida.orm.nodes.process.process.ProcessNode`'s and returns the list of process nodes called by this node. |
There was a problem hiding this comment.
Not relevant for this PR, but I realise now that we could have defined this only for WorkflowNodes, actually, since the list will always be empty for CalculationNodes
There was a problem hiding this comment.
Yeah, and I think the ProcessNode should also have a inputs and outputs implementation. Now I cannot reference it
| node.res.some_key | ||
|
|
||
| In an interactive shell, the available keys are also tab-completed. | ||
| If you type ``node.res.`` followed by the tab key twice, a list of the available keys is printed. |
There was a problem hiding this comment.
I would add that, if the default output node link label is output_parameters, then node.res.some_key is equivalent to node.outputs.output_parameters.dict.some_key and stress that we are looking at the keys of a different node (this, together with the fact that verdi node outputcat/outputls act on a different node, does not seem to be super intuitive at first for new users, in my experience)
This section adds completely new documentation on the graph exploring methods `get_incoming` and `get_outgoing` that were not yet documented. It also documents the `creator`, `caller` and `creator` properties that also weren't documented yet. The `working_with_aiida/resultmanager.rst` is removed and its content, that documents the `inputs` and `outputs` properties of process nodes as well as the `res` property of the `CalcJobNode` class, is added to the new section in heavily rewritten form.
sphuber
force-pushed
the
fix/4464/docs-exploring-provenance-graph
branch
from
3633cbf to
808b262
Compare
|
@giovannipizzi addressed all your comments |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #4464
This section adds completely new documentation on the graph exploring
methods
get_incomingandget_outgoingthat were not yet documented.It also documents the
creator,callerandcreatorproperties thatalso weren't documented yet.
The
working_with_aiida/resultmanager.rstis removed and its content,that documents the
inputsandoutputsproperties of process nodes aswell as the
resproperty of theCalcJobNodeclass, is added to thenew section in heavily rewritten form.