App Developers: Adding Automation to Existing Apps
Apps developed using the Cytoscape App Ladder or other paths add functionality to the Cytoscape GUI. App developers can now enable apps for use in scripting and external programming environments. The best approach for doing this depends on how the app code is structured and the intended user community.
This page is an overview describing trade-offs involving app code structure, target community and effort involved in enabling automation in an app.
The main choices focus on whether the app will expose Cytoscape Commands, Cytoscape Functions or both:
Cytoscape Commands are most suited to operations whose parameters have simple structure and are easily accessed via a script, though they can be accessed from a programming language.
Cytoscape Functions are best suited for complex parameters and are best accessed from a programming language.
Regardless of specific trade-offs, it is critical that apps provide excellent documentation that enables script writers to successfully call apps -- documentation should have very high priority in the development process. It is addressed on this page and in the App Developer Swagger Best Practices page.
For more information, see the Cytoscape Automation FAQ.
Adding automation via Commands or Functions can involve different trade-offs and considerations. The cost of implementing either method is subjective to the nature of the App and its code, as well as the individual developers competencies, preferences, and skills. We've outlined some key benefits and drawbacks that might influence automation implementation in the chart below:
Regardless of which method or combination of methods is used, the end result should always be an App with clearly documented automation that provides benefit to scripters and coders.
As we previously mentioned, there are many decisions to be made when considering the route to take in adding automation to an app, and these all depend on the unique situation that each app presents. To illustrate some of this process, we offer a case study of how automation was added to the Diffusion App.
The Diffusion app allows users of Cytoscape to interact with the Diffusion web service to analyze networks. The current code for this app resides in this GitHub repository.
For the purpose of studying how automation was added, we maintain two snapshots of the repository, which be can referenced if necessary with the following links:
Our automation first leverages existing Diffusion TaskFactories to create Cytoscape Commands and appropriate Swagger documentation. When limitations of Commands become apparent (e.g., limited choices for documentation, parameter and return value complexity, and granularity), we then work to expose Cytoscape Functions through explicit JAX-RS and Swagger annotations.
In the Cytoscape GUI, the primary functionality of the Diffusion App is accessed through the Diffuse sub-menu that appears when a popup menu is opened in a Cytoscape Graph View. The two submenu items, Selected Nodes and Selected Nodes with Options execute the necessary query to the Diffusion service, and when results are returned, a result panel is shown to aid in analyzing them.
We chose to automate the functionality of selecting either the Diffuse-->Selected Nodes or Diffuse-->Selected Nodes with Options menu items. These would add useful data to the graph, and their results could be passed on to the caller for further analysis in R or Python, going beyond the functionality offered by the output panel.
The original implementation of Diffusion exposed two TaskFactory implementations to perform our two operations: DiffuseSelectedTask.java and DiffuseSelectedWithOptionsTask. The App author had implemented these with Tunable annotations, making them relatively simple to add as Cytoscape Commands.
If your app doesn't implement TaskFactories/Tunables, you may want to consider refactoring your app to use them. You can gauge how expensive or appropriate this approach would be by examining the TaskFactory Sample App and comparing it to the process of adding Cytoscape Functions via JAX-RS used in the CyREST Basic Sample App.
We registered our two TaskFactories as Commands, adding them to CyREST Command API, as demonstrated in the code snippet below:
...
diffusionTaskFactoryProps.setProperty(COMMAND_NAMESPACE, "diffusion");
diffusionTaskFactoryProps.setProperty(COMMAND, "diffuse");
diffusionTaskFactoryProps.setProperty(COMMAND_DESCRIPTION, "Execute Diffusion on Selected Nodes");
...
wOptsProps.setProperty(COMMAND_NAMESPACE, "diffusion");
wOptsProps.setProperty(COMMAND, "diffuse_advanced");
wOptsProps.setProperty(COMMAND_DESCRIPTION, "Execute Diffusion with Options");
...This made our endpoint available to script writers via the Cytoscape Command Tool, and accessible via REST using the GET /v1/commands/diffusion/diffuse and GET /v1/commands/diffusion/diffuse_advanced paths.
For a more detailed explanation of adding automation with TaskFactories see the TaskFactory Sample App.
One limiting aspect of the previous implementation was that DiffuseSelectedTask and DiffuseSelectedWithOptionsTask offered no feedback to the Command Line or REST caller (e.g., the data produced by the each operation, which was saved to columns in the node table). To return this data, DiffuseSelectedTask was expanded to store the names of the new columns, and return them via the getResults(Class) method of the ObservableTask interface.
@Override
public <R> R getResults(Class<? extends R> type) {
if (type.equals(String.class))
{
return (R)(diffusionResultColumns != null ? "Created result columns: ("+diffusionResultColumns.heatColumn+"),("+diffusionResultColumns.rankColumn+")" : "No result columns available");
}
else if (type.isAssignableFrom(DiffusionResultColumns.class)){
return (R) diffusionResultColumns;
}
return null;
}Take note that a POJO class called DiffusionResultColumns is referenced, but is not accessible by Cytoscape's Command Line Dialog directly (only String results are returned in the Command Line Dialog). We will examine in detail in the Output Model section below why this was done.
Exposing our TaskFactories as Commands and offering usable feedback for users was a definite improvement, however, the documentation exposed to users was less informative than we wanted (or called for in the App Developer Swagger Best Practices page). The state of the documentation at this point is illustrated in the Swagger UI shown below (with deficiencies shown in red):
Note that the parameter descriptions are cryptic and there is no general description of the Command's function and use. Additionally, there is little description of the return result.
This is a critical deficiency and will likely lead the failure of script writers to effectively use this endpoint.
As described App Developer Swagger Best Practices page, users are best served by the kind of information presented in the highly successful *NIX man pages model, where every page has:
- Command/Function name
- Narrative describing intended use and context, and details of actual usage
- List of parameters and meanings
- Return results
To enable documentation improvements, we created Cytoscape Functions by making JAX-RS- and Swagger-annotated wrappers for both TaskFactories. This would allow us to provide documentation via Swagger, and to better structure the input parameters and output result. This wrapper and its supporting classes were created in the new org.cytoscape.diffusion.rest package. The DiffusionResource class provided the JAX-RS resource, DiffusionTaskObserver helped observe the execution of the existing tasks, and DiffusionParameters and the previously mentioned DiffusionResultColumns provided models for input parameters and output data.
The code snippet below shows the method responsible for most of the functionality of DiffusionResource:
@POST
@Produces("application/json")
@Consumes("application/json")
@Path("{networkSUID}/views/{networkViewSUID}/diffuse_with_options")
@ApiOperation(value = "Execute Diffusion Analysis on a Specific Network View with Options",
notes = GENERIC_SWAGGER_NOTES, //These notes are repeated throughout our Swagger, so are defined in a static variable.
response = DiffusionAppResponse.class)
@ApiResponses(value = {
@ApiResponse(code = 404, message = "Network does not exist", response = CIResponse.class)
})
public Response diffuseWithOptions(
@ApiParam(value="Network SUID (see GET /v1/networks)") @PathParam("networkSUID") long networkSUID, @ApiParam(value="Network View SUID (see GET /v1/networks/{networkId}/views)") @PathParam("networkViewSUID") long networkViewSUID,
@ApiParam(value = "Diffusion Parameters", required = true) DiffusionParameters diffusionParameters) {
... // Find the current CyNetworkView
DiffusionTaskObserver taskObserver = new DiffusionTaskObserver(this, "diffuse_with_options", TASK_EXECUTION_ERROR_CODE);
... // Create the tunableMap passed to the TaskManager
TaskIterator taskIterator = diffusionWithOptionsTaskFactory.createTaskIterator(cyNetworkView);
taskManager.setExecutionContext(tunableMap);
taskManager.execute(taskIterator, taskObserver);
return Response.status(taskObserver.response.errors.size() == 0 ? Response.Status.OK : Response.Status.INTERNAL_SERVER_ERROR)
.type(MediaType.APPLICATION_JSON)
.entity(taskObserver.response).build();
}The result of this code is the Swagger UI that is rendered below (with guidance in blue):
With better narrative, parameter explanation and function result content, this version much better exhibits best practices.
Functionally, the diffuseWithOptions(...) method creates data to pass to the TaskFactory Tunables and then returns the generated results. The Input and Output Models, defined by the DiffusionParameters and DiffusionResultColumns, are used by CyREST to generate the JSON message and response bodies for the function.
Below, we will examine the various annotations and models in the code above and see how they were used to create our function and its documentation. For a more detailed explanation of adding automation with JAX-RS and Swagger see the CyREST Basic and CyREST Best Practices Sample Apps.
The annotations in the code snippet below are JAX-RS annotations responsible for setting the HTTP method for the operation, defining its media types, as well as the path by which it is to be accessed.
@Path("/diffusion/v1/")
public class DiffusionResource {
...
}@POST
@Produces("application/json")
@Consumes("application/json")
@Path("{networkSUID}/views/{networkViewSUID}/diffuse_with_options")All of these annotations are very thoroughly explained in this section of the Building RESTful Web Services with JAX-RS tutorial. We will briefly explain how each of them applies to our implementation.
@POST indicates that CyREST should treat this method as an HTTP POST, meaning that it creates data in the underlying model. In this specific case, columns and data are generated.
@Produces indicates that CyREST should translate the return type of this method to JSON.
@Consumes indicates that CyREST should translate a message body from JSON into the relevant body parameter. In this specific case, the message body is translated to a DiffusionParameters object.
@Path annotations define the path via which CyREST will access a method. Note that there are two places in which path is applied: for the enclosing class, and for individual methods. The full path is always generated by placing the class path before the method path. In our case, our class path /diffusion/v1/ is combined with our method path {networkSUID}/views/{networkViewSUID}/diffuse_with_options to effectively make the path /diffusion/v1/{networkSUID}/views/{networkViewSUID}/diffuse_with_options. All paths in the Diffusion App were chosen by following the Cytoscape Function Best Practices guidelines for App Resource Paths, using the unique and versioned root /diffusion/v1/.
Any parts of the path enclosed in braces, such as {networkSUID} and {networkViewSUID}, are treated as placeholders for parameters. For example, calling this operation from a client for a network with SUID 52, and a network view with SUID 770 would require a POST request to a URL like the following: http://localhost:1234/diffusion/v1/52/views/770/diffuse_with_options
The annotations in the code snippets below are responsible for assigning a short summary of the operation as well as implementation notes to be included in the Swagger UI.
public static final String GENERIC_SWAGGER_NOTES = "Diffusion will send the selected network view and its selected nodes to "
+ "a web-based REST service to calculate network propagation. Results are returned and represented by columns "
+ "in the node table." + '\n' + '\n'
+ "Columns are created for each execution of Diffusion and their names are returned in the response." + '\n' + '\n';@ApiOperation(value = "Execute Diffusion Analysis on a Specific Network View with Options",
notes = GENERIC_SWAGGER_NOTES, //These notes are repeated throughout our Swagger, so are defined in a static variable.
...)The resulting section of the Swagger UI is below.
The annotations in the code below are responsible for setting several input parameters for our operation:
...
public Response diffuseWithOptions(
@ApiParam(value="Network SUID (see GET /v1/networks)") @PathParam("networkSUID") long networkSUID, @ApiParam(value="Network View SUID (see GET /v1/networks/{networkId}/views)") @PathParam("networkViewSUID") long networkViewSUID,
@ApiParam(value = "Diffusion Parameters", required = true) DiffusionParameters diffusionParameters)networkSUID and networkViewSUID are automatically derived from the path of the request, which is defined by @Path("{networkSUID}/views/{networkViewSUID}/diffuse_with_options"), as previously explained.
@ApiParam(value = "Diffusion Parameters", required = true) DiffusionParameters diffusionParameters is not identified as a path parameter, which means that CyREST will translate the message body from JSON to a DiffusionParameters object, and pass it as the diffusionParameters parameter. The DiffusionParameters class therefore defines a model for input.
The code snippet below is from the DiffusionParameters class, and demonstrates all the annotations that build the Swagger documentation for the input model:
@ApiModel(value="Diffusion Parameters", description="Parameters for Diffusion analysis")
public class DiffusionParameters {
@ApiModelProperty(value = "A node column name intended to override the default table column 'diffusion_input'. This represents the query vector and corresponds to h in the diffusion equation.", example=DiffuseSelectedTask.DIFFUSION_INPUT_COL_NAME)
public String heatColumnName;
@ApiModelProperty(value = "The extent of spread over the network. This corresponds to t in the diffusion equation.", example="0.1")
public Double time;
}This code enables CyREST to construct a JSON structure corresponding to the input model below:
{
"heatColumnName": "diffusion_input",
"time": 0.1
}In the Swagger UI, this combination of input parameters will be presented in the following section:
Similarly to how DiffusionParameters defines the input model, the taskObserver.response value is a CIResponse object, which is defined in the CIResponse section of the Cytoscape Function Best Practices. The CIResponse contains a DiffusionResultColumns object in the data field or, if an error occurred, a list of errors in the errors field. We define the DiffusionResultColumns model in the code snippet below:
@ApiModel(value="Diffusion Result Columns")
public class DiffusionResultColumns {
@ApiModelProperty(value = "The node column containing the result of the diffusion process, corresponding to d in the diffusion equation.", required=true, example="diffusion_output_heat")
public String heatColumn;
@ApiModelProperty(value = "The node column containing rank according to output heat. This column is recommended for analysis, as it is very robust to parameter choice.", required=true, example="diffusion_output_rank")
public String rankColumn;
}The JSON corresponding to DiffusionResultColumns will look like the example below:
{
"heatColumn": "diffusion_output_heat",
"rankColumn": "diffusion_output_rank"
}Before the above JSON reaches the caller, the DiffusionResultColumn JSON will be enclosed in the data field of a CIResponse object, resulting in the final form of the JSON shown in the example below:
{
"data": {
"heatColumn": "diffusion_output_heat",
"rankColumn": "diffusion_output_rank"
},
"errors": []
}In the Swagger UI, this will be presented in the following section:
Had the Function call encountered an error, such as a Cytoscape session with no current view, no data value would have been returned, but the cause of the error would have been included in the errors list similarly to the response below:
{
"data": {},
"errors": [
{
"status": 404,
"type": "urn:cytoscape:ci:diffusion-app:v1:diffuse_current_view_with_options:1",
"message": "Could not find current Network",
"link": "file:/Users/eripley/CytoscapeConfiguration/3/framework-cytoscape.log"
}
]
}Note that the error message for is human readable, and that the error type URN is clearly bound to the particular app, app version, and operation. This follows best practices for using errors in the CIResponse structure, which are defined on the Cytoscape Function Best Practices page.
There are other supporting classes methods in the org.cytoscape.diffusion.internal.rest package, but examining these are left as an optional exercise for interested developers.
The results of these changes were a set of REST operations that benefited scripters and coders, and had well defined and documented functionality, parameters, and output.