Skip to content

Commit

Permalink
Merge pull request #488 from rctauber/prefixes
Browse files Browse the repository at this point in the history
Add prefixes to output with --add-prefix and --add-prefixes
  • Loading branch information
jamesaoverton authored Jun 5, 2019
2 parents 2a7084e + 203173a commit 4d202cf
Show file tree
Hide file tree
Showing 7 changed files with 176 additions and 63 deletions.
7 changes: 7 additions & 0 deletions docs/examples/bar.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"@context" : {
"foo" : "http://foo#",
"bar" : "http://bar#",
"baz" : "http://baz#"
}
}
19 changes: 17 additions & 2 deletions docs/global.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Sometimes when working with very large ontologies, the JVM will run out of memor

#### Linux & MacOS

The Java options for ROBOT can be set by running `export ROBOT_JAVA_ARGS=<options>` prior to running a ROBOT command. This will only save the Java options in the current process space.
The Java options for ROBOT can be set by running `export ROBOT_JAVA_ARGS=<options>` prior to running a ROBOT command. This will only save the Java options in the current process space.

To set the Java options for ROBOT permanently, you will need to edit your `.bash_profile` and add the line `export ROBOT_JAVA_ARGS=<options>`. This is the same file you edited to add ROBOT to your system PATH, usually located in your root directory. You can verify that the variable is set by running `echo $ROBOT_JAVA_ARGS`.

Expand Down Expand Up @@ -58,11 +58,26 @@ The various prefix options can be used with any command. When chaining commands,
export-prefixes --prefix "bar: http://bar#" \
export-prefixes

The `--add-prefix` and `--add-prefixes` options allows you to specify prefix mappings in the same way as `--prefix` and `--prefixes`:

robot --noprefixes --add-prefixes foo.json \
--add-prefix "bar: http://bar#" --add-prefix "baz: http://baz#" \
export-prefixes --output results/bar.json

The difference is that the `--prefix`/`--prefixes` options do not include the new prefix in the header of the output ontology, whereas `--add-prefix`/`--add-prefixes` options do, for example in Turtle:

```
@prefix foo: <http://foo#> .
@prefix bar: <http://bar#> .
@prefix baz: <http://baz#> .
```


## XML Catalogs

OWLAPI, Protégé, and ROBOT use XML catalogs to specify where import files are located when loading an ontology. By default, this catalog is called `catalog-v001.xml`. ROBOT assumes that a `catalog-v001.xml` file exists in the input directory (the same directory as the `--input` ontology) and attempts to resolve imports based on that. Because Protégé also predicts that catalog, we recommend sticking to this standard. For more details, see [Importing Ontologies in Protégé and OWL 2](https://protegewiki.stanford.edu/wiki/Importing_Ontologies_in_P41).

That said, occasionally, you may want to use different catalog files for different purposes. This is especially useful for automated releases with [Makefiles](/makefile). ROBOT provides an option to specify the catalog location with `--catalog`.
That said, occasionally, you may want to use different catalog files for different purposes. This is especially useful for automated releases with [Makefiles](/makefile). ROBOT provides an option to specify the catalog location with `--catalog`.

For example, you may want to [`merge`](/merge) a set of edited import ontologies to create a module. You may have one set of imports for one module, and another set of imports for another module. You can also chain this command with [`annotate`](/annotate) to specify the output ontology's IRI.

Expand Down
8 changes: 1 addition & 7 deletions docs/rename.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,8 @@ Renames the base IRIs of all matching entites (e.g. change the prefix `http://pu
--prefix-mappings partial-rename.tsv \
--add-prefix "fb: http://foo.bar/"
--output results/partial-rename.owl
The `--add-prefix` option allows you to specify a prefix mapping in the same way as the [global prefix option](/global#prefixes). This will be added to the output ontology:

```
Prefix(fb:=<http://foo.bar/>)
```

The difference is that the global `--prefix` option does not include the prefix in the output ontology.
More information on the `--add-prefix` option can be found in [Global Options](/global#prefixes).

### Mappings Files

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package org.obolibrary.robot;

import com.github.jsonldjava.core.Context;
import java.io.*;
import java.net.URI;
import java.net.URISyntaxException;
Expand All @@ -18,6 +19,7 @@
import org.apache.commons.cli.ParseException;
import org.apache.commons.cli.PosixParser;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.IOExceptionWithCause;
import org.apache.commons.io.filefilter.WildcardFileFilter;
import org.geneontology.reasoner.ExpressionMaterializingReasonerFactory;
Expand Down Expand Up @@ -398,8 +400,8 @@ public static String getRequiredValue(CommandLine line, String name, String mess
}

/**
* Given a command line, return an initialized IOHelper. The --prefix, --prefixes, --noprefixes
* and --xml-entities options are handled.
* Given a command line, return an initialized IOHelper. The --prefix, --add-prefix, --prefixes,
* --add-prefixes, --noprefixes and --xml-entities options are handled.
*
* @param line the command line to use
* @return an initialized IOHelper
Expand All @@ -413,13 +415,17 @@ public static IOHelper getIOHelper(CommandLine line) throws IOException {
} else {
ioHelper = new IOHelper(!line.hasOption("noprefixes"));
}
prefixes = getOptionalValue(line, "add-prefixes");
if (prefixes != null) {
ioHelper.addPrefixes(prefixes);
}

for (String prefix : getOptionalValues(line, "prefix")) {
try {
ioHelper.addPrefix(prefix);
} catch (IllegalArgumentException e) {
throw new IllegalArgumentException(String.format(IOHelper.invalidPrefixError, prefix), e);
}
ioHelper.addPrefix(prefix);
}

for (String prefix : getOptionalValues(line, "add-prefix")) {
ioHelper.addPrefix(prefix);
}

ioHelper.setXMLEntityFlag(line.hasOption("xml-entities"));
Expand Down Expand Up @@ -625,9 +631,36 @@ public static IRI getOutputIRI(CommandLine line) {
*/
public static void maybeSaveOutput(CommandLine line, OWLOntology ontology) throws IOException {
IOHelper ioHelper = getIOHelper(line);
// Determine if OBO structure should be enforced
boolean checkOBO = CommandLineHelper.getBooleanValue(line, "check", true);

// Determine if prefixes should be added to the header of output
// Create a map of these to include in output (or an empty map)
Map<String, String> addPrefixes = getAddPrefixes(line);

// Get an output format or null
// If null, format will be guessed from the output path
String format = CommandLineHelper.getOptionalValue(line, "format");
OWLDocumentFormat df;
if (format != null) {
df = IOHelper.getFormat(format);
} else {
df = null;
}

// Save outputs
for (String path : getOptionValues(line, "output")) {
try {
ioHelper.saveOntology(ontology, path);
// maybe guess the document format
OWLDocumentFormat thisDf = df;
if (thisDf == null) {
if (path.endsWith(".gz")) {
path = path.substring(0, path.lastIndexOf("."));
}
String formatName = FilenameUtils.getExtension(path);
thisDf = IOHelper.getFormat(formatName);
}
ioHelper.saveOntology(ontology, thisDf, IRI.create(new File(path)), addPrefixes, checkOBO);
} catch (IllegalArgumentException e) {
// Exception from getFormat -- invalid format
throw new IllegalArgumentException(
Expand All @@ -654,6 +687,37 @@ public static IRI maybeCreateIRI(IOHelper ioHelper, String term, String field) {
return iri;
}

/**
* Given a command line, get a map of all prefixes to use and add to the output.
*
* @param line the command line to use
* @return a map of prefixes to add to output
* @throws IOException if the prefixes are not formatted correctly or a JSON file cannot be read
*/
public static Map<String, String> getAddPrefixes(CommandLine line) throws IOException {
Map<String, String> addPrefixes = new HashMap<>();
if (line.hasOption("add-prefix")) {
for (String pref : CommandLineHelper.getOptionalValues(line, "add-prefix")) {
String[] split = pref.split(": ");
if (split.length != 2) {
throw new IOException(String.format(IOHelper.invalidPrefixError, pref));
}
addPrefixes.put(split[0], split[1]);
}
}
if (line.hasOption("add-prefixes")) {
for (String prefixFilePath : CommandLineHelper.getOptionalValues(line, "add-prefixes")) {
File prefixFile = new File(prefixFilePath);
if (!prefixFile.exists()) {
throw new IOException(String.format(IOHelper.fileDoesNotExistError, prefixFilePath));
}
Context json = IOHelper.parseContext(FileUtils.readFileToString(prefixFile));
addPrefixes.putAll(json.getPrefixes(false));
}
}
return addPrefixes;
}

/**
* Given an IOHelper and a command line, check for the required options and return a set of IRIs
* for terms. Handles --terms and --term-file options.
Expand Down Expand Up @@ -841,6 +905,8 @@ public static Options getCommonOptions() {
o.addOption("p", "prefix", true, "add a prefix 'foo: http://bar'");
o.addOption("P", "prefixes", true, "use prefixes from JSON-LD file");
o.addOption("noprefixes", false, "do not use default prefixes");
o.addOption(null, "add-prefix", true, "add prefix 'foo: http://bar' to the output");
o.addOption(null, "add-prefixes", true, "add JSON-LD prefixes to the output");
o.addOption("x", "xml-entities", false, "use entity substitution with ontology XML output");
return o;
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
package org.obolibrary.robot;

import java.io.File;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.Options;
import org.apache.commons.io.FilenameUtils;
import org.semanticweb.owlapi.model.OWLOntology;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
Expand Down Expand Up @@ -139,26 +137,8 @@ public CommandState execute(CommandState state, String[] args) throws Exception
} else if (outputs.length > 1) {
throw new IllegalArgumentException(multipleOutputsError);
}
File outputFile = CommandLineHelper.getOutputFile(line);

// Check for a format
String formatName = CommandLineHelper.getOptionalValue(line, "format");
if (formatName == null) {
formatName = FilenameUtils.getExtension(outputFile.getName());
if (formatName.equals("")) {
throw new IllegalArgumentException(missingFormatError);
}
}

boolean checkOBO = true;
String check = CommandLineHelper.getDefaultValue(line, "check", "true");
if ("false".equals(check.toLowerCase())) {
checkOBO = false;
} else if (!"true".equals(check.toLowerCase())) {
throw new IllegalArgumentException(checkArgError);
}

ioHelper.saveOntology(ontology, IOHelper.getFormat(formatName), outputFile, checkOBO);
CommandLineHelper.maybeSaveOutput(line, ontology);
return state;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
import com.opencsv.CSVReaderBuilder;
import java.io.*;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.commons.cli.CommandLine;
import org.apache.commons.cli.Options;
Expand Down Expand Up @@ -59,12 +58,12 @@ public RenameCommand() {
o.addOption("o", "output", true, "save ontology to a file");
o.addOption("m", "mappings", true, "table of mappings for renaming");
o.addOption("r", "prefix-mappings", true, "table of prefix mappings for renaming");
o.addOption("A", "add-prefix", true, "add a new prefix to ontology file header");
o.addOption(
"d",
"allow-duplicates",
true,
"allow two or more terms to be renamed to the same full IRI");
o.addOption("A", "add-prefix", true, "add prefix 'foo: http://bar' to the output");
options = o;
}

Expand Down Expand Up @@ -136,12 +135,6 @@ public CommandState execute(CommandState state, String[] args) throws Exception
state = CommandLineHelper.updateInputOntology(ioHelper, state, line);
OWLOntology ontology = state.getOntology();

// Get additional prefixes to add to prefix manager
List<String> addPrefixes = CommandLineHelper.getOptionalValues(line, "add-prefix");
for (String pref : addPrefixes) {
ioHelper.addPrefix(pref);
}

String fullFile = CommandLineHelper.getOptionalValue(line, "mappings");
String prefixFile = CommandLineHelper.getOptionalValue(line, "prefix-mappings");
if (fullFile == null && prefixFile == null) {
Expand All @@ -165,12 +158,7 @@ public CommandState execute(CommandState state, String[] args) throws Exception
RenameOperation.renamePrefixes(ontology, ioHelper, mappings);
}

if (!addPrefixes.isEmpty()) {
ioHelper.addPrefixesAndSave(ontology, CommandLineHelper.getOutputFile(line), addPrefixes);
} else {
CommandLineHelper.maybeSaveOutput(line, ontology);
}

CommandLineHelper.maybeSaveOutput(line, ontology);
state.setOntology(ontology);
return state;
}
Expand Down
Loading

0 comments on commit 4d202cf

Please sign in to comment.