Cycling Api-X stack and loading Crayfish microservices into Api-X

[dannylamb]

Resolves: https://github.com/Islandora-CLAW/CLAW#738 and https://github.com/Islandora-CLAW/CLAW#739

Adds two tasks with the tags

• apix-reset
• apix-crayfish

apix-reset cycles karaf, tomcat, and activemq with waits inbetween to address fcrepo4-labs/fcrepo-api-x#149. apix-crayfish loads our crayfish microservices into Api-X via its loader service.

Setup

If you're bringing up a new box environment with these changes, you don't need to do anything else before testing. If you have an existing environment, you can pull in these changes and run the tasks with ansible-playbook -i inventory/vagrant -t apix-reset,apix-crayfish playbook.yml.

Both check conditions and skip running when not neccessary, so they should be idempotent. You should run them again and verify.

Testing

• Ingest a Tiff (you can use Drupal or go to Fedora directly)
• Issue a HEAD request against the Tiff to get its service doc in the link headers. For example, my tiff was located at http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275, so I did this:

daniel@daniel-Latitude-3560:~/Code/Environments/claw_vagrant$ curl -I http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275
HTTP/1.1 200 OK
Content-Type: image/tiff
Accept: */*
Accept-Ranges: bytes
Allow: DELETE,HEAD,GET,PUT,OPTIONS
Apix-Modality: intercept; outgoing
breadcrumbId: ID-claw-43671-1509456249000-4-42
Content-Disposition: attachment; filename=""; creation-date="Tue, 31 Oct 2017 13:35:45 GMT"; modification-date="Tue, 31 Oct 2017 13:35:45 GMT"; size=10409
ETag: "e0a384c439ac9bb813ecbc8c49dfdbc859190527"
Expires: Thu, 01 Jan 1970 00:00:00 UTC
Last-Modified: Tue, 31 Oct 2017 13:35:45 GMT
Link: <http://www.w3.org/ns/ldp#NonRDFSource>;rel="type"
Link: <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275/fcr:metadata>; rel="describedby"
Link: <http://www.w3.org/ns/ldp#Resource>;rel="type"
Link: <http://localhost:8081/discovery/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275>; rel="service"
Server: Apache-Coyote/1.1
Content-Length: 0

• Then issue a GET request on the service doc (it's the link header of rel=service in the HEAD response)

daniel@daniel-Latitude-3560:~/Code/Environments/claw_vagrant$ curl http://localhost:8081/discovery/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275
<>      a       <http://fedora.info/definitions/v4/service#ServiceDocument> ;
        <http://fedora.info/definitions/v4/service#isServiceDocumentFor>
                <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275> ;
        <http://www.openarchives.org/ore/terms/describes>
                <#services> .

<#c708fb29-8877-4c07-b705-62634e2b7039>
        a       <http://fedora.info/definitions/v4/service#ServiceInstance> ;
        <http://fedora.info/definitions/v4/service#hasEndpoint>
                <http://localhost:8081/services/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275/svc:ocr> ;
        <http://fedora.info/definitions/v4/service#isFunctionOf>
                <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275> ;
        <http://fedora.info/definitions/v4/service#isServiceInstanceOf>
                <http://islandora.ca/CLAW#OcrService> ;
        <http://fedora.info/definitions/v4/service#serviceInstanceExposedBy>
                <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275> .

<#services>  <http://www.openarchives.org/ore/terms/aggregates>
                <#c708fb29-8877-4c07-b705-62634e2b7039> , <#e301fdf5-cd4a-4169-afa0-c03989cb136d> , <#b25c2e9f-26c7-40e7-b24c-697856bd8d5e> .

<#b25c2e9f-26c7-40e7-b24c-697856bd8d5e>
        a       <http://fedora.info/definitions/v4/service#ServiceInstance> ;
        <http://fedora.info/definitions/v4/service#hasEndpoint>
                <http://localhost:8081/services/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275/svc:convert> ;
        <http://fedora.info/definitions/v4/service#isFunctionOf>
                <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275> ;
        <http://fedora.info/definitions/v4/service#isServiceInstanceOf>
                <http://islandora.ca/CLAW#ConvertService> ;
        <http://fedora.info/definitions/v4/service#serviceInstanceExposedBy>
                <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275> .

<#e301fdf5-cd4a-4169-afa0-c03989cb136d>
        a       <http://fedora.info/definitions/v4/service#ServiceInstance> ;
        <http://fedora.info/definitions/v4/service#hasEndpoint>
                <http://localhost:8081/services/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275/svc:identify> ;
        <http://fedora.info/definitions/v4/service#isFunctionOf>
                <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275> ;
        <http://fedora.info/definitions/v4/service#isServiceInstanceOf>
                <http://islandora.ca/CLAW#IdentifyService> ;
        <http://fedora.info/definitions/v4/service#serviceInstanceExposedBy>
                <http://localhost:8081/fcrepo/rest/eb/97/cf/93/eb97cf93-f162-4981-8007-f83319848275> .

You should see three service instances. One for ocr, convert, and identify.

[whikloj]

I really feel that with the complete gutting of Alpaca in Islandora/documentation#640 we should hold off on these PRs and get that merged. I know @dannylamb wanted to wait until after the sprint to avoid causing havok, but once the sprint is done we should prioritize merging 640. Then review all open tickets around the API-X/Karaf cycling.

[dannylamb]

@whikloj Now that Islandora/documentation#640 is in, I've tested and this is still needed. The stack starts up at least, but API-X's discovery services won't work without the cycling in this PR. 😭

[whikloj]

@dannylamb so I haven't had a lot of time to look at this, but my first thought is we probably don't want API-X pulling in the services. If they aren't getting registered, perhaps we can make the individual service send out a notification that API-X handles by doing the restart. That way if we add more microservices we don't have to update the API-X role for each.

[dannylamb]

@whikloj Yeah, that makes sense. I'll circle back around to this soon and see what I can do.

[dannylamb]

@whikloj Ok, I pushed the loading of services into the crayfish role and shuffled around some config. I'm testing it out now.

[dannylamb]

Looks like I broke things when merging #45. I'm going to test to confirm.

[whikloj]

@dannylamb I know you are busy, but if this PR is related to this error.

2018-02-05 20:39:05,789 | WARN  | Consumer[fedora] | UpdateListener                   | 64 - fcrepo-api-x-listener - 0.3.0.SNAPSHOT | Update to <http://localhost:8080/fcrepo/rest/> failed
java.lang.IllegalStateException: getService() returned null for [org.fcrepo.apix.model.components.ExtensionRegistry]

Maybe we should get it rebased and I can test it out. Or at least give us some steps to resolve it, because I can't update an islandora_image now.

[whikloj]

That one seems to be related to the root element, here is when trying to add a description to an already synced islandora_collection.

2018-02-05 20:46:38,494 | WARN  | Consumer[fedora] | UpdateListener                   | 64 - fcrepo-api-x-listener - 0.3.0.SNAPSHOT | Update to <http://localhost:8080/fcrepo/rest/1c/e6/1d/69/1ce61d69-431f-465f-be19-fbac87baca3e> failed
java.lang.IllegalStateException: getService() returned null for [org.fcrepo.apix.model.components.ExtensionRegistry]

[whikloj]

Nevermind, I guess this is not related. It was one part browser cache and one part failed update. But in the end this was not the problem.

[whikloj]

@dannylamb what do we think about this PR? Should we wait until API-X is working with Fedora 5.0.0 and see if we still need it?

[dannylamb]

Let's close it and I'll re-do things once API-X is a go.