CLARIAH CMDI Forms: backend services

Run the demo: Hello World Form

1. Start the docker container:

docker run -p 1210:1210 --name=ccf --rm -it ghcr.io/knaw-huc/service-huc-editor:2.0-RC5

2. Initialize the HelloWorld app:

curl -v -X PUT -H 'Authorization: Bearer foobar' http://localhost:1210/app/HelloWorld

3. Visit the HelloWorld app:

http://localhost:1210/app/HelloWorld

4. Visit the API documentation:

http://localhost:1210/docs#/

5. Take a look at the logs:

Next to the docker container log there is a more extensive log inside the container at /home/huc/huc-editor-service/logs/huc-editor-service.log

Setup your own:

##Create or select a CMDI profile

Create or select a CMDI profile in/from the Component Registry

Remember the ID of your profile, which can be seen in its XML representation, e.g. clarin.eu:cr1:p_1721373444008:

<ComponentSpec isProfile="true" CMDVersion="1.2" CMDOriginalVersion="1.2">
  <Header>
    <ID>clarin.eu:cr1:p_1721373444008</ID>
    <Name>ShowcaseForm</Name>
    <Description>A showcase profile for the CLARIAH CMDI Forms</Description>
    <Status>development</Status>
  </Header>
  ...
</ComponentSpec>

Note: CCF has some limitations!

• don't use attributes, as they are tied too close to XML and don't have a decent counterpart in JSON, RDF, ...

Start the service

With release image:

docker run -v ./conf:/home/huc/huc-editor-service/conf -v ./data:/home/huc/huc-editor-service/data -p 1210:1210 --name=ccf --rm -it ghcr.io/knaw-huc/service-huc-editor:2.0-RC5

With local image:

docker build -t ccf .
docker run -v ./conf:/home/huc/huc-editor-service/conf -v ./data:/home/huc/huc-editor-service/data -p 1210:1210 --name=ccf --rm -it ccf

This mounts both the local

• conf directory, containing the global configuration
• data directory, which will contain the app configuration, the profile, its tweak(s) and records

Initialize your app

curl -v -X PUT -H 'Authorization: Bearer foobar' http://0.0.0.0:1210/app/myApp?prof=clarin.eu:cr1:p_1721373444008&desrc="my Application"

Tweak the profile

To add cues to the profile you can download a template tweak file for your profile:

http://localhost:1210/app/helloWorld/profile/clarin.eu:cr1:p_1721373444008/tweak/template

For convenience this template replicates the full structure of the profile, but this is actually not needed: you can limit this to only the paths to the components or the elements that are tweaked. For example, only add label tweaks to the Hello element:

<ComponentSpec xmlns:clariah="http://www.clariah.eu/" xmlns:cue="http://www.clarin.eu/cmd/cues/1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" isProfile="true" CMDVersion="1.2" CMDOriginalVersion="1.2" xsi:noNamespaceSchemaLocation="https://infra.clarin.eu/CMDI/1.x/xsd/cmd-component.xsd">
    <Header>
        <ID>clarin.eu:cr1:p_1721373444008</ID>
        <Name>ShowcaseForm</Name>
        <Description>A showcase profile for the CLARIAH CMDI Forms</Description>
        <Status>development</Status>
    </Header>
    <Component name="ShowcaseForm">
        <Element name="Hello">
            <clariah:label xml:lang="fr">Bonjour</clariah:label>
            <clariah:label xml:lang="nl">Hallo</clariah:label>            
        </Element>
    </Component>
</ComponentSpec>        

To create a tweak file use:

curl -X POST -H 'Authorization: Bearer foobar' -H 'Content-Type: application/xml'  http://localhost:1210/app/helloWorld/profile/clarin.eu:cr1:p_1721373444008/tweak -v --data-binary '@./tweak.xml'

This will return you the location of the tweak file created, e.g., http://localhost:1210/app/helloWorld/profile/clarin.eu:cr1:p_1721373444008/tweak/1

An specfic tweak file can be updated using PUT, e.g.,

curl -X PUT -H 'Authorization: Bearer foobar' -H 'Content-Type: application/xml'  http://localhost:1210/app/helloWorld/profile/clarin.eu:cr1:p_1721373444008/tweak/1 -v --data-binary '@./tweak.xml'

Or deleted, e.g.:

curl -v -X PUT -H 'Authorization: Bearer foobar' http://localhost:1210/app/helloWorld/profile/clarin.eu:cr1:p_1721373444008/tweak/1

Note: the tweak file is not actually deleted on disk, but marked as such.

Multiple tweak files can exist and will be applied in order. The (future) purpose is to have dedicated tweak files for labels in specific languages, or a tweak file for indexing details. The result can be seen by requesting the profile, e.g., http://localhost:1210/app/helloWorld/profile/clarin.eu:cr1:p_1721373444008

Here some example tweak files can be found:

• clariah vocabulary
• hi data envelopes
• hi nde
• iisg adoptie (HuC only)
• iisg afstand (HuC only)
• niod yugo dre

The next sections list the cues you can add to elements and components.

CMDI cues

Since CMDI 1.2 cues for tools can be specified. CCF supports the following cues using this mechanism:

• cue:class: will add this as a CSS class, which can be used to style the elememt or component or to associate JavaScript triggers;
• cue:displayOrder: change the order of the elements or components; this allows to mix them in the editor, which can't be done in the profile specification
• cue:hide: if set to true this element or component won't be shown in the editor.
• cue:inputHeight: change how many lines an input box is;
• cue:inputWidth: change how many characters width an input box is;
• cue:readonly: if set to true the value of this element can't be changed;

Cues are attributes in the namespace http://www.clarin.eu/cmd/cues/1 on the element or component tag, e.g.,

<Element name="begeleidendeInstantiesPersonen" xmlns:cue="http://www.clarin.eu/cmd/cues/1" cue:displayOrder="45" cue:inputWidth="80" cue:inputHeight="6">

CCF cues

Multilingual labels

CMDI cues being attributes don't work together with xml:lang language attributes. To resolve this CCF uses the label element in the http://www.clariah.eu/ namespace for multilingual labels, e.g.,

<Element name="Hello" xmlns:clariah="http://www.clariah.eu/">
    <clariah:label xml:lang="fr">Bonjour</clariah:label>
    <clariah:label xml:lang="nl">Hallo</clariah:label>            
</Element>

Vocabularies

Note: CCF doesn't support the CMDI external vocabularies syntax yet!

Currently you can specify an autocompleteURL in the http://www.clariah.eu/ namespace:

<Element name="researchActivity" cue:class="skosType" xmlns:clariah="http://www.clariah.eu/">
    <clariah:autoCompleteURI>/proxy/skosmos/sd/tadirah</clariah:autoCompleteURI>
</Element>

The URI follows the follow pattern:

• proxy: use the CCF proxy
• the proxy recipe, e.g., skosmos
• a specific instance of the recipe, e.g., sd (see ./resources/proxies/ for an overview)
• a vocab in that instance, e.g., tadirah

Note: add the cue:class="skosType" to get an icon to open up a dialog for browsing the vocabulary next to the default autocomplete.

CMDI auto values

CMDI 1.2 also added a way to specify auto values for an element, here the CCF supports:

• now for a date value scheme, will set the element to the current date

<Element name="modified">
    <AutoValue>now</AutoValue>
</Element>

Other tweaks

In the tweak file propererties of elements can be changed within limits:

• minimum and maximum cardinality within the original bounds
• a vocabulary can be added as long as the values match the original value scheme

Configure the app

The configuration of the app can be retrieved:

curl -v -X GT -H 'Authorization: Bearer foobar' http://localhost:1210/app/HelloWorld/config
.toml

To update the config.toml use PUT with application/toml as the Content-Type:

curl -X PUT -H 'Authorization: Bearer foobar' -H 'Content-Type: application/toml' http://localhost:1210/app/HelloWorld/config -v --data-binary '@./config.toml'

Here are some example configurations:

• clariah vocabulary
• hi data envelopes
• hi nde
• iisg adoptie (HuC only)
• iisg afstand (HuC only)
• niod yugo dre

The CMDI version in the app

Set the version of CMDI to be used for record serialisation.

[app]
cmdi_version="1.2" #or 1.1, which is (for legacy reasons) the default

The profiles in the app

An app is based on one or more CMD profiles. There should be at least one profile and one profile should be the default:

[app]
def_prof="clarin.eu:cr1:p_1721373444008"

Each profile needs to have an id, e.g. HelloWorld and a main entry in the config:

[app.prof.HelloWorld]
prof="clarin.eu:cr1:p_1721373444008"
title="string((/cmd:CMD/cmd:Components//cmd:*[empty(cmd:*)][normalize-space(text())!=''])[1])" #for CMDI version 1.1
#title="string((/cmd:CMD/cmd:Components//cmdp:*[empty(cmdp:*)][normalize-space(text())!=''])[1])" #for CMDI version 1.2

The title XPath is used to retrieve the title of the record in the HTML and PDF views (note: the cmd and cmdp namespace prefixes are available and set compliant with the configured CMDI version).

The columns in the record list

The per profile record list contains by default the creation date of the record, but other fields can be added in the apps configuration, e.g.,

[app.prof.HelloWorld.list.who]
xpath="string(/cmd:CMD/cmd:Components/cmd:ShowcaseForm/cmd:Hello)" # for CMDI 1.1
# xpath="string(/cmd:CMD/cmd:Components/cmdp:ShowcaseForm/cmdp:Hello)" # for CMDI 1.2
label="Hello"
sort="true"
filter="true"

where

• who is the id of the column;
• xpath retrieves the value from the record (note: the cmd and cmdp namespace prefixes are available and set compliant with the configured CMDI version);
• label is the header of the column;
• sort indicates if the column is sortable by clicking on the header
• filter indicates if the column has a filter where you can type (true) or a dropdown of the possible values ('select', note: the single quotes are mandatory!)

Styling

Set a CSS style that overwrites the default CSS style, e.g.:

[app.html]
style="data-envelopes.css"

This CSS file should be placed in the static/css/ directory within the app directory, e.g., .../apps/data-envelopes/static/css/data-envelopes.css.

Access

Note: CCF doesn't support SSO yet!

User based access control is currently supported via htpasswd, e.g.:

htpasswd -b -c ./htp test test

To configure access control in the config add an app.access section, e.g.:

[app.access]
users="./htp.test" 
read="users"
write="users" 

where

• users points to the htpassword file;
• read indicates who have read access: authenticated users, owner or any (default)
• write indicates who have write access: authenticated users, owner or any (default)

Default credentials

In the htp.test htpassword file the following test users are available:

• user test with password test
• user demo with password demo

ALWAYS change these when running a CCF production deployment!

Hooks

For all CRUD (Create, Read, Update, Delete) pre and post python hooks are supported:

[app.hooks.record]
reate_pre="create_record_pre"
create_post="create_record_post"
#read_pre=
read_post="count"
#update_pre=
update_post="count"
#delete_pre=
delete_post="count"

The pre hooks recieve the records as a PyXdmNode, so ith can be inspected or even modified. Return the (modified) record to let the CRUD action take place, return None and a message to abort the CRUD action. The post hook cannot influence the CRUD action anymore.

Place hooks.py in in the src/ directory within the app directory, e.g., .../apps/vocabs/src/hooks.py.

import logging
from saxonche import PyXdmNode

def create_record_pre(crud:str, app: str, prof: str, nr:str, user:str, rec:PyXdmNode):
    #inspect/modify rec
    return rec, "OK"
    # or return None, "create not allowed!"

def create_record_post(crud:str, app: str, prof: str, nr:str, user:str):
    logging.debug((f"record[{nr}] created!"))

cnt = 0
def count(crud:str, app: str, prof: str, nr:str, rec, user:str):
   cnt = cnt+1

Configure the services

Default credentials

• admin API key: foobar

ALWAYS change these when running a CCF production deployment!

Global settings can be edited in the ./conf/settings.toml TOML file.

Access to the admin API

The token for the admin API is set in the ./conf/.secrets.toml:

[default]
SERVICE_HUC_EDITOR_API_KEY="foobar"