Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to load metadata blocks and adjust Solr in Docker #11204

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/release-notes/11004-docker-block-solr.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The tutorial on running Dataverse in Docker has been updated to include [how to load a metadata block](https://dataverse-guide--11204.org.readthedocs.build/en/11204/container/running/demo.html#additional-metadata-blocks) and then update Solr to know about the new fields. See also #11004 and #11204
47 changes: 47 additions & 0 deletions doc/sphinx-guides/source/container/running/demo.rst
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,53 @@ PID Providers

Dataverse supports multiple Persistent ID (PID) providers. The ``compose.yml`` file uses the Permalink PID provider. Follow :ref:`pids-configuration` to reconfigure as needed.

.. _additional-metadata-blocks:

Additional Metadata Blocks
++++++++++++++++++++++++++

Metadata fields such as "Title" are part of a metadata block such as "Citation". See :ref:`metadata-references` in the User Guide for the metadata blocks that ship with Dataverse.

At a high level, we will be loading a metadata block and then adjusting our Solr config to know about it.

Care should be taken when adding additional metadata blocks. There is no way to `preview <https://github.com/IQSS/dataverse/issues/2551>`_ or `delete <https://github.com/IQSS/dataverse/issues/9628>`_ a metadata block so please use a throwaway environment.

:ref:`metadata-references` lists some experimental metadata blocks. In the example below, we'll use the CodeMeta block.

First, download a metadata block or create one by following :doc:`/admin/metadatacustomization` in the Admin Guide.

Load the metadata block like this:

``curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file codemeta.tsv``

Next, reconfigure Solr to know about the new metadata block.

You can back up your existing Solr schema like this:

``cp docker-dev-volumes/solr/data/data/collection1/conf/schema.xml docker-dev-volumes/solr/data/data/collection1/conf/schema.xml.orig``

You can see the existing fields Solr know about like this:

``curl http://localhost:8983/solr/collection1/schema/fields``

Update your Solr schema with the following command:

``curl http://localhost:8080/api/admin/index/solr/schema | docker run -i --rm -v ./docker-dev-volumes/solr/data:/var/solr gdcc/configbaker:unstable update-fields.sh /var/solr/data/collection1/conf/schema.xml``

Then, reload Solr:

``curl "http://localhost:8983/solr/admin/cores?action=RELOAD&core=collection1"``

You can get a diff of your old and new Solr schema like this:

``diff docker-dev-volumes/solr/data/data/collection1/conf/schema.xml.orig docker-dev-volumes/solr/data/data/collection1/conf/schema.xml``

You should be able to see the new fields from the metadata block you added in the following output:

``curl http://localhost:8983/solr/collection1/schema/fields``

At this point you can proceed with testing the metadata block in the Dataverse UI. First you'll need to enable it for a collection (see :ref:`general-information` in the User Guide section about collection). Afterwards, create a new dataset, save it, and then edit the metadata for that dataset. Your metadata block should appear.

Next Steps
----------

Expand Down
2 changes: 1 addition & 1 deletion doc/sphinx-guides/source/developers/tips.rst
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Both developers and sysadmins need to update the Solr schema from time to time.

At this point you can do a ``git diff`` and see if your changes make sense before committing.

Sysadmins are welcome to run ``update-fields.sh`` however they like. See :ref:`update-solr-schema` in the Admin Guide for details.
Sysadmins are welcome to run ``update-fields.sh`` however they like. See :ref:`update-solr-schema` in the Admin Guide and :ref:`additional-metadata-blocks` in the Container Guide for details.

Git
---
Expand Down