Merge pull request #46 from IQSS/develop

Update from develop IQSS

README.md

@@ -18,6 +18,7 @@ Dataverse is a trademark of President and Fellows of Harvard College and is regi
 [![Dataverse Project logo](src/main/webapp/resources/images/dataverseproject_logo.jpg?raw=true "Dataverse Project")](http://dataverse.org)
 
 [![API Test Status](https://jenkins.dataverse.org/buildStatus/icon?job=IQSS-dataverse-develop&subject=API%20Test%20Status)](https://jenkins.dataverse.org/job/IQSS-dataverse-develop/)
+[![API Test Coverage](https://img.shields.io/jenkins/coverage/jacoco?jobUrl=https%3A%2F%2Fjenkins.dataverse.org%2Fjob%2FIQSS-dataverse-develop&label=API%20Test%20Coverage)](https://jenkins.dataverse.org/job/IQSS-dataverse-develop/)
 [![Unit Test Status](https://img.shields.io/travis/IQSS/dataverse?label=Unit%20Test%20Status)](https://travis-ci.org/IQSS/dataverse)
 [![Unit Test Coverage](https://img.shields.io/coveralls/github/IQSS/dataverse?label=Unit%20Test%20Coverage)](https://coveralls.io/github/IQSS/dataverse?branch=develop)
 

conf/solr/7.7.2/schema.xml

@@ -293,7 +293,7 @@
     <!-- Dataverse copyField from http://localhost:8080/api/admin/index/solr/schema -->
     <xi:include href="schema_dv_mdb_copies.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
 
-<!-- End: Dataverse Specific -->
+<!-- End: Dataverse-specific -->
 
     <!-- This can be enabled, in case the client does not know what fields may be searched. It isn't enabled by default
          because it's very expensive to index everything twice. -->

doc/release-notes/4.19-release-notes.md

@@ -83,7 +83,7 @@ Additional fields are now available via the Search API, mostly related to inform
 
 ## Complete List of Changes
 
-For the complete list of code changes in this release, see the <a href="https://github.com/IQSS/dataverse/milestone/86?closed=1">4.19 milestone</a> in Github.
+For the complete list of code changes in this release, see the <a href="https://github.com/IQSS/dataverse/milestone/87?closed=1">4.19 milestone</a> in Github.
 
 For help with upgrading, installing, or general questions please post to the <a href="https://groups.google.com/forum/#!forum/dataverse-community">Dataverse Google Group</a> or email support@dataverse.org.
 

doc/release-notes/6485-multiple-stores.md

@@ -0,0 +1,36 @@
+# Multiple Store Support
+Dataverse can now be configured to store files in more than one place at the same time (multiple file, s3, and/or swift stores). 
+
+General information about this capability can be found in the <a href="http://guides.dataverse.org/en/latest/installation/config.html">Configuration Guide</a> - File Storage section.
+
+**Upgrade Information:** 
+
+**Existing installations will need to make configuration changes to adopt this version, regardless of whether additional stores are to be added or not.**
+
+Multistore support requires that each store be assigned a label, id, and type - see the documentation for a more complete explanation. For an existing store, the recommended upgrade path is to assign the store id based on it's type, i.e. a 'file' store would get id 'file', an 's3' store would have the id 's3'. 
+
+With this choice, no manual changes to datafile 'storageidentifier' entries are needed in the database. (If you do not name your existing store using this convention, you will need to edit the database to maintain access to existing files!).
+
+The following set of commands to change the Glassfish JVM options will adapt an existing file or s3 store for this upgrade:
+For a file store:
+
+    ./asadmin create-jvm-options "\-Ddataverse.files.file.type=file"
+    ./asadmin create-jvm-options "\-Ddataverse.files.file.label=file"
+    ./asadmin create-jvm-options "\-Ddataverse.files.file.directory=<your directory>"
+
+For an s3 store:
+
+    ./asadmin create-jvm-options "\-Ddataverse.files.s3.type=s3"
+    ./asadmin create-jvm-options "\-Ddataverse.files.s3.label=s3"
+    ./asadmin delete-jvm-options "-Ddataverse.files.s3-bucket-name=<your_bucket_name>"
+    ./asadmin create-jvm-options "-Ddataverse.files.s3.bucket-name=<your_bucket_name>"
+
+Any additional S3 options you have set will need to be replaced as well, following the pattern in the last two lines above - delete the option including a '-' after 's3' and creating the same option with the '-' replaced by a '.', using the same value you currently have configured.  
+
+Once these options are set, restarting the glassfish service is all that is needed to complete the change.  
+
+<<<<<<< HEAD
+Note that the "\-Ddataverse.files.directory", if defined, continues to control where temporary files are stored (in the /temp subdir of that directory), independent of the location of any 'file' store defined above.
+=======
+Note that the "\-Ddataverse.files.directory", if defined, continues to control where temporary files are stored (in the /temp subdir of that directory), independent of the location of any 'file' store defined above.
+>>>>>>> branch 'IQSS/6485' of https://github.com/TexasDigitalLibrary/dataverse.git

doc/release-notes/6510-duplicate-datafiles-and-datatables.md

@@ -0,0 +1,22 @@
+We recently discovered a *potential* data integrity issue in
+Dataverse databases.  One manifests itself as duplicate DataFile
+objects created for the same uploaded file (https://github.com/IQSS/dataverse/issues/6522); the other as duplicate
+DataTable (tabular metadata) objects linked to the same
+DataFile (https://github.com/IQSS/dataverse/issues/6510). This issue impacted approximately .03% of datasets in Harvard's Dataverse.  
+
+To see if any datasets in your installation have been impacted by this data integrity issue, we've provided a diagnostic script here:
+
+https://github.com/IQSS/dataverse/raw/develop/scripts/issues/6510/check_datafiles_6522_6510.sh
+
+The script relies on the PostgreSQL utility psql to access the
+database. You will need to edit the credentials at the top of the script
+to match your database configuration.
+
+If neither of the two issues is present in your database, you will see
+a message "... no duplicate DataFile objects in your database" and "no
+tabular files affected by this issue in your database".
+
+If either, or both kinds of duplicates are detected, the script will
+provide further instructions. We will need you to send us the produced
+output. We will then assist you in resolving the issues in your
+database.

doc/release-notes/6644-role-name-change.md

@@ -0,0 +1 @@
+Note for integrators - the role alias has changed, so if anything was hard-coded to "editor" instead of "contributor" it'll need to be updated. 

doc/release-notes/6711-coverage-badge

@@ -0,0 +1,3 @@
+Integration Test Coverage Reporting
+
+API-based integration tests are run every time a branch is merged to develop and the percentage of code covered by these integration tests is now shown on a badge at the bottom of the README.md file that serves as the homepage of Dataverse Github Repository.

doc/release-notes/6725-analytics-bug.md

@@ -0,0 +1,3 @@
+# Google Analytics Download Tracking Bug
+
+The button tracking capability discussed in the installation guide (http://guides.dataverse.org/en/4.20/installation/config.html#id88) relies on an analytics-code.html file that must be configured using the :WebAnalyticsCode setting. The example file provided in the installation guide is no longer compatible with recent Dataverse releases (>v4.16). Installations using this feature should update their analytics-code.html file by following the installation instructions using the updated example file. (Alternately, sites can modify their existing files to include the one-line change made in the example file at line 120.)

doc/sphinx-guides/source/_static/installation/files/var/www/dataverse/branding/analytics-code.html

@@ -117,7 +117,7 @@
       var row = target.parents('tr')[0];
       if(row != null) {
         //finds the file id/DOI in the Dataset page
-        label = $(row).find('td.col-file-metadata > a').attr('href');
+        label = $(row).find('div.file-metadata-block > a').attr('href');
       } else {
         //finds the file id/DOI in the file page
         label = $('#fileForm').attr('action');

doc/sphinx-guides/source/admin/dataverses-datasets.rst

@@ -38,7 +38,27 @@ Add Dataverse RoleAssignments to Child Dataverses
 
 Recursively assigns the users and groups having a role(s),that are in the set configured to be inheritable via the :InheritParentRoleAssignments setting, on a specified dataverse to have the same role assignments on all of the dataverses that have been created within it. The response indicates success or failure and lists the individuals/groups and dataverses involved in the update. Only accessible to superusers. ::
  
-    curl -H "X-Dataverse-key: $API_TOKEN" http://$SERVER/api/admin/dataverse/$dataverse-alias//addRoleAssignmentsToChildren
+    curl -H "X-Dataverse-key: $API_TOKEN" http://$SERVER/api/admin/dataverse/$dataverse-alias/addRoleAssignmentsToChildren
+    
+Configure a Dataverse to store all new files in a specific file store
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To direct new files (uploaded when datasets are created or edited) for all datasets in a given dataverse, the store can be specified via the API as shown below, or by editing the 'General Information' for a Dataverse on the Dataverse page. Only accessible to superusers. ::
+ 
+    curl -H "X-Dataverse-key: $API_TOKEN" -X PUT -d $storageDriverLabel http://$SERVER/api/admin/dataverse/$dataverse-alias/storageDriver
+    
+The current driver can be seen using:
+
+    curl -H "X-Dataverse-key: $API_TOKEN" http://$SERVER/api/admin/dataverse/$dataverse-alias/storageDriver
+
+and can be reset to the default store with:
+
+    curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE http://$SERVER/api/admin/dataverse/$dataverse-alias/storageDriver
+
+The available drivers can be listed with:
+
+    curl -H "X-Dataverse-key: $API_TOKEN" http://$SERVER/api/admin/storageDrivers
+
 
 Datasets
 --------