-
Notifications
You must be signed in to change notification settings - Fork 490
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #46 from IQSS/develop
Update from develop IQSS
- Loading branch information
Showing
73 changed files
with
2,049 additions
and
807 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# Multiple Store Support | ||
Dataverse can now be configured to store files in more than one place at the same time (multiple file, s3, and/or swift stores). | ||
|
||
General information about this capability can be found in the <a href="http://guides.dataverse.org/en/latest/installation/config.html">Configuration Guide</a> - File Storage section. | ||
|
||
**Upgrade Information:** | ||
|
||
**Existing installations will need to make configuration changes to adopt this version, regardless of whether additional stores are to be added or not.** | ||
|
||
Multistore support requires that each store be assigned a label, id, and type - see the documentation for a more complete explanation. For an existing store, the recommended upgrade path is to assign the store id based on it's type, i.e. a 'file' store would get id 'file', an 's3' store would have the id 's3'. | ||
|
||
With this choice, no manual changes to datafile 'storageidentifier' entries are needed in the database. (If you do not name your existing store using this convention, you will need to edit the database to maintain access to existing files!). | ||
|
||
The following set of commands to change the Glassfish JVM options will adapt an existing file or s3 store for this upgrade: | ||
For a file store: | ||
|
||
./asadmin create-jvm-options "\-Ddataverse.files.file.type=file" | ||
./asadmin create-jvm-options "\-Ddataverse.files.file.label=file" | ||
./asadmin create-jvm-options "\-Ddataverse.files.file.directory=<your directory>" | ||
|
||
For an s3 store: | ||
|
||
./asadmin create-jvm-options "\-Ddataverse.files.s3.type=s3" | ||
./asadmin create-jvm-options "\-Ddataverse.files.s3.label=s3" | ||
./asadmin delete-jvm-options "-Ddataverse.files.s3-bucket-name=<your_bucket_name>" | ||
./asadmin create-jvm-options "-Ddataverse.files.s3.bucket-name=<your_bucket_name>" | ||
|
||
Any additional S3 options you have set will need to be replaced as well, following the pattern in the last two lines above - delete the option including a '-' after 's3' and creating the same option with the '-' replaced by a '.', using the same value you currently have configured. | ||
|
||
Once these options are set, restarting the glassfish service is all that is needed to complete the change. | ||
|
||
<<<<<<< HEAD | ||
Note that the "\-Ddataverse.files.directory", if defined, continues to control where temporary files are stored (in the /temp subdir of that directory), independent of the location of any 'file' store defined above. | ||
======= | ||
Note that the "\-Ddataverse.files.directory", if defined, continues to control where temporary files are stored (in the /temp subdir of that directory), independent of the location of any 'file' store defined above. | ||
>>>>>>> branch 'IQSS/6485' of https://github.com/TexasDigitalLibrary/dataverse.git |
22 changes: 22 additions & 0 deletions
22
doc/release-notes/6510-duplicate-datafiles-and-datatables.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
We recently discovered a *potential* data integrity issue in | ||
Dataverse databases. One manifests itself as duplicate DataFile | ||
objects created for the same uploaded file (https://github.com/IQSS/dataverse/issues/6522); the other as duplicate | ||
DataTable (tabular metadata) objects linked to the same | ||
DataFile (https://github.com/IQSS/dataverse/issues/6510). This issue impacted approximately .03% of datasets in Harvard's Dataverse. | ||
|
||
To see if any datasets in your installation have been impacted by this data integrity issue, we've provided a diagnostic script here: | ||
|
||
https://github.com/IQSS/dataverse/raw/develop/scripts/issues/6510/check_datafiles_6522_6510.sh | ||
|
||
The script relies on the PostgreSQL utility psql to access the | ||
database. You will need to edit the credentials at the top of the script | ||
to match your database configuration. | ||
|
||
If neither of the two issues is present in your database, you will see | ||
a message "... no duplicate DataFile objects in your database" and "no | ||
tabular files affected by this issue in your database". | ||
|
||
If either, or both kinds of duplicates are detected, the script will | ||
provide further instructions. We will need you to send us the produced | ||
output. We will then assist you in resolving the issues in your | ||
database. |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Note for integrators - the role alias has changed, so if anything was hard-coded to "editor" instead of "contributor" it'll need to be updated. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
Integration Test Coverage Reporting | ||
|
||
API-based integration tests are run every time a branch is merged to develop and the percentage of code covered by these integration tests is now shown on a badge at the bottom of the README.md file that serves as the homepage of Dataverse Github Repository. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Google Analytics Download Tracking Bug | ||
|
||
The button tracking capability discussed in the installation guide (http://guides.dataverse.org/en/4.20/installation/config.html#id88) relies on an analytics-code.html file that must be configured using the :WebAnalyticsCode setting. The example file provided in the installation guide is no longer compatible with recent Dataverse releases (>v4.16). Installations using this feature should update their analytics-code.html file by following the installation instructions using the updated example file. (Alternately, sites can modify their existing files to include the one-line change made in the example file at line 120.) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.