From f6e2371a8b8a7d99a74bd651365e4cdd180ae8d7 Mon Sep 17 00:00:00 2001 From: Derek Murphy Date: Mon, 5 Jun 2017 11:33:55 -0400 Subject: [PATCH] Updated Guides [ref: #3614] Deleted "Administration" from Installation Guide and moved its contents into new sections in the Admin Guide, where they make more sense. In the process, added documentation of the User List feature. --- doc/sphinx-guides/source/admin/index.rst | 4 + .../source/admin/maintenance.rst | 9 ++ doc/sphinx-guides/source/admin/monitoring.rst | 11 ++ .../source/admin/solr-search-index.rst | 47 ++++++++ .../source/admin/troubleshooting.rst | 15 ++- .../source/admin/user-administration.rst | 39 +++++++ .../source/installation/administration.rst | 103 ------------------ .../source/installation/config.rst | 6 +- .../source/installation/index.rst | 1 - .../source/installation/shibboleth.rst | 2 +- 10 files changed, 127 insertions(+), 110 deletions(-) create mode 100644 doc/sphinx-guides/source/admin/maintenance.rst create mode 100644 doc/sphinx-guides/source/admin/monitoring.rst create mode 100644 doc/sphinx-guides/source/admin/solr-search-index.rst create mode 100644 doc/sphinx-guides/source/admin/user-administration.rst delete mode 100644 doc/sphinx-guides/source/installation/administration.rst diff --git a/doc/sphinx-guides/source/admin/index.rst b/doc/sphinx-guides/source/admin/index.rst index 5224e9e2a19..046bc326813 100755 --- a/doc/sphinx-guides/source/admin/index.rst +++ b/doc/sphinx-guides/source/admin/index.rst @@ -19,4 +19,8 @@ Contents: metadataexport timers geoconnect-worldmap + user-administration + solr-search-index + monitoring + maintenance troubleshooting diff --git a/doc/sphinx-guides/source/admin/maintenance.rst b/doc/sphinx-guides/source/admin/maintenance.rst new file mode 100644 index 00000000000..09eadb097c4 --- /dev/null +++ b/doc/sphinx-guides/source/admin/maintenance.rst @@ -0,0 +1,9 @@ +Maintenance +=========== + +.. contents:: Contents: + :local: + +When you have scheduled down time for your production servers, we provide a :download:`sample maintenance page <../_static/installation/files/etc/maintenance/maintenance.xhtml>` for you to use. To download, right-click and select "Save Link As". + +The maintenance page is intended to be a static page served by Apache to provide users with a nicer, more informative experience when the site is unavailable. \ No newline at end of file diff --git a/doc/sphinx-guides/source/admin/monitoring.rst b/doc/sphinx-guides/source/admin/monitoring.rst new file mode 100644 index 00000000000..5e2eb95abca --- /dev/null +++ b/doc/sphinx-guides/source/admin/monitoring.rst @@ -0,0 +1,11 @@ +Monitoring +=========== + +.. contents:: Contents: + :local: + +In production you'll want to monitor the usual suspects such as CPU, memory, free disk space, etc. + +https://github.com/IQSS/dataverse/issues/2595 contains some information on enabling monitoring of Glassfish, which is disabled by default. + +There is a database table called ``actionlogrecord`` that captures events that may be of interest. See https://github.com/IQSS/dataverse/issues/2729 for more discussion around this table. diff --git a/doc/sphinx-guides/source/admin/solr-search-index.rst b/doc/sphinx-guides/source/admin/solr-search-index.rst new file mode 100644 index 00000000000..a599d9c4daa --- /dev/null +++ b/doc/sphinx-guides/source/admin/solr-search-index.rst @@ -0,0 +1,47 @@ +Solr Search Index +================= + +Dataverse requires Solr to be operational at all times. If you stop Solr, you should see a error about this on the home page, which is powered by the search index Solr provides. You can set up Solr by following the steps in our Installation Guide's :doc:`/installation/prerequisites` and :doc:`/installation/config` sections explaining how to configure it. This section you're reading now is about the care and feeding of the search index. PostgreSQL is the "source of truth" and the Dataverse application will copy data from PostgreSQL into Solr. For this reason, the search index can be rebuilt at any time. Depending on the amount of data you have, this can be a slow process. You are encouraged to experiment with production data to get a sense of how long a full reindexing will take. + +.. contents:: Contents: + :local: + +Full Reindex +------------- + +There are two ways to perform a full reindex of the Dataverse search index. Starting with a "clear" ensures a completely clean index but involves downtime. Reindexing in place doesn't involve downtime but does not ensure a completely clean index. + +Clear and Reindex ++++++++++++++++++ + +Clearing Data from Solr +~~~~~~~~~~~~~~~~~~~~~~~ + +Please note that the moment you issue this command, it will appear to end users looking at the home page that all data is gone! This is because the home page is powered by the search index. + +``curl http://localhost:8080/api/admin/index/clear`` + +Start Async Reindex +~~~~~~~~~~~~~~~~~~~ + +Please note that this operation may take hours depending on the amount of data in your system. This known issue is being tracked at https://github.com/IQSS/dataverse/issues/50 + +``curl http://localhost:8080/api/admin/index`` + +Reindex in Place ++++++++++++++++++ + +An alternative to completely clearing the search index is to reindex in place. + +Clear Index Timestamps +~~~~~~~~~~~~~~~~~~~~~~ + +``curl -X DELETE http://localhost:8080/api/admin/index/timestamps`` + +Start or Continue Async Reindex +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If indexing stops, this command should pick up where it left off based on which index timestamps have been set, which is why we start by clearing these timestamps above. These timestamps are stored in the ``dvobject`` database table. + +``curl http://localhost:8080/api/admin/index/continue`` + diff --git a/doc/sphinx-guides/source/admin/troubleshooting.rst b/doc/sphinx-guides/source/admin/troubleshooting.rst index 792d1067c56..fb7ed8a8326 100644 --- a/doc/sphinx-guides/source/admin/troubleshooting.rst +++ b/doc/sphinx-guides/source/admin/troubleshooting.rst @@ -3,10 +3,21 @@ Troubleshooting =============== -.. contents:: :local: - This new (as of v.4.6) section of the Admin guide is for tips on how to diagnose and fix system problems. +.. contents:: Contents: + :local: + +Glassfish +--------- + +``server.log`` is the main place to look when you encounter problems. Hopefully an error message has been logged. If there's a stack trace, it may be of interest to developers, especially they can trace line numbers back to a tagged version. + +For debugging purposes, you may find it helpful to increase logging levels as mentioned in the :doc:`/developers/debugging` section of the Developer Guide. + +Our guides focus on using the command line to manage Glassfish but you might be interested in an admin GUI at http://localhost:4848 + + Deployment fails, "EJB Timer Service not available" --------------------------------------------------- diff --git a/doc/sphinx-guides/source/admin/user-administration.rst b/doc/sphinx-guides/source/admin/user-administration.rst new file mode 100644 index 00000000000..370947547d3 --- /dev/null +++ b/doc/sphinx-guides/source/admin/user-administration.rst @@ -0,0 +1,39 @@ +User Administration +=================== + +This section focuses on user administration tools and tasks. + +.. contents:: Contents: + :local: + +Manage Users Table +------------------ + +The Manage Users table gives the network administrator a list of all user accounts in table form. It lists username, full name, email address, and whether or not the user has Superuser status. + +Usernames are listed alphabetically and clicking on a username takes you to the account page that contains detailed information on that account. + +You can access the Manage Users table by clicking the "Manage Users" button on the Dashboard, which is linked from the header of all Dataverse pages (if you're loggied in as an administrator). + +Confirm Email +------------- + +Dataverse encourages builtin/local users to verify their email address upon signup or email change so that sysadmins can be assured that users can be contacted. + +The app will send a standard welcome email with a URL the user can click, which, when activated, will store a ``lastconfirmed`` timestamp in the ``authenticateduser`` table of the database. Any time this is "null" for a user (immediately after signup and/or changing of their Dataverse email address), their current email on file is considered to not be verified. The link that is sent expires after a time (the default is 24 hours), but this is configurable by a superuser via the ``:MinutesUntilConfirmEmailTokenExpires`` config option. + +Should users' URL token expire, they will see a "Verify Email" button on the account information page to send another URL. + +Sysadmins can determine which users have verified their email addresses by looking for the presence of the value ``emailLastConfirmed`` in the JSON output from listing users (see the "Admin" section of the :doc:`/api/native-api`). As mentioned in the :doc:`/user/account` section of the User Guide, the email addresses for Shibboleth users are re-confirmed on every login. + +Deleting an API Token +--------------------- + +If an API token is compromised it should be deleted. Users can generate a new one for themselves as explained in the :doc:`/user/account` section of the User Guide, but you may want to preemptively delete tokens from the database. + +Using the API token 7ae33670-be21-491d-a244-008149856437 as an example: + +``delete from apitoken where tokenstring = '7ae33670-be21-491d-a244-008149856437';`` + +You should expect the output ``DELETE 1`` after issuing the command above. + diff --git a/doc/sphinx-guides/source/installation/administration.rst b/doc/sphinx-guides/source/installation/administration.rst deleted file mode 100644 index b08b510c434..00000000000 --- a/doc/sphinx-guides/source/installation/administration.rst +++ /dev/null @@ -1,103 +0,0 @@ -Administration -============== - -This section focuses on system and database administration tasks. Please see the :doc:`/user/index` for tasks having to do with having the "Admin" role on a dataverse or dataset. - -.. contents:: :local: - -Solr Search Index ------------------ - -Dataverse requires Solr to be operational at all times. If you stop Solr, you should see a error about this on the home page, which is powered by the search index Solr provides. You set up Solr by following the steps in the :doc:`prerequisites` section and the :doc:`config` section explained how to configure it. This section is about the care and feeding of the search index. PostgreSQL is the "source of truth" and the Dataverse application will copy data from PostgreSQL into Solr. For this reason, the search index can be rebuilt at any time but depending on the amount of data you have, this can be a slow process. You are encouraged to experiment with production data to get a sense of how long a full reindexing will take. - -Full Reindex -++++++++++++ - -There are two ways to perform a full reindex of the Dataverse search index. Starting with a "clear" ensures a completely clean index but involves downtime. Reindexing in place doesn't involve downtime but does not ensure a completely clean index. - -Clear and Reindex -~~~~~~~~~~~~~~~~~ - -Clearing Data from Solr -....................... - -Please note that the moment you issue this command, it will appear to end users looking at the home page that all data is gone! This is because the home page is powered by the search index. - -``curl http://localhost:8080/api/admin/index/clear`` - -Start Async Reindex -................... - -Please note that this operation may take hours depending on the amount of data in your system. This known issue is being tracked at https://github.com/IQSS/dataverse/issues/50 - -``curl http://localhost:8080/api/admin/index`` - -Reindex in Place -~~~~~~~~~~~~~~~~ - -An alternative to completely clearing the search index is to reindex in place. - -Clear Index Timestamps -...................... - -``curl -X DELETE http://localhost:8080/api/admin/index/timestamps`` - -Start or Continue Async Reindex -................................ - -If indexing stops, this command should pick up where it left off based on which index timestamps have been set, which is why we start by clearing these timestamps above. These timestamps are stored in the ``dvobject`` database table. - -``curl http://localhost:8080/api/admin/index/continue`` - -Glassfish ---------- - -``server.log`` is the main place to look when you encounter problems. Hopefully an error message has been logged. If there's a stack trace, it may be of interest to developers, especially they can trace line numbers back to a tagged version. - -For debugging purposes, you may find it helpful to increase logging levels as mentioned in the :doc:`/developers/debugging` section of the Developer Guide. - -This guide has focused on using the command line to manage Glassfish but you might be interested in an admin GUI at http://localhost:4848 - -Monitoring ----------- - -In production you'll want to monitor the usual suspects such as CPU, memory, free disk space, etc. - -https://github.com/IQSS/dataverse/issues/2595 contains some information on enabling monitoring of Glassfish, which is disabled by default. - -There is a database table called ``actionlogrecord`` that captures events that may be of interest. See https://github.com/IQSS/dataverse/issues/2729 for more discussion around this table. - -Maintenance ------------ - -When you have scheduled down time for your production servers, we provide a :download:`sample maintenance page <../_static/installation/files/etc/maintenance/maintenance.xhtml>` for you to use. To download, right-click and select "Save Link As". - -The maintenance page is intended to be a static page served by Apache to provide users with a nicer, more informative experience when the site is unavailable. - -User Administration -------------------- - -There isn't much in the way of user administration tools built in to Dataverse. - -Confirm Email -+++++++++++++ - -Dataverse encourages builtin/local users to verify their email address upon signup or email change so that sysadmins can be assured that users can be contacted. - -The app will send a standard welcome email with a URL the user can click, which, when activated, will store a ``lastconfirmed`` timestamp in the ``authenticateduser`` table of the database. Any time this is "null" for a user (immediately after signup and/or changing of their Dataverse email address), their current email on file is considered to not be verified. The link that is sent expires after a time (the default is 24 hours), but this is configurable by a superuser via the ``:MinutesUntilConfirmEmailTokenExpires`` config option. - -Should users' URL token expire, they will see a "Verify Email" button on the account information page to send another URL. - -Sysadmins can determine which users have verified their email addresses by looking for the presence of the value ``emailLastConfirmed`` in the JSON output from listing users (see the "Admin" section of the :doc:`/api/native-api`). As mentioned in the :doc:`/user/account` section of the User Guide, the email addresses for Shibboleth users are re-confirmed on every login. - -Deleting an API Token -+++++++++++++++++++++ - -If an API token is compromised it should be deleted. Users can generate a new one for themselves as explained in the :doc:`/user/account` section of the User Guide, but you may want to preemptively delete tokens from the database. - -Using the API token 7ae33670-be21-491d-a244-008149856437 as an example: - -``delete from apitoken where tokenstring = '7ae33670-be21-491d-a244-008149856437';`` - -You should expect the output ``DELETE 1`` after issuing the command above. - diff --git a/doc/sphinx-guides/source/installation/config.rst b/doc/sphinx-guides/source/installation/config.rst index 368cfaa7c78..ee722845ea7 100644 --- a/doc/sphinx-guides/source/installation/config.rst +++ b/doc/sphinx-guides/source/installation/config.rst @@ -6,7 +6,7 @@ Now that you've successfully logged into Dataverse with a superuser account afte Settings within Dataverse itself are managed via JVM options or by manipulating values in the ``setting`` table directly or through API calls. Configuring Solr requires manipulating XML files. -Once you have finished securing and configuring your Dataverse installation, proceed to the :doc:`administration` section. Advanced configuration topics are covered in the :doc:`r-rapache-tworavens`, :doc:`shibboleth` and :doc:`oauth2` sections. +Once you have finished securing and configuring your Dataverse installation, you may proceed to the :doc:`/admin/index` for more information on the ongoing administration of a Dataverse installation. Advanced configuration topics are covered in the :doc:`r-rapache-tworavens`, :doc:`shibboleth` and :doc:`oauth2` sections. .. contents:: :local: @@ -226,7 +226,7 @@ Congratulations! You've gone live! It's time to announce your new data resposito Administration of Your Dataverse Installation +++++++++++++++++++++++++++++++++++++++++++++ -Now that you're live you'll want to review the :doc:`/admin/index`. Please note that there is also an :doc:`administration` section of this Installation Guide that will be moved to the newer Admin Guide in the future. +Now that you're live you'll want to review the :doc:`/admin/index` for more information about the ongoing administration of a Dataverse installation. JVM Options ----------- @@ -661,7 +661,7 @@ Allow for migration of non-conformant data (especially dates) from DVN 3.x to Da :MinutesUntilConfirmEmailTokenExpires +++++++++++++++++++++++++++++++++++++ -The duration in minutes before "Confirm Email" URLs expire. The default is 1440 minutes (24 hours). See also :doc:`/installation/administration`. +The duration in minutes before "Confirm Email" URLs expire. The default is 1440 minutes (24 hours). See also the :doc:`/admin/user-administration` section of our Admin Guide. :DefaultAuthProvider ++++++++++++++++++++ diff --git a/doc/sphinx-guides/source/installation/index.rst b/doc/sphinx-guides/source/installation/index.rst index 469bb75a481..010177f1d10 100755 --- a/doc/sphinx-guides/source/installation/index.rst +++ b/doc/sphinx-guides/source/installation/index.rst @@ -15,7 +15,6 @@ Contents: prerequisites installation-main config - administration upgrading r-rapache-tworavens geoconnect diff --git a/doc/sphinx-guides/source/installation/shibboleth.rst b/doc/sphinx-guides/source/installation/shibboleth.rst index 49ba9868f19..32e9aff8aec 100644 --- a/doc/sphinx-guides/source/installation/shibboleth.rst +++ b/doc/sphinx-guides/source/installation/shibboleth.rst @@ -348,7 +348,7 @@ If you have more than one Glassfish server, you should use the same ``sp-cert.pe Debugging --------- -The :doc:`administration` section explains how to increase Glassfish logging levels. The relevant classes and packages are: +The :doc:`/admin/troubleshooting` section of the Admin Guide explains how to increase Glassfish logging levels. The relevant classes and packages are: - edu.harvard.iq.dataverse.Shib - edu.harvard.iq.dataverse.authorization.providers.shib