Merge branch 'develop' into 4393-guestbook-not-enforcing-reqd-fields

doc/sphinx-guides/source/_static/admin/ipGroupAll.json

@@ -0,0 +1,14 @@
+{
+  "alias": "ipGroupAll",
+  "name": "IP group to match all IPv4 and IPv6 addresses",
+  "ranges": [
+    [
+      "0.0.0.0",
+      "255.255.255.255"
+    ],
+    [
+      "::",
+      "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"
+    ]
+  ]
+}

doc/sphinx-guides/source/admin/index.rst

@@ -21,6 +21,7 @@ These "superuser" tasks are managed via the new page called the Dashboard. A use
    geoconnect-worldmap
    user-administration
    solr-search-index
+   ip-groups
    monitoring
    maintenance
    troubleshooting

doc/sphinx-guides/source/admin/ip-groups.rst

@@ -0,0 +1,43 @@
+IP Groups
+=========
+
+IP Groups can be used to permit download of restricted files by IP addresses rather than people. For example, you may want to allow restricted files to be downloaded by researchers who physically enter a library and make use of the library's network.
+
+.. contents:: Contents:
+	:local:
+
+Listing IP Groups
+-----------------
+
+IP Groups can be listed with the following curl command:
+
+``curl http://localhost:8080/api/admin/groups/ip``
+
+Creating an IP Group
+--------------------
+
+IP Groups must be expressed as ranges in IPv4 or IPv6 format. For illustrative purposes, here is a example of the entire IPv4 and IPv6 range that you can :download:`download <../_static/admin/ipGroupAll.json>` and edit to have a narrower range to meet your needs. If you need your IP Group to only encompass a single IP address, you must enter that IP address for the "start" and "end" of the range. If you don't use IPv6 addresses, you can delete that section of the JSON. Please note that the "alias" must be unique if you define multiple IP Groups. You should give it a meaningful "name" since both "alias" and "name" will appear and be searchable in the GUI when your users are assigning roles.
+
+.. literalinclude:: ../_static/admin/ipGroupAll.json
+
+Let's say you download the example above and edit it to give it a range used by your library, giving it a filename of ``ipGroup1.json`` and putting it in the ``/tmp`` directory. Next, load it into Dataverse using the following curl command:
+
+``curl -X POST -H 'Content-type: application/json' http://localhost:8080/api/admin/groups/ip --upload-file /tmp/ipGroup1.json``
+
+Note that you can update a group the same way, as long as you use the same alias.
+
+Listing an IP Group
+--------------------
+
+Let's say you used "ipGroup1" as the alias of the IP Group you created above. To list just that IP Group, you can include the alias in the curl command like this:
+
+``curl http://localhost:8080/api/admin/groups/ip/ipGroup1``
+
+Deleting an IP Group
+--------------------
+
+It is not recommended to delete an IP Group that has been assigned roles. If you want to delete an IP Group, you should first remove its permissions.
+
+To delete an IP Group with an alias of "ipGroup1", use the curl command below:
+
+``curl -X DELETE http://localhost:8080/api/admin/groups/ip/ipGroup1``

doc/sphinx-guides/source/api/native-api.rst

@@ -770,30 +770,6 @@ List a role assignee (i.e. a user or a group)::
 
 The ``$identifier`` should start with an ``@`` if it's a user. Groups start with ``&``. "Built in" users and groups start with ``:``. Private URL users start with ``#``.
 
-IpGroups
-^^^^^^^^
-
-Lists all the ip groups::
-
-  GET http://$SERVER/api/admin/groups/ip
-
-Adds a new ip group. POST data should specify the group in JSON format. Examples are available at the ``data`` folder. Using this method, an IP Group is always created, but its ``alias`` might be different than the one appearing in the
-JSON file, to ensure it is unique. ::
-
-  POST http://$SERVER/api/admin/groups/ip
-
-Creates or updates the ip group ``$groupAlias``. ::
-
-    POST http://$SERVER/api/admin/groups/ip/$groupAlias
-
-Returns a the group in a JSON format. ``$groupIdtf`` can either be the group id in the database (in case it is numeric), or the group alias. ::
-
-  GET http://$SERVER/api/admin/groups/ip/$groupIdtf
-
-Deletes the group specified by ``groupIdtf``. ``groupIdtf`` can either be the group id in the database (in case it is numeric), or the group alias. Note that a group can be deleted only if there are no roles assigned to it. ::
-
-  DELETE http://$SERVER/api/admin/groups/ip/$groupIdtf
-
 Saved Search
 ^^^^^^^^^^^^
 

doc/sphinx-guides/source/user/dataset-management.rst

@@ -246,27 +246,31 @@ This is where you will enable a particular Guestbook for your dataset, which is
 Roles & Permissions
 =====================
 
+Dataverse user accounts can be granted roles that define which actions they are allowed to take on specific dataverses, datasets, and/or files. Each role comes with a set of permissions, which define the specific actions that users may take.
+
+Roles and permissions may also be granted to groups. Groups can be defined as a collection of Dataverse user accounts, a collection of IP addresses (e.g. all users of a library's computers), or a collection of all users who log in using a particular institutional login (e.g. everyone who logs in with a particular university's account credentials).
+
 Dataset-Level 
 -------------
 
 Admins or curators of a dataset can assign roles and permissions to the users of that dataset. If you are an admin or curator of a dataset, then you can get to the dataset permissions page by clicking the "Edit" button, highlighting "Permissions" from the dropdown list, and clicking "Dataset".
 
 When you access a dataset's permissions page, you will see two sections:
 
-**Users/Groups:** Here you can assign roles to specific users or groups of users, determining which actions they are permitted to take on your dataset. You can also reference a list of all users who have roles assigned to them for your dataset and remove their roles if you please. Some of the users listed may have roles assigned at the dataverse level, in which case those roles can only be removed from the dataverse permissions page.
+**Users/Groups:** Here you can assign roles to specific users or groups, determining which actions they are permitted to take on your dataset. You can also reference a list of all users who have roles assigned to them for your dataset and remove their roles if you please. Some of the users listed may have roles assigned at the dataverse level, in which case those roles can only be removed from the dataverse permissions page.
 
 **Roles:** Here you can reference a full list of roles that can be assigned to users of your dataset. Each role lists the permissions that it offers.
 
 File-Level
 ----------
 
-If you have restricted access to specific files in your dataset, you can grant specific users or groups access to those files while still keeping them restricted to the general public. If you are an admin or curator of a dataset, then you can get to the file-level permissions page by clicking the "Edit" button, highlighting "Permissions" from the dropdown list, and clicking "File".
+If specific files in your dataset are restricted access, then you can grant specific users or groups access to those files while still keeping them restricted to the general public. If you are an admin or curator of a dataset, then you can get to the file-level permissions page by clicking the "Edit" button, highlighting "Permissions" from the dropdown list, and clicking "File".
 
 When you access a dataset's file-level permissions page, you will see two sections:
 
 **Users/Groups:** Here you can see which users or groups have been granted access to which files. You can click the "Grant Access to Users/Groups" button to see a box where you can grant access to specific files within your dataset to specific users or groups. If any users have requested access to a file in your dataset, you can grant or reject their access request here.
 
-**Restricted Files:** In this section, you can see the same information, but broken down by each individual file in your dataset. For each file, you can click the "Assign Access" button to see a box where you can grant access to that file to specific users.
+**Restricted Files:** In this section, you can see the same information, but broken down by each individual file in your dataset. For each file, you can click the "Assign Access" button to see a box where you can grant access to that file to specific users or groups.
 
 .. _thumbnails-widgets:
 

doc/sphinx-guides/source/user/dataverse-management.rst

@@ -92,6 +92,10 @@ Adding Widgets to an OpenScholar Website
 
 Roles & Permissions 
 =======================================================
+Dataverse user accounts can be granted roles that define which actions they are allowed to take on specific dataverses, datasets, and/or files. Each role comes with a set of permissions, which define the specific actions that users may take.
+
+Roles and permissions may also be granted to groups. Groups can be defined as a collection of Dataverse user accounts, a collection of IP addresses (e.g. all users of a library's computers), or a collection of all users who log in using a particular institutional login (e.g. everyone who logs in with a particular university's account credentials).
+
 Admins of a dataverse can assign roles and permissions to the users of that dataverse. If you are an admin on a dataverse, then you will find the link to the Permissions page under the Edit dropdown on the dataverse page. 
 
 |image2|
@@ -104,7 +108,7 @@ When you access a dataverse's permissions page, you will see three sections:
 
 **Permissions:** Here you can decide the requirements that determine which types of users can add datasets and sub dataverses to your dataverse, and what permissions they'll be granted when they do so.
 
-**Users/Groups:** Here you can assign roles to specific users or groups of users, determining which actions they are permitted to take on your dataverse. You can also reference a list of all users who have roles assigned to them for your dataverse and remove their roles if you please.
+**Users/Groups:** Here you can assign roles to specific users or groups, determining which actions they are permitted to take on your dataverse. You can also reference a list of all users who have roles assigned to them for your dataverse and remove their roles if you please.
 
 **Roles:** Here you can reference a full list of roles that can be assigned to users of your dataverse. Each role lists the permissions that it offers.
 

src/main/java/edu/harvard/iq/dataverse/FileDownloadHelper.java

@@ -39,6 +39,9 @@ public class FileDownloadHelper implements java.io.Serializable {
     private static final Logger logger = Logger.getLogger(FileDownloadHelper.class.getCanonicalName());
     @Inject
     DataverseSession session;
+
+    @Inject
+    DataverseRequestServiceBean dvRequestService;
 
     @EJB
     PermissionServiceBean  permissionService;
@@ -90,6 +93,8 @@ public void setPositionField(UIInput positionField) {
     UIInput positionField;
 
 
+
+
     private final Map<Long, Boolean> fileDownloadPermissionMap = new HashMap<>(); // { FileMetadata.id : Boolean } 
 
      public void nameValueChangeListener(AjaxBehaviorEvent e) {
@@ -327,26 +332,17 @@ public boolean canDownloadFile(FileMetadata fileMetadata){
             return false;
         } 
 
-        // --------------------------------------------------------------------        
-        // Grab the fileMetadata.id and restriction flag                
-        // --------------------------------------------------------------------
         Long fid = fileMetadata.getId();
         //logger.info("calling candownloadfile on filemetadata "+fid);
+        // Note that `isRestricted` at the FileMetadata level is for expressing intent by version. Enforcement is done with `isRestricted` at the DataFile level.
         boolean isRestrictedFile = fileMetadata.isRestricted();
 
-        // --------------------------------------------------------------------
         // Has this file been checked? Look at the DatasetPage hash
-        // --------------------------------------------------------------------
         if (this.fileDownloadPermissionMap.containsKey(fid)){
             // Yes, return previous answer
             //logger.info("using cached result for candownloadfile on filemetadata "+fid);
             return this.fileDownloadPermissionMap.get(fid);
         }
-        //----------------------------------------------------------------------
-        //(0) Before we do any testing - if version is deaccessioned and user
-        // does not have edit dataset permission then may download
-        //----------------------------------------------------------------------
-
        if (fileMetadata.getDatasetVersion().isDeaccessioned()) {
            if (this.doesSessionUserHavePermission(Permission.EditDataset, fileMetadata)) {
                // Yes, save answer and return true
@@ -358,66 +354,20 @@ public boolean canDownloadFile(FileMetadata fileMetadata){
            }
        }
 
-        // --------------------------------------------------------------------
-        // (1) Is the file Unrestricted ?        
-        // --------------------------------------------------------------------
         if (!isRestrictedFile){
             // Yes, save answer and return true
             this.fileDownloadPermissionMap.put(fid, true);
             return true;
         }
 
-        // --------------------------------------------------------------------
-        // Conditions (2) through (4) are for Restricted files
-        // --------------------------------------------------------------------
-
-        // --------------------------------------------------------------------
-        // (2) In Dataverse 4.3 and earlier we required that users be authenticated
-        // to download files, but in developing the Private URL feature, we have
-        // added a new subclass of "User" called "PrivateUrlUser" that returns false
-        // for isAuthenticated but that should be able to download restricted files
-        // when given the Member role (which includes the DownloadFile permission).
-        // This is consistent with how Builtin and Shib users (both are
-        // AuthenticatedUsers) can download restricted files when they are granted
-        // the Member role. For this reason condition 2 has been changed. Previously,
-        // we required isSessionUserAuthenticated to return true. Now we require
-        // that the User is not an instance of GuestUser, which is similar in
-        // spirit to the previous check.
-        // --------------------------------------------------------------------
-
-        if (session.getUser() instanceof GuestUser){
-            this.fileDownloadPermissionMap.put(fid, false);
-            return false;
-        }
-
-
-        // --------------------------------------------------------------------
-        // (3) Does the User have DownloadFile Permission at the **Dataset** level 
-        // --------------------------------------------------------------------
-
-
-        if (this.doesSessionUserHavePermission(Permission.DownloadFile, fileMetadata)){
-            // Yes, save answer and return true
+        // See if the DataverseRequest, which contains IP Groups, has permission to download the file.
+        if (permissionService.requestOn(dvRequestService.getDataverseRequest(), fileMetadata.getDataFile()).has(Permission.DownloadFile)) {
+            logger.fine("The DataverseRequest (User plus IP address) has access to download the file.");
             this.fileDownloadPermissionMap.put(fid, true);
             return true;
         }
 
-
-        // --------------------------------------------------------------------
-        // (4) Does the user has DownloadFile permission on the DataFile            
-        // --------------------------------------------------------------------
-        /*
-        if (this.permissionService.on(fileMetadata.getDataFile()).has(Permission.DownloadFile)){
-            this.fileDownloadPermissionMap.put(fid, true);
-            return true;
-        }
-        */
-
-        // --------------------------------------------------------------------
-        // (6) No download....
-        // --------------------------------------------------------------------
         this.fileDownloadPermissionMap.put(fid, false);
-
         return false;
     }
 

src/main/java/edu/harvard/iq/dataverse/FileMetadata.java

@@ -64,6 +64,13 @@ public class FileMetadata implements Serializable {
     @Column(columnDefinition = "TEXT")
     private String description = "";
 
+    /**
+     * At the FileMetadata level, "restricted" is a historical indication of the
+     * data owner's intent for the file by version. Permissions are actually
+     * enforced based on the "restricted" boolean at the *DataFile* level. On
+     * publish, the latest intent is copied from the FileMetadata level to the
+     * DataFile level.
+     */
     @Expose
     private boolean restricted;
 

src/main/java/edu/harvard/iq/dataverse/PermissionServiceBean.java

@@ -211,17 +211,15 @@ public Set<Permission> permissionsFor( DataverseRequest req, DvObject dvo ) {
 
         // Add permissions specifically given to the user
         permissions.addAll( permissionsForSingleRoleAssignee(req.getUser(),dvo) );
-
-        /*
+
         Set<Group> groups = groupService.groupsFor(req,dvo);
 
         // Add permissions gained from groups
         for ( Group g : groups ) {
             final Set<Permission> groupPremissions = permissionsForSingleRoleAssignee(g,dvo);
             permissions.addAll(groupPremissions);
         }
-        */
-
+
         if ( ! req.getUser().isAuthenticated() ) {
             permissions.removeAll( PERMISSIONS_FOR_AUTHENTICATED_USERS_ONLY );
         }