diff --git a/doc/release-notes/7409-remove-worldmap.md b/doc/release-notes/7409-remove-worldmap.md new file mode 100644 index 00000000000..3e6ea3ed06e --- /dev/null +++ b/doc/release-notes/7409-remove-worldmap.md @@ -0,0 +1,9 @@ +## Release Highlights + +### Worldmap/Geoconnect Integration Now Obsolete + +As of this release, the Geoconnect/Worldmap integration is no longer available. The Harvard University Worldmap is going through a migration process, and instead of updating this code to work with the new infrastructure, the decision was made to pursue future Geospatial exploration/analysis through other tools, following the External Tools Framework in the Dataverse Software. + +## Notes to Admins + +Take a backup of the Worldmap links, if any. \ No newline at end of file diff --git a/doc/release-notes/7502-more-mime-types.md b/doc/release-notes/7502-more-mime-types.md new file mode 100644 index 00000000000..0c57dc3e389 --- /dev/null +++ b/doc/release-notes/7502-more-mime-types.md @@ -0,0 +1,7 @@ +## Upgrade Steps + +In addition to mapping friendly names to these file types, the types are further mapped to aggregate file types facets on the homepage. A full reindex is required for the facets to be refreshed. + +Kick off full reindex + +http://guides.dataverse.org/en/4.20/admin/solr-search-index.html diff --git a/doc/sphinx-guides/source/admin/geoconnect-worldmap.rst b/doc/sphinx-guides/source/admin/geoconnect-worldmap.rst deleted file mode 100644 index fb8250c0650..00000000000 --- a/doc/sphinx-guides/source/admin/geoconnect-worldmap.rst +++ /dev/null @@ -1,52 +0,0 @@ -Geoconnect and WorldMap -======================= - -One of the optional components listed under "Architecture and Components" in the :doc:`/installation/prep` section of the Installation Guide is `Geoconnect `_, a piece of middleware that allows Dataverse users to create maps in `WorldMap `_ based on geospatial data stored in Dataverse. For more details on the feature from the user perspective, see the :doc:`/user/data-exploration/worldmap` section of the User Guide. - -.. contents:: |toctitle| - :local: - -Update "mapitlink" ------------------- - -SQL commands to point a Dataverse installation at different Geoconnect servers: - - -**Geoconnect Production** *geoconnect.datascience.iq.harvard.edu* - -.. code-block:: sql - - update worldmapauth_tokentype set mapitlink = 'https://geoconnect.datascience.iq.harvard.edu/shapefile/map-it', hostname='geoconnect.datascience.iq.harvard.edu' where name = 'GEOCONNECT'; - -**Heroku Test** *geoconnect-dev.herokuapp.com* - -.. code-block:: sql - - update worldmapauth_tokentype set mapitlink = 'https://geoconnect-dev.herokuapp.com/shapefile/map-it', hostname='geoconnect-dev.herokuapp.com' where name = 'GEOCONNECT'; - - -**View Current Settings** - -.. code-block:: sql - - SELECT * from worldmapauth_tokentype; - - -Removing Dead Explore Links ---------------------------- - -After a map has been created in WorldMap (assuming all the setup has been done), in Dataverse the file will display WorldMap as an explore option. In rare occasions, the map has been deleted on the WorldMap resulting in a dead link, a 404 page not found error. - -Functionality has been added on the Dataverse side to iterate through all the maps Dataverse knows about (stored in the ``maplayermetadata`` database table) and to check for the existence of each map in WorldMap. The status code returned from WorldMap (200, 404, etc.) is recorded in Dataverse along with a timestamp of when the check was performed. To perform this check, you can execute the following ``curl`` command: - -``curl -X POST http://localhost:8080/api/admin/geoconnect/mapLayerMetadatas/check`` - -The output above will contain the ``layerLink`` being checked as well as the HTTP response status code (200, 404, etc.) in the ``lastVerifiedStatus`` field. 200 means OK and 404 means not found. 500 might indicate that the map is only temporarily unavailable. The ``lastVerifiedStatus`` and ``lastVerifiedTime`` will be persisted to the ``maplayermetadata`` database table. - -Armed with this information about WorldMap returning a 404 for a map, you may want to delete any record of the map on the Dataverse side removing WorldMap as an explore option (and so that thumbnail files are cleaned up). To accomplish this, use the following ``curl`` command, substituting the id of the file: - -``curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE http://localhost:8080/api/files/{file_id}/map`` - -End users can also run the ``DELETE`` command above (if they have permission to edit the dataset) but it's more likely that the sysadmin reading this guide will run it for them. It is recommended that you add the "check" command above to cron so that Dataverse periodically checks on the status of maps in WorldMap. In addition to the "check all maps" command above, you can also check an individual map with the following ``curl`` command, substituting the id of the row from the ``maplayermetadata`` table: - -``curl -X POST http://localhost:8080/api/admin/geoconnect/mapLayerMetadatas/check/{id}`` diff --git a/doc/sphinx-guides/source/admin/index.rst b/doc/sphinx-guides/source/admin/index.rst index 55733ffd99a..b97d9161d50 100755 --- a/doc/sphinx-guides/source/admin/index.rst +++ b/doc/sphinx-guides/source/admin/index.rst @@ -21,7 +21,6 @@ This guide documents the functionality only available to superusers (such as "da timers make-data-count integrations - geoconnect-worldmap user-administration dataverses-datasets solr-search-index diff --git a/doc/sphinx-guides/source/admin/integrations.rst b/doc/sphinx-guides/source/admin/integrations.rst index 9e43081bb45..bea210e587e 100644 --- a/doc/sphinx-guides/source/admin/integrations.rst +++ b/doc/sphinx-guides/source/admin/integrations.rst @@ -85,13 +85,6 @@ TwoRavens is a web application for tabular data exploration and statistical anal For installation instructions, see the :doc:`external-tools` section. -WorldMap -++++++++ - -WorldMap helps researchers visualize and explore geospatial data by creating maps. - -For installation instructions, see :doc:`geoconnect-worldmap`. - Compute Button ++++++++++++++ diff --git a/doc/sphinx-guides/source/api/apps.rst b/doc/sphinx-guides/source/api/apps.rst index dca641b1feb..4429da9e306 100755 --- a/doc/sphinx-guides/source/api/apps.rst +++ b/doc/sphinx-guides/source/api/apps.rst @@ -65,13 +65,6 @@ OSF allows you to view, download, and upload files to and from a Dataverse datas https://github.com/CenterForOpenScience/osf.io/tree/develop/addons/dataverse -Geoconnect -~~~~~~~~~~ - -Geoconnect allows Dataverse files to be visualized and explored on `WorldMap `_. Read more about it in the :doc:`/user/data-exploration/worldmap` section of the User Guide. - -https://github.com/IQSS/geoconnect - dataverse-metrics ~~~~~~~~~~~~~~~~~ diff --git a/doc/sphinx-guides/source/api/intro.rst b/doc/sphinx-guides/source/api/intro.rst index 6ce77f5fe7d..3bbd117f68e 100755 --- a/doc/sphinx-guides/source/api/intro.rst +++ b/doc/sphinx-guides/source/api/intro.rst @@ -198,7 +198,6 @@ Please note that some APIs are only documented in other guides that are more sui - :doc:`/admin/metadatacustomization` - :doc:`/admin/metadataexport` - :doc:`/admin/make-data-count` - - :doc:`/admin/geoconnect-worldmap` - :doc:`/admin/solr-search-index` - Installation Guide diff --git a/doc/sphinx-guides/source/developers/geospatial.rst b/doc/sphinx-guides/source/developers/geospatial.rst index 13680a52358..63ed6fd8a30 100644 --- a/doc/sphinx-guides/source/developers/geospatial.rst +++ b/doc/sphinx-guides/source/developers/geospatial.rst @@ -5,19 +5,12 @@ Geospatial Data .. contents:: |toctitle| :local: -Geoconnect ----------- - -Geoconnect works as a middle layer, allowing geospatial data files in Dataverse to be visualized with Harvard WorldMap. To set up a Geoconnect development environment, you can follow the steps outlined in the `local_setup.md `_ guide. You will need Python and a few other prerequisites. - -As mentioned under the :ref:`architecture` section of Preparation in the Installation Guide, Geoconnect is an optional component of Dataverse, so this section is only necessary to follow it you are working on an issue related to this feature. - How Dataverse Ingests Shapefiles -------------------------------- -A shapefile is a set of files, often uploaded/transferred in ``.zip`` format. This set may contain up to fifteen files. A minimum of three specific files (``.shp``, ``.shx``, ``.dbf``) are needed to be a valid shapefile and a fourth file (``.prj``) is required for WorldMap -- or any type of meaningful visualization. +A shapefile is a set of files, often uploaded/transferred in ``.zip`` format. This set may contain up to fifteen files. A minimum of three specific files (``.shp``, ``.shx``, ``.dbf``) are needed to be a valid shapefile and a fourth file (``.prj``) is required for some applications -- or any type of meaningful visualization. -For ingest and connecting to WorldMap, four files are the minimum required: +For ingest, four files are the minimum required: - ``.shp`` - shape format; the feature geometry itself - ``.shx`` - shape index format; a positional index of the feature geometry to allow seeking forwards and backwards quickly @@ -89,127 +82,6 @@ For two "final" shapefile sets, ``bicycles.zip`` and ``subway_line.zip``, a new - Mimetype: ``application/zipped-shapefile`` - Mimetype Label: "Shapefile as ZIP Archive" -WorldMap JoinTargets + API Endpoint ------------------------------------ - -WorldMap supplies target layers -- or JoinTargets -- that a tabular file may be mapped against. A JSON description of these `CGA `_-curated JoinTargets may be retrieved via API at ``http://worldmap.harvard.edu/datatables/api/jointargets/``. Please note: login is required. You may use any WorldMap account credentials via HTTP Basic Auth. - -Example of JoinTarget information returned via the API: - -.. code-block:: json - - { - "data":[ - { - "layer":"geonode:census_tracts_2010_boston_6f6", - "name":"Census Tracts, Boston (GEOID10: State+County+Tract)", - "geocode_type_slug":"us-census-tract", - "geocode_type":"US Census Tract", - "attribute":{ - "attribute":"CT_ID_10", - "type":"xsd:string" - }, - "abstract":"As of the 2010 census, Boston, MA contains 7,288 city blocks [truncated for example]", - "title":"Census Tracts 2010, Boston (BARI)", - "expected_format":{ - "expected_zero_padded_length":-1, - "is_zero_padded":false, - "description":"Concatenation of state, county and tract for 2010 Census Tracts. Reference: https://www.census.gov/geo/maps-data/data/tract_rel_layout.html\r\n\r\nNote: Across the US, this can be a zero-padded \"string\" but the original Boston layer has this column as \"numeric\" ", - "name":"2010 Census Boston GEOID10 (State+County+Tract)" - }, - "year":2010, - "id":28 - }, - { - "layer":"geonode:addresses_2014_boston_1wr", - "name":"Addresses, Boston", - "geocode_type_slug":"boston-administrative-geography", - "geocode_type":"Boston, Administrative Geography", - "attribute":{ - "attribute":"LocationID", - "type":"xsd:int" - }, - "abstract":"Unique addresses present in the parcels data set, which itself is derived from [truncated for example]", - "title":"Addresses 2015, Boston (BARI)", - "expected_format":{ - "expected_zero_padded_length":-1, - "is_zero_padded":false, - "description":"Boston, Administrative Geography, Boston Address Location ID. Example: 1, 2, 3...nearly 120000", - "name":"Boston Address Location ID (integer)" - }, - "year":2015, - "id":18 - }, - { - "layer":"geonode:bra_neighborhood_statistical_areas_2012__ug9", - "name":"BRA Neighborhood Statistical Areas, Boston", - "geocode_type_slug":"boston-administrative-geography", - "geocode_type":"Boston, Administrative Geography", - "attribute":{ - "attribute":"BOSNA_R_ID", - "type":"xsd:double" - }, - "abstract":"BRA Neighborhood Statistical Areas 2015, Boston. Provided by [truncated for example]", - "title":"BRA Neighborhood Statistical Areas 2015, Boston (BARI)", - "expected_format":{ - "expected_zero_padded_length":-1, - "is_zero_padded":false, - "description":"Boston, Administrative Geography, Boston BRA Neighborhood Statistical Area ID (integer). Examples: 1, 2, 3, ... 68, 69", - "name":"Boston BRA Neighborhood Statistical Area ID (integer)" - }, - "year":2015, - "id":17 - } - ], - "success":true - } - -How Geoconnect Uses Join Target Information -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -When a user attempts to map a tabular file, the application looks in the Geoconnect database for ``JoinTargetInformation``. If this information is more than 10 minutes* old, the application will retrieve fresh information and save it to the db. - -(* Change the timing via the Django settings variable ``JOIN_TARGET_UPDATE_TIME``.) - -This JoinTarget info is used to populate HTML forms used to match a tabular file column to a JoinTarget column. Once a JoinTarget is chosen, the JoinTarget ID is an essential piece of information used to make an API call to the WorldMap and attempt to map the file. - -Retrieving Join Target Information from WorldMap API -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``get_join_targets()`` function in ``dataverse_layer_services.py`` uses the WorldMap API, retrieves a list of available tabular file JointTargets. (See the `dataverse_layer_services code in GitHub `_.) - -Saving Join Target Information to Geoconnect Database -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``get_latest_jointarget_information()`` in ``utils.py`` retrieves recent JoinTarget Information from the database. (See the `utils code in GitHub `_.) - -Setting Up WorldMap Sample Data -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For the dataset page, this script gives a query to add test WorldMap map data. After the query is run, the explore in WorldMap option should display for tabular files or shapefiles. In the example SQL queries below, substitute ``$DATASET_ID`` and ``$DATAFILE_ID`` with the appropriate ID's. - -To add sample map data for a tabular file: - -.. code:: - - INSERT INTO maplayermetadata (id, isjoinlayer, joindescription, embedmaplink, layerlink, layername, mapimagelink, worldmapusername, dataset_id, datafile_id) - VALUES (DEFAULT, true, 'This file was joined with WorldMap layer x, y, z', - 'https://worldmap.harvard.edu/maps/embed/?layer=geonode:zip_codes_2015_zip_s9i','https://worldmap.harvard.edu/data/geonode:zip_codes_2015_zip_s9i', - 'geonode:zip_codes_2015_zip_s9i', - 'http://worldmap.harvard.edu/download/wms/27289/png?layers=geonode%3Azip_codes_2015_zip_s9i&width=865&bbox=-71.1911091251%2C42.2270382738%2C-70.9228275369%2C42.3976144794&service=WMS&format=image%2Fpng&srs=EPSG%3A4326&request=GetMap&height=550', - 'admin',$DATASET_ID,$DATAFILE_ID}); - -To add sample map data for a tabular shapefile: - -.. code:: - - INSERT INTO maplayermetadata (id, isjoinlayer, embedmaplink, layerlink, layername, mapimagelink, worldmapusername, dataset_id, datafile_id) - VALUES (DEFAULT, false, - 'https://worldmap.harvard.edu/maps/embed/?layer=geonode:zip_codes_2015_zip_s9i','https://worldmap.harvard.edu/data/geonode:zip_codes_2015_zip_s9i', - 'geonode:zip_codes_2015_zip_s9i', - 'http://worldmap.harvard.edu/download/wms/27289/png?layers=geonode%3Azip_codes_2015_zip_s9i&width=865&bbox=-71.1911091251%2C42.2270382738%2C-70.9228275369%2C42.3976144794&service=WMS&format=image%2Fpng&srs=EPSG%3A4326&request=GetMap&height=550', - 'admin',$DATASET_ID,$DATAFILE_ID); - ---- Previous: :doc:`unf/index` | Next: :doc:`remote-users` diff --git a/doc/sphinx-guides/source/developers/intro.rst b/doc/sphinx-guides/source/developers/intro.rst index e7ed6a12a37..1cb2edf74d0 100755 --- a/doc/sphinx-guides/source/developers/intro.rst +++ b/doc/sphinx-guides/source/developers/intro.rst @@ -64,7 +64,6 @@ As a developer, you also may be interested in these projects related to Datavers - dataverse-metrics - aggregate and visualize metrics for installations of Dataverse around the world: https://github.com/IQSS/dataverse-metrics - Configuration management scripts - Ansible, Puppet, etc.: See :ref:`advanced` section in the Installation Guide. - :doc:`/developers/unf/index` (Java) - a Universal Numerical Fingerprint: https://github.com/IQSS/UNF -- GeoConnect (Python) - create a map by uploading files to Dataverse: https://github.com/IQSS/geoconnect - `DataTags `_ (Java and Scala) - tag datasets with privacy levels: https://github.com/IQSS/DataTags - `TwoRavens `_ (Javascript) - a `d3.js `_ interface for exploring data and running Zelig models: https://github.com/IQSS/TwoRavens - `Zelig `_ (R) - run statistical models on files uploaded to Dataverse: https://github.com/IQSS/Zelig diff --git a/doc/sphinx-guides/source/installation/config.rst b/doc/sphinx-guides/source/installation/config.rst index caa63b1b1cd..6c2f35c0633 100644 --- a/doc/sphinx-guides/source/installation/config.rst +++ b/doc/sphinx-guides/source/installation/config.rst @@ -774,7 +774,7 @@ Once you have created the analytics file, run this curl command to add it to you Tracking Button Clicks ++++++++++++++++++++++ -The basic analytics configuration above tracks page navigation. However, it does not capture potentially interesting events, such as those from users clicking buttons on pages, that do not result in a new page opening. In Dataverse, these events include file downloads, requesting access to restricted data, exporting metadata, social media sharing, requesting citation text, launching external tools or WorldMap, contacting authors, and launching computations. +The basic analytics configuration above tracks page navigation. However, it does not capture potentially interesting events, such as those from users clicking buttons on pages, that do not result in a new page opening. In Dataverse, these events include file downloads, requesting access to restricted data, exporting metadata, social media sharing, requesting citation text, launching external tools, contacting authors, and launching computations. Both Google and Matomo provide the optional capability to track such events and Dataverse has added CSS style classes (btn-compute, btn-contact, btn-download, btn-explore, btn-export, btn-preview, btn-request, btn-share, and downloadCitation) to it's HTML to facilitate it. @@ -1758,20 +1758,6 @@ The ``:TwoRavensUrl`` option is no longer valid. See :doc:`r-rapache-tworavens` The ``:TwoRavensTabularView`` option is no longer valid. See :doc:`r-rapache-tworavens` and the :doc:`/admin/external-tools` section of the Admin Guide. -:GeoconnectCreateEditMaps -+++++++++++++++++++++++++ - -Set ``GeoconnectCreateEditMaps`` to true to allow the user to create maps using Geoconnect. This boolean enables the map configure tool option for a data file and the ingest to create a shape file. - -``curl -X PUT -d true http://localhost:8080/api/admin/settings/:GeoconnectCreateEditMaps`` - -:GeoconnectViewMaps -+++++++++++++++++++ - -Set ``GeoconnectViewMaps`` to true to allow a user to view existing maps. This boolean enables the map explore tool option for a data file. - -``curl -X PUT -d true http://localhost:8080/api/admin/settings/:GeoconnectViewMaps`` - .. _:DatasetPublishPopupCustomText: :DatasetPublishPopupCustomText diff --git a/doc/sphinx-guides/source/installation/geoconnect.rst b/doc/sphinx-guides/source/installation/geoconnect.rst deleted file mode 100644 index 14a12b50a3f..00000000000 --- a/doc/sphinx-guides/source/installation/geoconnect.rst +++ /dev/null @@ -1,17 +0,0 @@ -Geoconnect -========== - -.. contents:: |toctitle| - :local: - -Geoconnect works as a middle layer, allowing geospatial data files in Dataverse to be visualized with Harvard WorldMap. - -To understand the feature from the user perspective, see the :doc:`/user/data-exploration/worldmap` section of the User Guide. - -As of this writing, the README at https://github.com/IQSS/geoconnect recommends not installing Geoconnect at this time due to an ongoing rewrite of the WorldMap code. If you are not deterred by this, read on! - -To set up a Geoconnect development environment, you can follow the steps outlined in the `local_setup.md `_ guide. Although those instructions are for a local development setup, they may assist in installing Geoconnect in your production environment. - -Harvard Dataverse runs Geoconnect on Heroku. To make use of Heroku, you will need a Heroku account, as well as a few other prerequisites. Follow the instructions outlined in the `heroku_setup.md `_ guide. The `heroku.py `_ settings file may also be adapted for other environments. Please note, for the production environment, remember to set ``DEBUG=False``. - -See also the :doc:`/admin/geoconnect-worldmap` section of the Admin Guide. diff --git a/doc/sphinx-guides/source/installation/index.rst b/doc/sphinx-guides/source/installation/index.rst index 8ed0571b466..931c4016394 100755 --- a/doc/sphinx-guides/source/installation/index.rst +++ b/doc/sphinx-guides/source/installation/index.rst @@ -17,7 +17,6 @@ Installation Guide config upgrading r-rapache-tworavens - geoconnect shibboleth oauth2 oidc diff --git a/doc/sphinx-guides/source/installation/installation-main.rst b/doc/sphinx-guides/source/installation/installation-main.rst index 8d289af716a..29600ff1e44 100755 --- a/doc/sphinx-guides/source/installation/installation-main.rst +++ b/doc/sphinx-guides/source/installation/installation-main.rst @@ -90,7 +90,6 @@ For more information, please see https://docs.payara.fish/documentation/payara-s - exporting to Schema.org format (and showing JSON-LD in HTML's tag) - exporting to DDI format - which Dataverse installation an "external tool" should return to -- which Dataverse installation Geoconnect should return to - URLs embedded in SWORD API responses The supplied site URL will be saved under the JVM option :ref:`dataverse.siteUrl`. diff --git a/doc/sphinx-guides/source/installation/prep.rst b/doc/sphinx-guides/source/installation/prep.rst index a32ab4ebeb1..4dea77755b6 100644 --- a/doc/sphinx-guides/source/installation/prep.rst +++ b/doc/sphinx-guides/source/installation/prep.rst @@ -69,7 +69,6 @@ There are a number of optional components you may choose to install or configure - Apache: a web server that can "reverse proxy" Jakarta EE applications (like Dataverse) and rewrite HTTP traffic. - Shibboleth: an authentication system described in :doc:`shibboleth`. Its use with Dataverse requires Apache. - OAuth2: an authentication system described in :doc:`oauth2`. -- Geoconnect: a system that allows users to create maps from geospatial files, described in :doc:`geoconnect`. See also the :doc:`/admin/integrations` section of the Admin Guide. diff --git a/doc/sphinx-guides/source/style/patterns.rst b/doc/sphinx-guides/source/style/patterns.rst index 92397910c14..aabc2cfd736 100644 --- a/doc/sphinx-guides/source/style/patterns.rst +++ b/doc/sphinx-guides/source/style/patterns.rst @@ -587,7 +587,7 @@ Another variation of icon-only buttons uses the ``.btn-link`` style class from B
  • - WorldMap + Data Explorer
    • diff --git a/doc/sphinx-guides/source/user/data-exploration/index.rst b/doc/sphinx-guides/source/user/data-exploration/index.rst index 52833e3ae50..02c386a3d9b 100755 --- a/doc/sphinx-guides/source/user/data-exploration/index.rst +++ b/doc/sphinx-guides/source/user/data-exploration/index.rst @@ -13,4 +13,3 @@ Contents: .. toctree:: tworavens - worldmap diff --git a/doc/sphinx-guides/source/user/data-exploration/worldmap.rst b/doc/sphinx-guides/source/user/data-exploration/worldmap.rst deleted file mode 100644 index 3f884560cff..00000000000 --- a/doc/sphinx-guides/source/user/data-exploration/worldmap.rst +++ /dev/null @@ -1,153 +0,0 @@ -.. _world-map: - -WorldMap: Geospatial Data Exploration -+++++++++++++++++++++++++++++++++++++ - -.. contents:: |toctitle| - :local: - -Dataverse and WorldMap -====================== - -`WorldMap `_ is developed by the Center for Geographic Analysis (CGA) at Harvard and is open source software that helps researchers visualize and explore their data in maps. The WorldMap and Dataverse collaboration allows researchers to upload shapefiles or tabular files to Dataverse for long term storage and receive a persistent identifier (through DOI), then easily navigate into WorldMap to interact with the data. - -Note: WorldMap hosts their own `user guide `_ that covers some of the same material as this page. - -What is Geoconnect? -=================== - -Geoconnect is a platform that integrates Dataverse and WorldMap, allowing researchers to visualize their geospatial data. Geoconnect can be used to create maps of shapefiles or of tabular files containing geospatial information. Geoconnect is an optional component of Dataverse, so if you are interested in this feature but don't see it in the installation of Dataverse you are using, contact support for that installation to request they enable Geoconnect. - -Once the map has been created using Geoconnect, can be viewed by choosing the WorldMap explore tool option for that data file. - -Mapping shapefiles with Geoconnect -================================== - -Geoconnect is capable of mapping shapefiles which are uploaded to Dataverse in .zip format. Specifically, Dataverse recognizes a zipped shapefile by: - -1. Examining the contents of the .zip file - -2. Checking for the existence of four similarly named files with the following extensions: .dbf, .prj, .shp, .shx - -Once you have uploaded your .zip shapefile, a Map Data button will appear next to the file in the dataset. In order to use this button, you'll need to publish your dataset. Once your dataset has been published, you can click on the Map Data button to be brought to Geoconnect, the portal between Dataverse and WorldMap that will allow you to create your map. - -To get started with visualizing your shapefile, click on the blue "Visualize on WorldMap" button in Geoconnect. It may take up to 45 seconds for the data to be sent to WorldMap and then back to Geoconnect. - -Once this process has finished, you will be taken to a new page where you can style your map through Attribute, Classification Method, Number of Intervals, and Colors. Clicking "Apply Changes" will send your map to both Dataverse and WorldMap, creating a preview of your map that will be visible on your file page and your dataset page. - -Clicking "View on WorldMap" will open WorldMap in a new tab, allowing you to see how your map will be displayed there. - -You can delete your map with the "Delete" button. If you decide to delete the map, it will no longer appear on WorldMap, and your dataset in Dataverse will no longer display the map preview. - -When you're satisfied with your map, you may click "Return to the Dataverse" to go back to Dataverse. - -In the future, to replace your shapefile's map with a new one, simply click the Map Data button on the dataset or file page to return to the Geoconnect edit map page. - -Mapping tabular files with Geoconnect -===================================== - -Geoconnect can map tabular files that contain geospatial information such as latitude/longitude coordinates, census tracts, zip codes, Boston election wards, etc. - - -Preparing a tabular file to be mapped -------------------------------------- - -**1. Ingest** - -Geospatial tabular files need a bit of preparation in Dataverse before they can be mapped in Geoconnect. When you upload your file, Dataverse will take about ten seconds to ingest it. During the ingest process it will identify the file as tabular data. - -|image1| - - -**2.Tag as Geospatial** - -Next, you'll need to let Dataverse know that your tabular file contains geospatial data. Select your file, click the "Edit Files" button, and select "Tags" from the dropdown menu. This will take you to the Edit Tags menu (pictured below). Under the "Tabular Data Tags" dropdown, select "Geospatial". Then click "Save Changes". - -|image2| - - -**3. Publish & Map Data** - -At this point, the configure tool option for the file will be available for your file. Publish this new version of your dataset to map your data. - - -Creating the map ----------------- - -If your tabular file contains **latitude and longitude** columns, then the process is simple: those columns may be directly mapped. Otherwise, you will need to use a *spatial join*. Spatial joins tell WorldMap how to read your tabular data file in order to create a map that accurately represents it. - -To carry out a spatial join, you'll manually connect - -- Geospatial column(s) from your Dataverse tabular file - - e.g., a census tract column from your table - -with - -- A WorldMap "target layer" that contains the same geospatial information - - e.g., WorldMap's "target layer" containing census tract parameters - -The following screenshots illustrate the mapping process: - -**1. Once you've pressed the "Map Data" button, you're brought to this page:** - -|image4| - -**2. Choose a Geospatial Data Type** - -|image5| - -**3. Choose a column from your file to match the WorldMap Layer you selected** - -|image6| - -**4. Choose from the list of WorldMap Layers available for the Geospatial Data Type you selected** - -|image7| - -**5.Submit the data for mapping!** - -|image8| - -**6. View Results** - -At this point you will be presented with a basic map that can be styled to your specifications. The example pictured below includes an error message - some of the rows weren't able to be matched properly. In this case, you could still go forward with your map, but without the information from the unmatched rows. - -|image9| - -Finalizing your map -=================== - -Now that you have created your map: - -- It exists on the WorldMap platform and may be viewed there -- with all of WorldMap's capabilities. - -- Dataverse will contain a preview of the map and links to the larger version on WorldMap. - -The map editor (pictured above) provides a set of options you can use to style your map. Clicking "Apply Changes" saves the current version of your map to Dataverse and Worldmap. The "Return to the Dataverse" button brings you back to Dataverse. "View on WorldMap" takes you to the map's page on WorldMap, which offers additional views and options. - -Return to the editor for further changes to your map in the future by choosing the map configure tool option for your data file in Dataverse. - -Removing your map -================= - -You can delete your map at any time. Select the map configure tool option for the data file in Dataverse, then in Geoconnect select "Delete Map". This completely removes the map and underlying data from the WorldMap platform. - - -.. |image1| image:: ./img/geoconnect-tabular-1.png - :class: img-responsive -.. |image2| image:: ./img/geoconnect-tabular-2.png - :class: img-responsive -.. |image3| image:: ./img/geoconnect-tabular-3.png - :class: img-responsive -.. |image4| image:: ./img/geoconnect-tabular-4.png - :class: img-responsive -.. |image5| image:: ./img/geoconnect-tabular-5.png - :class: img-responsive -.. |image6| image:: ./img/geoconnect-tabular-6.png - :class: img-responsive -.. |image7| image:: ./img/geoconnect-tabular-7.png - :class: img-responsive -.. |image8| image:: ./img/geoconnect-tabular-8.png - :class: img-responsive -.. |image9| image:: ./img/geoconnect-tabular-9.png - :class: img-responsive diff --git a/doc/sphinx-guides/source/user/dataset-management.rst b/doc/sphinx-guides/source/user/dataset-management.rst index e1ca2ad4c41..a618bb93793 100755 --- a/doc/sphinx-guides/source/user/dataset-management.rst +++ b/doc/sphinx-guides/source/user/dataset-management.rst @@ -181,34 +181,6 @@ Additional download options available for tabular data (found in the same drop-d See also :ref:`restricted-tabular-files`. -Geospatial ----------- - -Geospatial `shapefiles `_ can be further explored and manipulated through an optional integration with `WorldMap <../user/data-exploration/worldmap.html>`_, a geospatial data visualization and analysis tool developed by the `Center for Geographic Analysis `_ at Harvard University. A shapefile is a set of files, often uploaded/transferred in .zip format. This set may contain up to 15 files. A minimum of 3 specific files (.shp, .shx, .dbf) are needed to be a valid shapefile and a 4th file (.prj) is required for WorldMap--or any type of meaningful visualization. - -For ingest into Dataverse and connecting to WorldMap, these 4 files are the minimum required: - -* .shp - shape format; the feature geometry itself -* .shx - shape index format; a positional index of the feature geometry to allow seeking forwards and backwards quickly -* .dbf - attribute format; columnar attributes for each shape, in dBase IV format -* .prj - projection format; the coordinate system and projection information, a plain text file describing the projection using well-known text format - -For a zipped shapefile, we require 4 files with these extensions. Other files may be included within the zipped shapefile, but they are not required: - -* .shp -* .shx -* .prj -* .dbf - -For example, if these files were included within a .zip, the Geoconnect configure tool option is displayed: - -* subway_line.shp -* subway_line.shx -* subway_line.prj -* subway_line.dbf - -Once you publish your dataset with your shape files, you will be able to use the configure tool `Geoconnect `_ to visualize and manipulate these files for users to explore this geospatial data on `WorldMap `__. Please note: In order to map your data file, a copy will be sent to Harvard's `WorldMap `__ platform. You have the ability to delete any maps, and associated data, from the Harvard WorldMap platform, at any time. - Astronomy (FITS) ---------------- diff --git a/scripts/api/setup-optional-harvard.sh b/scripts/api/setup-optional-harvard.sh index 39ccbb310b6..e51846bd9e0 100755 --- a/scripts/api/setup-optional-harvard.sh +++ b/scripts/api/setup-optional-harvard.sh @@ -35,9 +35,6 @@ echo "- Enabling Shibboleth" curl -X POST -H "Content-type: application/json" http://localhost:8080/api/admin/authenticationProviders --upload-file ../../doc/sphinx-guides/source/_static/installation/files/etc/shibboleth/shibAuthProvider.json echo "- Enabling TwoRavens" curl -X POST -H 'Content-type: application/json' --upload-file ../../doc/sphinx-guides/source/_static/installation/files/root/external-tools/twoRavens.json http://localhost:8080/api/admin/externalTools -echo "- Enabling Geoconnect" -curl -s -X PUT -d true "$SERVER/admin/settings/:GeoconnectCreateEditMaps" -curl -s -X PUT -d true "$SERVER/admin/settings/:GeoconnectViewMaps" echo "- Setting system email" curl -X PUT -d "Harvard Dataverse Support " http://localhost:8080/api/admin/settings/:SystemEmail curl -X PUT -d ", The President & Fellows of Harvard College" http://localhost:8080/api/admin/settings/:FooterCopyright diff --git a/src/main/java/ValidationMessages.properties b/src/main/java/ValidationMessages.properties index da5c336b54e..877f76b0f9b 100644 --- a/src/main/java/ValidationMessages.properties +++ b/src/main/java/ValidationMessages.properties @@ -34,13 +34,6 @@ filename.illegalCharacters=File Name cannot contain any of the following charact directoryname.illegalCharacters=Directory Name cannot contain invalid characters. Valid characters are a-Z, 0-9, '_', '-', '.', '\\', '/' and ' ' (white space). filename.blank=Please specify a file name. - -map.layername=Please specify a layer name. -map.layerlink=Please specify a layer link. -map.link=Please specify am embedded map link. -map.imagelink=Please specify a map image link. -map.username=Please specify a WorldMap username. - oauth.username=Please enter your username. password.retype=The new password is blank: re-type it again. diff --git a/src/main/java/edu/harvard/iq/dataverse/DataFile.java b/src/main/java/edu/harvard/iq/dataverse/DataFile.java index 2f0981c80af..cb2b6e221e8 100644 --- a/src/main/java/edu/harvard/iq/dataverse/DataFile.java +++ b/src/main/java/edu/harvard/iq/dataverse/DataFile.java @@ -7,7 +7,6 @@ import com.google.gson.annotations.Expose; import com.google.gson.annotations.SerializedName; import edu.harvard.iq.dataverse.DatasetVersion.VersionState; -import edu.harvard.iq.dataverse.api.WorldMapRelatedData; import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; import edu.harvard.iq.dataverse.dataaccess.DataAccess; import edu.harvard.iq.dataverse.dataaccess.StorageIO; @@ -19,7 +18,6 @@ import edu.harvard.iq.dataverse.util.FileUtil; import edu.harvard.iq.dataverse.util.ShapefileHandler; import edu.harvard.iq.dataverse.util.StringUtil; -import edu.harvard.iq.dataverse.worldmapauth.WorldMapToken; import java.io.IOException; import java.util.List; import java.util.ArrayList; @@ -217,14 +215,7 @@ public List getGuestbookResponses() { public void setGuestbookResponses(List guestbookResponses) { this.guestbookResponses = guestbookResponses; } - - // The WorldMap LayerMetadata and AuthToken are here to facilitate a - // clean cascade delete when the DataFile is deleted: - @OneToOne(mappedBy="dataFile", orphanRemoval = true, cascade={CascadeType.REMOVE, CascadeType.MERGE, CascadeType.PERSIST}) - private MapLayerMetadata mapLayerMetadata; - @OneToMany(mappedBy="dataFile", orphanRemoval = true, cascade={CascadeType.REMOVE, CascadeType.MERGE, CascadeType.PERSIST}) - private List worldMapTokens; - + private char ingestStatus = INGEST_STATUS_NONE; @OneToOne(mappedBy = "thumbnailFile") @@ -725,21 +716,7 @@ public Dataset getThumbnailForDataset() { public void setAsThumbnailForDataset(Dataset dataset) { thumbnailForDataset = dataset; } - - /** - * URL to use with the WorldMapRelatedData API - * Used within dataset.xhtml - * - * @param dataverseUserID - * @return URL for "Map It" functionality - */ - public String getMapItURL(Long dataverseUserID){ - if (dataverseUserID==null){ - return null; - } - return WorldMapRelatedData.getMapItURL(this.getId(), dataverseUserID); - } - + /* 8/10/2014 - Using the current "open access" url */ diff --git a/src/main/java/edu/harvard/iq/dataverse/DatasetPage.java b/src/main/java/edu/harvard/iq/dataverse/DatasetPage.java index 19ce99435b2..b6c353090fe 100644 --- a/src/main/java/edu/harvard/iq/dataverse/DatasetPage.java +++ b/src/main/java/edu/harvard/iq/dataverse/DatasetPage.java @@ -89,7 +89,6 @@ import java.util.HashSet; import javax.faces.model.SelectItem; import java.util.logging.Level; -import edu.harvard.iq.dataverse.datasetutility.WorldMapPermissionHelper; import edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException; import edu.harvard.iq.dataverse.engine.command.impl.AbstractSubmitToArchiveCommand; import edu.harvard.iq.dataverse.engine.command.impl.CreateNewDatasetCommand; @@ -185,8 +184,6 @@ public enum DisplayMode { @EJB UserNotificationServiceBean userNotificationService; @EJB - MapLayerMetadataServiceBean mapLayerMetadataService; - @EJB BuiltinUserServiceBean builtinUserService; @EJB DataverseFieldTypeInputLevelServiceBean dataverseFieldTypeInputLevelService; @@ -225,8 +222,6 @@ public enum DisplayMode { @Inject FileDownloadHelper fileDownloadHelper; @Inject - WorldMapPermissionHelper worldMapPermissionHelper; - @Inject ThumbnailServiceWrapper thumbnailServiceWrapper; @Inject SettingsWrapper settingsWrapper; @@ -1359,8 +1354,6 @@ public void setNoDVsRemaining(boolean noDVsRemaining) { this.noDVsRemaining = noDVsRemaining; } - private final Map mapLayerMetadataLookup = new HashMap<>(); - private GuestbookResponse guestbookResponse; private Guestbook selectedGuestbook; @@ -1716,78 +1709,10 @@ public boolean isShapefileType(FileMetadata fm) { return fm.getDataFile().isShapefileType(); } - /* - Check if the FileMetadata.dataFile has an associated MapLayerMetadata object - - The MapLayerMetadata objects have been fetched at page inception by "loadMapLayerMetadataLookup()" - */ - public boolean hasMapLayerMetadata(FileMetadata fm) { - if (fm == null) { - return false; - } - if (fm.getDataFile() == null) { - return false; - } - return doesDataFileHaveMapLayerMetadata(fm.getDataFile()); - } - - /** - * Check if a DataFile has an associated MapLayerMetadata object - * - * The MapLayerMetadata objects have been fetched at page inception by - * "loadMapLayerMetadataLookup()" - */ - private boolean doesDataFileHaveMapLayerMetadata(DataFile df) { - if (df == null) { - return false; - } - if (df.getId() == null) { - return false; - } - return this.mapLayerMetadataLookup.containsKey(df.getId()); - } - - /** - * Using a DataFile id, retrieve an associated MapLayerMetadata object - * - * The MapLayerMetadata objects have been fetched at page inception by - * "loadMapLayerMetadataLookup()" - */ - public MapLayerMetadata getMapLayerMetadata(DataFile df) { - if (df == null) { - return null; - } - return this.mapLayerMetadataLookup.get(df.getId()); - } - private void msg(String s){ // System.out.println(s); } - /** - * Create a hashmap consisting of { DataFile.id : MapLayerMetadata object} - * - * Very few DataFiles will have associated MapLayerMetadata objects so only - * use 1 query to get them - */ - private void loadMapLayerMetadataLookup() { - if (this.dataset == null) { - } - if (this.dataset.getId() == null) { - return; - } - List mapLayerMetadataList = mapLayerMetadataService.getMapLayerMetadataForDataset(this.dataset); - if (mapLayerMetadataList == null) { - return; - } - for (MapLayerMetadata layer_metadata : mapLayerMetadataList) { - mapLayerMetadataLookup.put(layer_metadata.getDataFile().getId(), layer_metadata); - } - - }// A DataFile may have a related MapLayerMetadata object - - - private List displayFileMetadata; public List getDisplayFileMetadata() { @@ -1967,8 +1892,6 @@ private String init(boolean initFull) { //setReleasedVersionTabList(resetReleasedVersionTabList()); //SEK - lazymodel may be needed for datascroller in future release // lazyModel = new LazyFileMetadataDataModel(workingVersion.getId(), datafileService ); - // populate MapLayerMetadata - this.loadMapLayerMetadataLookup(); // A DataFile may have a related MapLayerMetadata object this.guestbookResponse = guestbookResponseService.initGuestbookResponseForFragment(workingVersion, null, session); logger.fine("Checking if rsync support is enabled."); if (DataCaptureModuleUtil.rsyncSupportEnabled(settingsWrapper.getValueForKey(SettingsServiceBean.Key.UploadMethods)) @@ -2400,27 +2323,7 @@ private void refreshSelectedFiles(List filesToRefresh){ } readOnly = false; } - - public void testSelectedFilesForMapData(){ - setSelectedFilesHasMapLayer(false); - for (FileMetadata fmd : selectedFiles){ - if(worldMapPermissionHelper.hasMapLayerMetadata(fmd)){ - setSelectedFilesHasMapLayer(true); - return; //only need one for warning message - } - } - } - - private boolean selectedFilesHasMapLayer; - - public boolean isSelectedFilesHasMapLayer() { - return selectedFilesHasMapLayer; - } - public void setSelectedFilesHasMapLayer(boolean selectedFilesHasMapLayer) { - this.selectedFilesHasMapLayer = selectedFilesHasMapLayer; - } - private Integer chunkSize = 25; public Integer getChunkSize() { @@ -2634,18 +2537,6 @@ private DatasetVersion setDatasetVersionDeaccessionReasonAndURL(DatasetVersion d dvIn.setArchiveNote(getDeaccessionForwardURLFor()); return dvIn; } - - public boolean isMapLayerToBeDeletedOnPublish(){ - - for (FileMetadata fmd : workingVersion.getFileMetadatas()){ - if (worldMapPermissionHelper.hasMapLayerMetadata(fmd)){ - if (fmd.isRestricted() || fmd.isRestrictedUI()){ - return true; - } - } - } - return false; - } private String releaseDataset(boolean minor) { if (session.getUser() instanceof AuthenticatedUser) { @@ -5241,15 +5132,6 @@ public GuestbookResponseServiceBean getGuestbookResponseService() { public void setGuestbookResponseService(GuestbookResponseServiceBean guestbookResponseService) { this.guestbookResponseService = guestbookResponseService; } - - - public WorldMapPermissionHelper getWorldMapPermissionHelper() { - return worldMapPermissionHelper; - } - - public void setWorldMapPermissionHelper(WorldMapPermissionHelper worldMapPermissionHelper) { - this.worldMapPermissionHelper = worldMapPermissionHelper; - } /** * dataset title diff --git a/src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java b/src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java index 83152d1a75f..50681ad4fa2 100644 --- a/src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java +++ b/src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java @@ -1510,16 +1510,7 @@ public String getReturnToDatasetURL(String serverName, Dataset dset) { } return serverName + "/dataset.xhtml?id=" + dset.getId() + "&versionId=" + this.getId(); } - - /* - Per #3511 we are returning all users to the File Landing page - If we in the future we are going to return them to the referring page we will need the - getReturnToDatasetURL method and add something to the call to the api to - pass the referring page and some kind of decision point in the getWorldMapDatafileInfo method in - WorldMapRelatedData - SEK 3/24/2017 - */ - + public String getReturnToFilePageURL (String serverName, Dataset dset, DataFile dataFile){ if (serverName == null || dataFile == null) { return null; diff --git a/src/main/java/edu/harvard/iq/dataverse/DataverseFieldTypeInputLevelServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/DataverseFieldTypeInputLevelServiceBean.java index ce9b44f734b..42a1290fdbd 100644 --- a/src/main/java/edu/harvard/iq/dataverse/DataverseFieldTypeInputLevelServiceBean.java +++ b/src/main/java/edu/harvard/iq/dataverse/DataverseFieldTypeInputLevelServiceBean.java @@ -77,11 +77,7 @@ public List findByDataverseIdAndDatasetFieldTypeId return null; } } - // - - // Query query = em.createQuery("select object(o) from MapLayerMetadata as o where o.dataset=:dataset");// order by o.name"); - // query.setParameter("dataset", dataset); - + public DataverseFieldTypeInputLevel findByDataverseIdDatasetFieldTypeId(Long dataverseId, Long datasetFieldTypeId) { Query query = em.createNamedQuery("DataverseFieldTypeInputLevel.findByDataverseIdDatasetFieldTypeId", DataverseFieldTypeInputLevel.class); query.setParameter("dataverseId", dataverseId); diff --git a/src/main/java/edu/harvard/iq/dataverse/EjbDataverseEngine.java b/src/main/java/edu/harvard/iq/dataverse/EjbDataverseEngine.java index 9606b24ff14..da9aba43498 100644 --- a/src/main/java/edu/harvard/iq/dataverse/EjbDataverseEngine.java +++ b/src/main/java/edu/harvard/iq/dataverse/EjbDataverseEngine.java @@ -161,9 +161,6 @@ public class EjbDataverseEngine { @EJB DatasetVersionServiceBean datasetVersionService; - @EJB - MapLayerMetadataServiceBean mapLayerMetadata; - @EJB DataCaptureModuleServiceBean dataCaptureModule; @@ -565,11 +562,6 @@ public WorkflowServiceBean workflows() { return workflowService; } - @Override - public MapLayerMetadataServiceBean mapLayerMetadata() { - return mapLayerMetadata; - } - @Override public DataCaptureModuleServiceBean dataCaptureModule() { return dataCaptureModule; diff --git a/src/main/java/edu/harvard/iq/dataverse/FileDownload.java b/src/main/java/edu/harvard/iq/dataverse/FileDownload.java index 8463a8278c0..fad03d2a0a1 100644 --- a/src/main/java/edu/harvard/iq/dataverse/FileDownload.java +++ b/src/main/java/edu/harvard/iq/dataverse/FileDownload.java @@ -54,13 +54,11 @@ public class FileDownload implements Serializable { /** * Possible values for downloadType include "Download", "Subset", - * "WorldMap", or the displayName of an ExternalTool. + * or the displayName of an ExternalTool. * - * TODO: Types like "Download" and "Subset" and probably "WorldMap" should + * TODO: Types like "Download" and "Subset" should * be defined once as constants (likely an enum) rather than having these * strings duplicated in various places when setDownloadtype() is called. - * (Some day it would be nice to convert WorldMap into an ExternalTool but - * it's not worth the effort at this time.) */ private String downloadtype; private String sessionId; diff --git a/src/main/java/edu/harvard/iq/dataverse/FileDownloadHelper.java b/src/main/java/edu/harvard/iq/dataverse/FileDownloadHelper.java index 80625d9cc8e..95ceaec5f67 100644 --- a/src/main/java/edu/harvard/iq/dataverse/FileDownloadHelper.java +++ b/src/main/java/edu/harvard/iq/dataverse/FileDownloadHelper.java @@ -143,14 +143,6 @@ public void writeGuestbookAndLaunchPackagePopup(GuestbookResponse guestbookRespo fileDownloadService.writeGuestbookResponseRecord(guestbookResponse); } - public String startWorldMapDownloadLink(GuestbookResponse guestbookResponse, FileMetadata fmd){ - - guestbookResponse.setDownloadtype("WorldMap"); - String retVal = fileDownloadService.startWorldMapDownloadLink(guestbookResponse, fmd); - PrimeFaces.current().executeScript("PF('downloadPopup').hide()"); - return retVal; - } - /** * Writes a guestbook entry for either popup scenario: guestbook or terms. */ @@ -198,12 +190,6 @@ public void setSelectedFileId(String selectedFileId) { * WARNING: Before calling this, make sure the user has download * permission for the file!! (See DatasetPage.canDownloadFile()) * - * Should there be a Explore WorldMap Button for this file? - * See table in: https://github.com/IQSS/dataverse/issues/1618 - * - * (1) Does the file have MapLayerMetadata? - * (2) Are the proper settings in place - * * @param fileMetadata * @return boolean */ diff --git a/src/main/java/edu/harvard/iq/dataverse/FileDownloadServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/FileDownloadServiceBean.java index a91a28d3d96..4b0272d4a36 100644 --- a/src/main/java/edu/harvard/iq/dataverse/FileDownloadServiceBean.java +++ b/src/main/java/edu/harvard/iq/dataverse/FileDownloadServiceBean.java @@ -8,7 +8,6 @@ import edu.harvard.iq.dataverse.authorization.users.User; import edu.harvard.iq.dataverse.dataaccess.DataAccess; import edu.harvard.iq.dataverse.dataaccess.StorageIO; -import edu.harvard.iq.dataverse.datasetutility.WorldMapPermissionHelper; import edu.harvard.iq.dataverse.engine.command.exception.CommandException; import edu.harvard.iq.dataverse.engine.command.impl.CreateGuestbookResponseCommand; import edu.harvard.iq.dataverse.engine.command.impl.RequestAccessCommand; @@ -86,7 +85,6 @@ public class FileDownloadServiceBean implements java.io.Serializable { @Inject DataverseRequestServiceBean dvRequestService; - @Inject WorldMapPermissionHelper worldMapPermissionHelper; @Inject FileDownloadHelper fileDownloadHelper; @Inject MakeDataCountLoggingServiceBean mdcLogService; @@ -334,45 +332,10 @@ public void explore(GuestbookResponse guestbookResponse, FileMetadata fmd, Exter } } - public String startWorldMapDownloadLink(GuestbookResponse guestbookResponse, FileMetadata fmd){ - - if (guestbookResponse != null && guestbookResponse.isWriteResponse() && ((fmd != null && fmd.getDataFile() != null) || guestbookResponse.getDataFile() != null)){ - if(guestbookResponse.getDataFile() == null && fmd != null){ - guestbookResponse.setDataFile(fmd.getDataFile()); - } - if (fmd == null || !fmd.getDatasetVersion().isDraft()){ - writeGuestbookResponseRecord(guestbookResponse); - } - } - DataFile file = null; - if (fmd != null){ - file = fmd.getDataFile(); - } - if (guestbookResponse != null && guestbookResponse.getDataFile() != null && file == null){ - file = guestbookResponse.getDataFile(); - } - - - String retVal = worldMapPermissionHelper.getMapLayerMetadata(file).getLayerLink(); - - try { - FacesContext.getCurrentInstance().getExternalContext().redirect(retVal); - return retVal; - } catch (IOException ex) { - logger.info("Failed to issue a redirect to file download url."); - } - return retVal; - } - public Boolean canSeeTwoRavensExploreButton(){ return false; } - - - public Boolean canUserSeeExploreWorldMapButton(){ - return false; - } - + public void downloadDatasetCitationXML(Dataset dataset) { downloadCitationXML(null, dataset, false); } diff --git a/src/main/java/edu/harvard/iq/dataverse/FilePage.java b/src/main/java/edu/harvard/iq/dataverse/FilePage.java index 17a9904d2b1..da315424220 100644 --- a/src/main/java/edu/harvard/iq/dataverse/FilePage.java +++ b/src/main/java/edu/harvard/iq/dataverse/FilePage.java @@ -14,7 +14,6 @@ import edu.harvard.iq.dataverse.authorization.users.PrivateUrlUser; import edu.harvard.iq.dataverse.authorization.users.User; import edu.harvard.iq.dataverse.dataaccess.StorageIO; -import edu.harvard.iq.dataverse.datasetutility.WorldMapPermissionHelper; import edu.harvard.iq.dataverse.engine.command.Command; import edu.harvard.iq.dataverse.engine.command.exception.CommandException; import edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException; @@ -123,18 +122,9 @@ public class FilePage implements java.io.Serializable { PermissionsWrapper permissionsWrapper; @Inject FileDownloadHelper fileDownloadHelper; - @Inject WorldMapPermissionHelper worldMapPermissionHelper; @Inject MakeDataCountLoggingServiceBean mdcLogService; - public WorldMapPermissionHelper getWorldMapPermissionHelper() { - return worldMapPermissionHelper; - } - - public void setWorldMapPermissionHelper(WorldMapPermissionHelper worldMapPermissionHelper) { - this.worldMapPermissionHelper = worldMapPermissionHelper; - } - private static final Logger logger = Logger.getLogger(FilePage.class.getCanonicalName()); private boolean fileDeleteInProgress = false; @@ -233,7 +223,7 @@ public String init() { configureTools = externalToolService.findFileToolsByTypeAndContentType(ExternalTool.Type.CONFIGURE, contentType); exploreTools = externalToolService.findFileToolsByTypeAndContentType(ExternalTool.Type.EXPLORE, contentType); Collections.sort(exploreTools, CompareExternalToolName); - toolsWithPreviews = addMapLayerAndSortExternalTools(); + toolsWithPreviews = sortExternalTools(); if(!toolsWithPreviews.isEmpty()){ setSelectedTool(toolsWithPreviews.get(0)); } @@ -261,32 +251,12 @@ public Long getDatasetVersionId() { public void setDatasetVersionId(Long datasetVersionId) { this.datasetVersionId = datasetVersionId; } - - private List addMapLayerAndSortExternalTools(){ - List retList = externalToolService.findFileToolsByTypeAndContentType(ExternalTool.Type.PREVIEW, file.getContentType()); - if(!retList.isEmpty()){ - retList.forEach((et) -> { - et.setWorldMapTool(false); - }); - } - if (file != null && worldMapPermissionHelper.getMapLayerMetadata(file) != null && worldMapPermissionHelper.getMapLayerMetadata(file).getEmbedMapLink() != null) { - ExternalTool wpTool = new ExternalTool(); - wpTool.setDisplayName("World Map"); - wpTool.setToolParameters("{}"); - wpTool.setToolUrl(worldMapPermissionHelper.getMapLayerMetadata(file).getEmbedMapLink()); - wpTool.setWorldMapTool(true); - retList.add(wpTool); - } + private List sortExternalTools(){ + List retList = externalToolService.findFileToolsByTypeAndContentType(ExternalTool.Type.PREVIEW, file.getContentType()); Collections.sort(retList, CompareExternalToolName); - return retList; } - - /* - worldMapPermissionHelper.getMapLayerMetadata(FilePage.fileMetadata.dataFile).getEmbedMapLink() - */ - public boolean isDownloadPopupRequired() { if(fileMetadata.getId() == null || fileMetadata.getDatasetVersion().getId() == null ){ diff --git a/src/main/java/edu/harvard/iq/dataverse/MailServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/MailServiceBean.java index 0ea36dfdc7a..177efd9ed4d 100644 --- a/src/main/java/edu/harvard/iq/dataverse/MailServiceBean.java +++ b/src/main/java/edu/harvard/iq/dataverse/MailServiceBean.java @@ -440,19 +440,6 @@ public String getMessageTextBasedOnNotification(UserNotification userNotificatio )); logger.fine(datasetCreatedMessage); return messageText += datasetCreatedMessage; - case MAPLAYERUPDATED: - version = (DatasetVersion) targetObject; - pattern = BundleUtil.getStringFromBundle("notification.email.worldMap.added"); - String[] paramArrayMapLayer = {version.getDataset().getDisplayName(), getDatasetLink(version.getDataset())}; - messageText += MessageFormat.format(pattern, paramArrayMapLayer); - return messageText; - case MAPLAYERDELETEFAILED: - FileMetadata targetFileMetadata = (FileMetadata) targetObject; - version = targetFileMetadata.getDatasetVersion(); - pattern = BundleUtil.getStringFromBundle("notification.email.maplayer.deletefailed.text"); - String[] paramArrayMapLayerDelete = {targetFileMetadata.getLabel(), getDatasetLink(version.getDataset())}; - messageText += MessageFormat.format(pattern, paramArrayMapLayerDelete); - return messageText; case SUBMITTEDDS: version = (DatasetVersion) targetObject; String mightHaveSubmissionComment = ""; @@ -598,9 +585,6 @@ private Object getObjectOfNotification (UserNotification userNotification){ case GRANTFILEACCESS: case REJECTFILEACCESS: return datasetService.find(userNotification.getObjectId()); - case MAPLAYERDELETEFAILED: - return dataFileService.findFileMetadata(userNotification.getObjectId()); - case MAPLAYERUPDATED: case CREATEDS: case SUBMITTEDDS: case PUBLISHEDDS: diff --git a/src/main/java/edu/harvard/iq/dataverse/MapLayerMetadata.java b/src/main/java/edu/harvard/iq/dataverse/MapLayerMetadata.java deleted file mode 100644 index a7dd4d9eef1..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/MapLayerMetadata.java +++ /dev/null @@ -1,304 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ - -package edu.harvard.iq.dataverse; - -import java.io.Serializable; -import java.sql.Timestamp; -import java.util.Arrays; -import java.util.List; -import javax.persistence.Column; -import javax.persistence.Entity; -import javax.persistence.GeneratedValue; -import javax.persistence.GenerationType; -import javax.persistence.Id; -import javax.persistence.Index; -import javax.persistence.JoinColumn; -import javax.persistence.ManyToOne; -import javax.persistence.NamedQueries; -import javax.persistence.NamedQuery; -import javax.persistence.Table; -import javax.persistence.Transient; -import org.hibernate.validator.constraints.NotBlank; - -/** - * File metadata: specifically WorldMap layer information for a specific DataFile - * - * @author raprasad - */ -@NamedQueries({ - @NamedQuery(name = "MapLayerMetadata.findAll", - query = "SELECT mlm FROM MapLayerMetadata mlm"),}) -@Entity -@Table(indexes = {@Index(columnList="dataset_id")}) -public class MapLayerMetadata implements Serializable { - - - @Transient - public final static String dataType = "MapLayerMetadata"; - - @Transient - public final static List MANDATORY_JSON_FIELDS = Arrays.asList("layerName", "layerLink", "embedMapLink", "worldmapUsername"); - - - private static final long serialVersionUID = 1L; - - @Id - @GeneratedValue(strategy = GenerationType.IDENTITY) - private Long id; - - // ForeignKey to DataFile - //x@ManyToOne - // For now, make this unique: Each DataFile may only have one map - @JoinColumn(nullable=false, unique=true) - private DataFile dataFile; - - // ForeignKey to Dataset. - // This is always reachable via the datafile.getOwner(); - // However, save the Dataset itself to potentially save an extra step - @ManyToOne - @JoinColumn(nullable=false) - private Dataset dataset; - - @Column(nullable=false) - @NotBlank(message = "{map.layername}") - private String layerName; - - @Column(nullable=false) - @NotBlank(message = "{map.layerlink}") - private String layerLink; - - @Column(nullable=false) - @NotBlank(message = "{map.link}") - private String embedMapLink; - - @Column(nullable=true) - @NotBlank(message = "{map.imagelink}") - private String mapImageLink; - - @Column(nullable=false) - @NotBlank(message = "{map.username}") - private String worldmapUsername; - - - /** - * Was this layer created by joining a tabular file - * to an existing file? - */ - private boolean isJoinLayer; - - /** - * Description if this was created via a tabular join, - */ - @Column(columnDefinition = "TEXT") - private String joinDescription; - - /** - * Links to alternative representations of the map - * in JSON format - */ - @Column(columnDefinition = "TEXT") - private String mapLayerLinks; - - /** - * The HTTP Status code (200, 404, etc.) returned when you check to see if - * the map/layer exists on the WorldMap side. - */ - @Column(nullable=true) - private int lastVerifiedStatus; - - /** - * The time that lastVerifiedStatus was last recorded. - */ - @Column(nullable=true) - private Timestamp lastVerifiedTime; - - /** - * Get property layerName. - * @return value of property layerName. - */ - public String getLayerName() { - return this.layerName; - } - - /** - * Set property layerName. - * @param layerName new value of property layerName. - */ - public void setLayerName(String layerName) { - this.layerName = layerName; - } - - /** - * Get property layerLink. - * @return value of property layerLink. - */ - public String getLayerLink() { - return this.layerLink; - } - - /** - * Set property layerLink. - * @param layerLink new value of property layerLink. - */ - public void setLayerLink(String layerLink) { - this.layerLink = layerLink; - } - - /** - * Get property mapImageLink. - * @return value of property mapImageLink. - */ - public String getMapImageLink() { - return this.mapImageLink; - } - - /** - * Set property mapImageLink. - * @param mapImageLink new value of property layerLink. - */ - public void setMapImageLink(String mapImageLink) { - this.mapImageLink = mapImageLink; - } - - - /** - * Get property embedMapLink. - * @return value of property embedMapLink. - */ - public String getEmbedMapLink() { - return this.embedMapLink; - } - - /** - * Set property embedMapLink. - * @param embedMapLink new value of property embedMapLink. - */ - public void setEmbedMapLink(String embedMapLink) { - this.embedMapLink = embedMapLink; - } - - /** - * Get property worldmapUsername. - * @return value of property worldmapUsername. - */ - public String getWorldmapUsername() { - return this.worldmapUsername; - } - - /** - * Set property worldmapUsername. - * @param worldmapUsername new value of property worldmapUsername. - */ - public void setWorldmapUsername(String worldmapUsername) { - this.worldmapUsername = worldmapUsername; - } - - - /** - * Get property isJoinLayer. - * @return value of property isJoinLayer. - */ - public boolean isJoinLayer(){ - return this.isJoinLayer; - } - - /** - * Set property isJoinLayer. - * @param bool new value of property isJoinLayer. - */ - public void setIsJoinLayer(boolean bool) { - this.isJoinLayer = bool; - } - - /** - * Get property joinDescription. - * @return value of property joinDescription. - */ - public String getJoinDescription() { - return this.joinDescription; - } - - /** - * Set property joinDescription. - * @param joinDescription new value of property joinDescription. - */ - public void setJoinDescription(String joinDescription) { - this.joinDescription = joinDescription; - } - - /** - * Get property mapLayerLinks - * @return value of property joinDescription. - */ - public String getMapLayerLinks() { - return this.mapLayerLinks; - } - - /** - * Set property joinDescription. - * @param joinDescription new value of property joinDescription. - */ - public void setMapLayerLinks(String mapLayerLinks) { - this.mapLayerLinks = mapLayerLinks; - } - - public Dataset getDataset() { - return dataset; - } - - public void setDataset(Dataset dataset) { - this.dataset = dataset; - } - - public DataFile getDataFile() { - return dataFile; - } - - public void setDataFile(DataFile dataFile) { - this.dataFile = dataFile; - } - - /** - * Getter for property id. - * @return Value of property id. - */ - public Long getId() { - return this.id; - } - - /** - * Setter for property id. - * @param id New value of property id. - */ - public void setId(Long id) { - this.id = id; - } - - public Timestamp getLastVerifiedTime() { - return lastVerifiedTime; - } - - public void setLastVerifiedTime(Timestamp lastVerifiedTime) { - this.lastVerifiedTime = lastVerifiedTime; - } - - public int getLastVerifiedStatus() { - return lastVerifiedStatus; - } - - public void setLastVerifiedStatus(int lastVerifiedStatus) { - this.lastVerifiedStatus = lastVerifiedStatus; - } - - @Override - public String toString() { - return "edu.harvard.iq.dataverse.MaplayerMetadata[id=" + this.id + "]"; - - //return "WorldMap Layer: " + this.layerName + " for DataFile: " + this.dataFile.toString(); - } - -} diff --git a/src/main/java/edu/harvard/iq/dataverse/MapLayerMetadataServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/MapLayerMetadataServiceBean.java deleted file mode 100755 index dbbe8b3d8bb..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/MapLayerMetadataServiceBean.java +++ /dev/null @@ -1,406 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ - -package edu.harvard.iq.dataverse; - -import edu.harvard.iq.dataverse.api.WorldMapRelatedData; -import edu.harvard.iq.dataverse.authorization.Permission; -import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; -import edu.harvard.iq.dataverse.authorization.users.User; -import edu.harvard.iq.dataverse.dataaccess.StorageIO; -import edu.harvard.iq.dataverse.util.SystemConfig; -import edu.harvard.iq.dataverse.worldmapauth.WorldMapToken; -import edu.harvard.iq.dataverse.worldmapauth.WorldMapTokenServiceBean; -import java.io.IOException; -import java.io.InputStream; -import java.net.HttpURLConnection; -import java.net.URL; -import java.net.URLEncoder; -import java.util.List; -import java.util.logging.Level; -import java.util.logging.Logger; -import javax.ejb.EJB; -import javax.ejb.Stateless; -import javax.inject.Named; -import javax.persistence.EntityManager; -import javax.persistence.NoResultException; -import javax.persistence.PersistenceContext; -import javax.persistence.Query; -import javax.persistence.TypedQuery; - -/** - * - * @author raprasad - */ -@Stateless -@Named -public class MapLayerMetadataServiceBean { - - - @PersistenceContext(unitName = "VDCNet-ejbPU") - private EntityManager em; - - @EJB - DataFileServiceBean dataFileService; - - @EJB - PermissionServiceBean permissionService; - - @EJB - SystemConfig systemConfig; - - @EJB - WorldMapTokenServiceBean tokenServiceBean; - - private static final Logger logger = Logger.getLogger(MapLayerMetadataServiceBean.class.getCanonicalName()); - - private static final String GEOCONNECT_MAP_DELETE_API = "/tabular/delete-map-no-ui/"; - - public MapLayerMetadata find(Object pk) { - if (pk==null){ - return null; - } - return em.find(MapLayerMetadata.class, pk); - } - - public MapLayerMetadata save( MapLayerMetadata layer_metadata) { - if (layer_metadata==null){ - return null; - } - if ( layer_metadata.getId() == null ) { - em.persist(layer_metadata); - return layer_metadata; - } else { - return em.merge( layer_metadata ); - } - } - - - - /* - Given a datafile id, return the associated MapLayerMetadata object - - */ - public MapLayerMetadata findMetadataByDatafile(DataFile datafile){ - - if (datafile == null){ - return null; - } - - try{ - // String sqlStatement = - Query query = em.createQuery("select m from MapLayerMetadata m WHERE m.dataFile=:datafile", MapLayerMetadata.class); - query.setParameter("datafile", datafile); - query.setMaxResults(1); - //entityManager.createQuery(SQL_QUERY).setParameter(arg0,arg1).setMaxResults(10).getResultList(); - return (MapLayerMetadata) query.getSingleResult(); - } catch ( NoResultException nre ) { - return null; - } - } - - - /* - Delete a mapLayerMetadata object. - - First check if the given user has permission to edit this data. - - */ - public boolean deleteMapLayerMetadataObject(MapLayerMetadata mapLayerMetadata, User user){ - logger.info("deleteMapLayerMetadataObject"); - - if ((mapLayerMetadata == null)||(user==null)){ - return false; - } - - if (permissionService.userOn(user, mapLayerMetadata.getDataFile().getOwner()).has(Permission.EditDataset)) { - - // Remove thumbnails associated with the map metadata - // (this also sets theto set the "preview image" flag to false) - // - boolean success = this.deleteOlderMapThumbnails(mapLayerMetadata.getDataFile()); - - // Remove the actual map metadata - // - em.remove(em.merge(mapLayerMetadata)); - - return true; - } - return false; - } - - - public MapLayerMetadata findMetadataByLayerNameAndDatafile(String layer_name){//, DataFile datafile) { - if ((layer_name == null)){//||(datafile==null)){ - return null; - } - //Query query = em.createQuery("select o.id from MapLayerMetadta as o where o.layer_name =:layerName and o.datafile_id =:datafileID;"); - //Query query = em.createQuery("select m from MapLayerMetadata m where m.layer_name =:layerName ;"); - try{ - return em.createQuery("select m from MapLayerMetadata m WHERE m.layerName=:layerName", MapLayerMetadata.class) - .setParameter("layerName", layer_name) - .getSingleResult(); - } catch ( NoResultException nre ) { - return null; - } - } - - - - public List getMapLayerMetadataForDataset(Dataset dataset){ - if (dataset == null){ - return null; - } - TypedQuery query = em.createQuery("select object(o) from MapLayerMetadata as o where o.dataset=:dataset", MapLayerMetadata.class);// order by o.name"); - query.setParameter("dataset", dataset); - return query.getResultList(); - } - - - /** - * Before downloading a file for map icons (see "retrieveMapImageForIcon" below), - * first remove any existing .img and .img.* files - * - * e.g. delete all that start with (DataFile name) + ".img" - * - * @param mapLayerMetadata - * @return - * @throws IOException - */ - private boolean deleteOlderMapThumbnails(DataFile dataFile) { - if (dataFile==null){ - logger.warning("dataFile is null"); - return false; - } - - // Set the preview image available flag to false - dataFile.setPreviewImageAvailable(false); - dataFile = dataFileService.save(dataFile); - - - try { - StorageIO storageIO = dataFile.getStorageIO(); - - if (storageIO == null) { - logger.warning("Null storageIO in deleteOlderMapThumbnails()"); - return false; - } - storageIO.open(); - List cachedObjectsTags = storageIO.listAuxObjects(); - - if (cachedObjectsTags != null) { - String iconBaseTag = "img"; - String iconThumbTagPrefix = "thumb"; - for (String cachedFileTag : cachedObjectsTags) { - logger.info("found AUX tag: "+cachedFileTag); - if (iconBaseTag.equals(cachedFileTag) || cachedFileTag.startsWith(iconThumbTagPrefix)) { - logger.info("deleting cached AUX object "+cachedFileTag); - storageIO.deleteAuxObject(cachedFileTag); - } - } - } - /* - * Below is the old-style code that was assuming that all the files are - * stored on a local filesystem. The StorageIO code, above, should - * be used instead for all the operations on the physical files associated - * with DataFiles. - // Get the parent directory - // - Path fileDirname = dataAccess.getFileSystemPath().getParent(); - if (fileDirname == null){ - logger.warning("DataFile directory has null path. Directory path: " + dataAccess.getFileSystemPath().toString()); - return false; - } - - // Verify that the directory exists - // - File fileDirectory = new File(fileDirname.normalize().toString()); - if (!(fileDirectory.isDirectory())){ - logger.warning("DataFile directory is not actually a directory. Directory path: " + fileDirectory.toString()); - return false; - } - - /* Iterate through directory and delete any ".img" files for this DataFile - - Example: - Datafile name: 14a5e4abf7d-e7eebfb6474d - Types of files that would be deleted (if they exist): - 14a5e4abf7d-e7eebfb6474d.img - 14a5e4abf7d-e7eebfb6474d.img.thumb64 - 14a5e4abf7d-e7eebfb6474d.img.thumb400 - *//* - String iconBaseFilename = dataAccess.getFileSystemPath().toString() + ".img"; - String iconThumbFilename = dataAccess.getFileSystemPath().toString() + ".thumb"; - - for (File singleFile : fileDirectory.listFiles()) { - if (singleFile.toString().startsWith(iconBaseFilename) || singleFile.toString().startsWith(iconThumbFilename)) { - //logger.info("file found: " + singleFile.toString()); - singleFile.delete(); - //results.add(file.getName()); - } - }*/ - } catch (IOException ioEx) { - logger.warning("IOException in deleteOlderMapThumbnails(): "+ioEx.getMessage()); - return false; - } - - return true; - } - - /** - * Use the mapLayerMetadata.mapImageLink to retrieve a PNG file directly from WorldMap - * - * Next step: Save this image as the default icon - * - * Example mapImageLink: http://worldmap.harvard.edu/download/wms/14708/png?layers=geonode:power_plants_enipedia_jan_2014_kvg&width=948&bbox=76.04800165,18.31860358,132.0322222,50.78441&service=WMS&format=image/png&srs=EPSG:4326&request=GetMap&height=550 - * - * Parameter by parameter (note width/height): - * http://worldmap.harvard.edu/download/wms/14708/png? - * layers=geonode:power_plants_enipedia_jan_2014_kvg - * width=948 - * bbox=76.04800165,18.31860358,132.0322222,50.78441 - * service=WMS - * format=image/png - * srs=EPSG:4326 - * request=GetMap - * height=550 - * - * @param mapLayerMetadata - * @return boolean - * @throws IOException - */ - public boolean retrieveMapImageForIcon(MapLayerMetadata mapLayerMetadata) throws IOException { - if (mapLayerMetadata==null){ - logger.warning("mapLayerMetadata is null"); - return false; - } - - this.deleteOlderMapThumbnails(mapLayerMetadata.getDataFile()); - - if ((mapLayerMetadata.getMapImageLink()==null)||mapLayerMetadata.getMapImageLink().isEmpty()){ - logger.warning("mapLayerMetadata does not have a 'map_image_link' attribute"); - return false; - } - - String imageUrl = mapLayerMetadata.getMapImageLink(); - imageUrl = imageUrl.replace("https:", "http:"); - logger.info("Attempt to retrieve map image: " + imageUrl); - - StorageIO dataAccess = null; - try { - dataAccess = mapLayerMetadata.getDataFile().getStorageIO(); - } catch (IOException ioEx) { - dataAccess = null; - } - - if (dataAccess == null) { - logger.warning("Failed to open Access IO on DataFile "+mapLayerMetadata.getDataFile().getId()); - return false; - } - - URL url = new URL(imageUrl); - logger.info("retrieve url : " + imageUrl); - - logger.info("try to open InputStream"); - InputStream is = null; - - try{ - is = url.openStream(); - }catch(IOException exio){ - logger.warning("Error when retrieving map icon image. Exception: " + exio.getMessage()); - if (is!=null){ - try { is.close(); } catch (IOException ignore) {} - } - return false; - } - - try { - dataAccess.saveInputStreamAsAux(is, "img"); - } catch (IOException ioex) { - logger.warning("Failed to save WorldMap-generated image; "+ioex.getMessage()); - return false; - } - - logger.info("Done"); - return true; - } - - /* - * This method calls GeoConnect and requests that a map layer for this - * DataFile is deleted, from WorldMap and GeoConnect itself. - * The use case here is when a user restricts a tabular file for which - * a geospatial map has already been made. - * This process is initiated on the Dataverse side, without any GeoConnect - * UI in the picture. (The way a user normally deletes a layer map is by - * clicking 'Delete' on the GeoConnect map page). - * Otherwise this call follows the scheme used when accessing the - * /shapefile/map-it on GeoConnect - we send along a WorldMap token and a - * callback url for GC to download the file metadata.metadata - * Depending on how it goes we receive a yes or no response from the server. - */ - public void deleteMapLayerFromWorldMap(DataFile dataFile, AuthenticatedUser user) throws IOException { - - - if (dataFile == null){ - logger.severe("dataFile cannot be null"); - return; - } - - if (user == null){ - logger.severe("user cannot be null"); - return; - } - - // Worldmap token: - WorldMapToken token = tokenServiceBean.getNewToken(dataFile, user); - if (token == null){ - logger.severe("token should NOT be null"); - return; - } - - logger.info("-- new token id: " + token.getId()); - // Callback url for geoConnect: - String callback_url = URLEncoder.encode(systemConfig.getDataverseSiteUrl() + WorldMapRelatedData.GET_WORLDMAP_DATAFILE_API_PATH); - - String geoConnectAddress = token.getApplication().getMapitLink(); - /* - * this is a bit of a hack - there should be a cleaner way to get the base - * geoconnect URL from the token! - */ - geoConnectAddress = geoConnectAddress.replace("/shapefile/map-it", ""); - - logger.log(Level.INFO, "callback_url: {0}", callback_url); - - //String geoConnectCommand = geoConnectAddress + GEOCONNECT_MAP_DELETE_API + token.getApplication().getMapitLink() + "/" + token.getToken() + "/?cb=" + callback_url; - String geoConnectCommand = geoConnectAddress + GEOCONNECT_MAP_DELETE_API + token.getToken() + "/?cb=" + callback_url; - logger.info("-- new token id 2: " + token.getId()); - - - logger.info("Attempting to call GeoConnect to request that the WorldMap layer for DataFile "+dataFile.getId()+":\n"+geoConnectCommand); - URL geoConnectUrl = new URL(geoConnectCommand); - - HttpURLConnection geoConnectConnection = (HttpURLConnection)geoConnectUrl.openConnection(); - - geoConnectConnection.setRequestMethod("GET"); - geoConnectConnection.connect(); - - int code = geoConnectConnection.getResponseCode(); - logger.info("response code: " + code); - if (code != 200) { - throw new IOException ("Failed to delete Map Layer via GeoConnect. /tabular/delete-map HTTP code response: "+code+""); - } - logger.info("response :" + geoConnectConnection.getContent()); - - } - - public List findAll() { - TypedQuery typedQuery = em.createNamedQuery("MapLayerMetadata.findAll", MapLayerMetadata.class); - List mapLayerMetadatas = typedQuery.getResultList(); - return mapLayerMetadatas; - } - -} - diff --git a/src/main/java/edu/harvard/iq/dataverse/UserNotification.java b/src/main/java/edu/harvard/iq/dataverse/UserNotification.java index d404d82c950..bac1df0f2ed 100644 --- a/src/main/java/edu/harvard/iq/dataverse/UserNotification.java +++ b/src/main/java/edu/harvard/iq/dataverse/UserNotification.java @@ -27,7 +27,7 @@ public class UserNotification implements Serializable { public enum Type { - ASSIGNROLE, REVOKEROLE, CREATEDV, CREATEDS, CREATEACC, MAPLAYERUPDATED, MAPLAYERDELETEFAILED, SUBMITTEDDS, RETURNEDDS, PUBLISHEDDS, REQUESTFILEACCESS, GRANTFILEACCESS, REJECTFILEACCESS, FILESYSTEMIMPORT, CHECKSUMIMPORT, CHECKSUMFAIL, CONFIRMEMAIL, APIGENERATED, INGESTCOMPLETED, INGESTCOMPLETEDWITHERRORS, PUBLISHFAILED_PIDREG + ASSIGNROLE, REVOKEROLE, CREATEDV, CREATEDS, CREATEACC, SUBMITTEDDS, RETURNEDDS, PUBLISHEDDS, REQUESTFILEACCESS, GRANTFILEACCESS, REJECTFILEACCESS, FILESYSTEMIMPORT, CHECKSUMIMPORT, CHECKSUMFAIL, CONFIRMEMAIL, APIGENERATED, INGESTCOMPLETED, INGESTCOMPLETEDWITHERRORS, PUBLISHFAILED_PIDREG }; private static final long serialVersionUID = 1L; diff --git a/src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java b/src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java index 2a82acc7622..551c0a797fc 100644 --- a/src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java +++ b/src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java @@ -18,7 +18,6 @@ import edu.harvard.iq.dataverse.DvObjectServiceBean; import edu.harvard.iq.dataverse.EjbDataverseEngine; import edu.harvard.iq.dataverse.GuestbookResponseServiceBean; -import edu.harvard.iq.dataverse.MapLayerMetadataServiceBean; import edu.harvard.iq.dataverse.MetadataBlock; import edu.harvard.iq.dataverse.MetadataBlockServiceBean; import edu.harvard.iq.dataverse.PermissionServiceBean; @@ -211,9 +210,6 @@ String getWrappedMessageWhenJson() { @EJB protected DatasetVersionServiceBean datasetVersionSvc; - @EJB - protected MapLayerMetadataServiceBean mapLayerMetadataSrv; - @EJB protected SystemConfig systemConfig; diff --git a/src/main/java/edu/harvard/iq/dataverse/api/Access.java b/src/main/java/edu/harvard/iq/dataverse/api/Access.java index 0c7a4224648..df9e39547b9 100644 --- a/src/main/java/edu/harvard/iq/dataverse/api/Access.java +++ b/src/main/java/edu/harvard/iq/dataverse/api/Access.java @@ -68,7 +68,6 @@ import edu.harvard.iq.dataverse.util.StringUtil; import edu.harvard.iq.dataverse.util.SystemConfig; import edu.harvard.iq.dataverse.util.json.JsonPrinter; -import edu.harvard.iq.dataverse.worldmapauth.WorldMapTokenServiceBean; import java.util.logging.Logger; import javax.ejb.EJB; @@ -166,8 +165,6 @@ public class Access extends AbstractApiBean { PermissionServiceBean permissionService; @Inject DataverseSession session; - @EJB - WorldMapTokenServiceBean worldMapTokenServiceBean; @Inject DataverseRequestServiceBean dvRequestService; @EJB @@ -1697,11 +1694,6 @@ private boolean isAccessAuthorized(DataFile df, String apiToken) { return true; } - - // We don't want to return false just yet. - // If all else fails, we'll want to use the special WorldMapAuth - // token authentication before we give up. - //return false; } else { // OK, this is a restricted file. @@ -1755,30 +1747,9 @@ private boolean isAccessAuthorized(DataFile df, String apiToken) { } } } - - - // And if all that failed, we'll still check if the download can be authorized based - // on the special WorldMap token: - if ((apiToken != null)&&(apiToken.length()==64)){ - /* - WorldMap token check - - WorldMap tokens are 64 chars in length - - - Use the worldMapTokenServiceBean to verify token - and check permissions against the requested DataFile - */ - if (!(this.worldMapTokenServiceBean.isWorldMapTokenAuthorizedForDataFileDownload(apiToken, df))){ - return false; - } - - // Yes! User may access file - // - logger.fine("WorldMap token-based auth: Token is valid for the requested datafile"); - return true; - - } else if ((apiToken != null)&&(apiToken.length()!=64)) { + if ((apiToken != null)) { // Will try to obtain the user information from the API token, // if supplied: @@ -1794,7 +1765,7 @@ private boolean isAccessAuthorized(DataFile df, String apiToken) { return false; } - if (permissionService.requestOn(createDataverseRequest(user), df).has(Permission.DownloadFile)) { + if (permissionService.requestOn(createDataverseRequest(user), df).has(Permission.DownloadFile)) { if (published) { logger.log(Level.FINE, "API token-based auth: User {0} has rights to access the datafile.", user.getIdentifier()); return true; diff --git a/src/main/java/edu/harvard/iq/dataverse/api/Files.java b/src/main/java/edu/harvard/iq/dataverse/api/Files.java index 81db7f9dec1..defc2c4d9ab 100644 --- a/src/main/java/edu/harvard/iq/dataverse/api/Files.java +++ b/src/main/java/edu/harvard/iq/dataverse/api/Files.java @@ -22,7 +22,6 @@ import edu.harvard.iq.dataverse.datasetutility.OptionalFileParams; import edu.harvard.iq.dataverse.engine.command.DataverseRequest; import edu.harvard.iq.dataverse.engine.command.exception.CommandException; -import edu.harvard.iq.dataverse.engine.command.impl.DeleteMapLayerMetadataCommand; import edu.harvard.iq.dataverse.engine.command.impl.GetDataFileCommand; import edu.harvard.iq.dataverse.engine.command.impl.GetDraftFileMetadataIfAvailableCommand; import edu.harvard.iq.dataverse.engine.command.impl.RedetectFileTypeCommand; @@ -468,31 +467,7 @@ public Response getFileMetadata(@PathParam("id") String fileIdOrPersistentId, @P public Response getFileMetadataDraft(@PathParam("id") String fileIdOrPersistentId, @PathParam("versionId") String versionId, @Context UriInfo uriInfo, @Context HttpHeaders headers, @Context HttpServletResponse response, Boolean getDraft) throws WrappedResponse, Exception { return getFileMetadata(fileIdOrPersistentId, versionId, uriInfo, headers, response, true); } - - // TODO: Rather than only supporting looking up files by their database IDs, consider supporting persistent identifiers. - // TODO: Rename this start with "delete" rather than "get". - @DELETE - @Path("{id}/map") - public Response getMapLayerMetadatas(@PathParam("id") Long idSupplied) { - DataverseRequest dataverseRequest = null; - try { - dataverseRequest = createDataverseRequest(findUserOrDie()); - } catch (WrappedResponse wr) { - return error(BAD_REQUEST, "Couldn't find user to execute command: " + wr.getLocalizedMessage()); - } - DataFile dataFile = fileService.find(idSupplied); - try { - boolean deleted = engineSvc.submit(new DeleteMapLayerMetadataCommand(dataverseRequest, dataFile)); - if (deleted) { - return ok("Map deleted from file id " + dataFile.getId()); - } else { - return error(BAD_REQUEST, "Could not delete map from file id " + dataFile.getId()); - } - } catch (CommandException ex) { - return error(BAD_REQUEST, "Problem trying to delete map from file id " + dataFile.getId() + ": " + ex.getLocalizedMessage()); - } - } - + @Path("{id}/uningest") @POST public Response uningestDatafile(@PathParam("id") String id) { diff --git a/src/main/java/edu/harvard/iq/dataverse/api/Geoconnect.java b/src/main/java/edu/harvard/iq/dataverse/api/Geoconnect.java deleted file mode 100644 index 72991e41fef..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/api/Geoconnect.java +++ /dev/null @@ -1,78 +0,0 @@ -package edu.harvard.iq.dataverse.api; - -import com.mashape.unirest.http.Unirest; -import com.mashape.unirest.http.exceptions.UnirestException; -import com.mashape.unirest.request.GetRequest; -import edu.harvard.iq.dataverse.MapLayerMetadata; -import java.sql.Timestamp; -import java.util.Date; -import java.util.logging.Logger; -import javax.json.Json; -import javax.json.JsonArrayBuilder; -import javax.json.JsonObjectBuilder; -import javax.ws.rs.POST; -import javax.ws.rs.Path; -import javax.ws.rs.PathParam; -import javax.ws.rs.core.Response; -import static javax.ws.rs.core.Response.Status.NOT_FOUND; - -/** - * @todo consolidate with WorldMapRelatedData (/api/worldmap)? - */ -@Path("admin/geoconnect") -public class Geoconnect extends AbstractApiBean { - - private static final Logger logger = Logger.getLogger(Geoconnect.class.getCanonicalName()); - - @POST - @Path("mapLayerMetadatas/check") - public Response checkMapLayerMetadatas() { - JsonArrayBuilder jsonArrayBuilder = Json.createArrayBuilder(); - mapLayerMetadataSrv.findAll().stream().map((unmodified) -> checkStatus(unmodified)).forEach((jsonObjectBuilder) -> { - jsonArrayBuilder.add(jsonObjectBuilder); - }); - return ok(jsonArrayBuilder); - } - - @POST - @Path("mapLayerMetadatas/check/{id}") - public Response checkMapLayerMetadatas(@PathParam("id") Long idSupplied) { - MapLayerMetadata mapLayerMetadata = mapLayerMetadataSrv.find(idSupplied); - if (mapLayerMetadata == null) { - return error(NOT_FOUND, "Could not find MapLayerMetadata based on id " + idSupplied); - } - return ok(checkStatus(mapLayerMetadata)); - } - - private JsonObjectBuilder checkStatus(MapLayerMetadata unmodified) { - JsonObjectBuilder jsonObjectBuilder = Json.createObjectBuilder(); - jsonObjectBuilder.add("fileId", unmodified.getDataFile().getId()); - jsonObjectBuilder.add("mapLayerMetadataId", unmodified.getId()); - jsonObjectBuilder.add("layerLink", unmodified.getLayerLink()); - jsonObjectBuilder.add("fileLandingPage", systemConfig.getDataverseSiteUrl() + "/file.xhtml?fileId=" + unmodified.getDataFile().getId()); - MapLayerMetadata modified = updateLastVerifiedTimeAndStatusCode(unmodified); - if (modified != null) { - jsonObjectBuilder.add("lastVerifiedStatus", modified.getLastVerifiedStatus()); - } else { - jsonObjectBuilder.add("error", "Could not check status of map associate with file id " + unmodified.getDataFile().getId()); - } - return jsonObjectBuilder; - } - - private MapLayerMetadata updateLastVerifiedTimeAndStatusCode(MapLayerMetadata mapLayerMetadata) { - String layerLink = mapLayerMetadata.getLayerLink(); - GetRequest getRequest = Unirest.get(layerLink); - try { - int statusCode = getRequest.asBinary().getStatus(); - mapLayerMetadata.setLastVerifiedStatus(statusCode); - Timestamp now = new Timestamp(new Date().getTime()); - mapLayerMetadata.setLastVerifiedTime(now); - logger.fine("Setting status code to " + statusCode + " and timestamp to " + now + " for MapLayerMetadata id " + mapLayerMetadata.getId() + "."); - return mapLayerMetadataSrv.save(mapLayerMetadata); - } catch (UnirestException ex) { - logger.info("Couldn't update last verfied status code or timestamp: " + ex.getLocalizedMessage()); - return null; - } - } - -} diff --git a/src/main/java/edu/harvard/iq/dataverse/api/WorldMapRelatedData.java b/src/main/java/edu/harvard/iq/dataverse/api/WorldMapRelatedData.java deleted file mode 100755 index e9868f91cee..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/api/WorldMapRelatedData.java +++ /dev/null @@ -1,795 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ -package edu.harvard.iq.dataverse.api; - -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.DataFileServiceBean; -import edu.harvard.iq.dataverse.Dataset; -import edu.harvard.iq.dataverse.DatasetServiceBean; -import edu.harvard.iq.dataverse.DatasetVersion; -import edu.harvard.iq.dataverse.Dataverse; -import edu.harvard.iq.dataverse.DataverseSession; -import edu.harvard.iq.dataverse.FileMetadata; -import edu.harvard.iq.dataverse.MapLayerMetadata; -import edu.harvard.iq.dataverse.MapLayerMetadataServiceBean; -import edu.harvard.iq.dataverse.PermissionServiceBean; -import edu.harvard.iq.dataverse.UserNotification; -import edu.harvard.iq.dataverse.worldmapauth.TokenApplicationTypeServiceBean; -import edu.harvard.iq.dataverse.UserNotificationServiceBean; -import edu.harvard.iq.dataverse.authorization.AuthenticationServiceBean; -import edu.harvard.iq.dataverse.authorization.Permission; -import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; -import edu.harvard.iq.dataverse.worldmapauth.WorldMapToken; -import edu.harvard.iq.dataverse.worldmapauth.WorldMapTokenServiceBean; -import edu.harvard.iq.dataverse.util.SystemConfig; -import java.io.IOException; -import java.io.StringReader; -import java.net.URI; -import java.net.URISyntaxException; -import java.net.URLEncoder; -import java.util.logging.Level; -import java.util.logging.Logger; -import javax.ejb.EJB; -import javax.inject.Inject; -import javax.json.Json; -import javax.json.JsonObject; -import javax.json.JsonObjectBuilder; -import javax.json.stream.JsonParsingException; -import javax.servlet.http.HttpServletRequest; -import javax.ws.rs.GET; -import javax.ws.rs.POST; -import javax.ws.rs.Path; -import javax.ws.rs.PathParam; -import javax.ws.rs.core.Context; -import javax.ws.rs.core.Response; - -/** - * - * @author raprasad - * - * Bean used to communicate information to the WorldMap (currently a Django project) - * - * Within geoconnect, the information sent is validated by the DataverseInfoModel in this project: - * - * https://github.com/IQSS/shared-dataverse-information - * - * @todo Audit these methods to ensure they don't pose a security risk. Consider - * changing the installer so that like "admin" this "worldmap" endpoint is - * blocked out of the box. See - * http://guides.dataverse.org/en/4.6.1/installation/config.html#blocking-api-endpoints - */ -@Path("worldmap") -public class WorldMapRelatedData extends AbstractApiBean { - - private static final Logger logger = Logger.getLogger(WorldMapRelatedData.class.getCanonicalName()); - - - private static final String BASE_PATH = "/api/worldmap/"; - - public static final String MAP_IT_API_PATH_FRAGMENT = "map-it/"; - public static final String MAP_IT_API_PATH = BASE_PATH + MAP_IT_API_PATH_FRAGMENT; - - public static final String MAP_IT_API_TOKEN_ONLY_PATH_FRAGMENT = "map-it-token-only/"; - public static final String MAP_IT_API_TOKEN_ONLY_PATH = BASE_PATH + MAP_IT_API_TOKEN_ONLY_PATH_FRAGMENT; - - public static final String GET_WORLDMAP_DATAFILE_API_PATH_FRAGMENT = "datafile/"; - public static final String GET_WORLDMAP_DATAFILE_API_PATH = BASE_PATH + GET_WORLDMAP_DATAFILE_API_PATH_FRAGMENT; - - - public static final String UPDATE_MAP_LAYER_DATA_API_PATH_FRAGMENT = "update-layer-metadata/"; - public static final String UPDATE_MAP_LAYER_DATA_API_PATH = BASE_PATH + UPDATE_MAP_LAYER_DATA_API_PATH_FRAGMENT; - - public static final String DELETE_MAP_LAYER_DATA_API_PATH_FRAGMENT = "delete-layer-metadata/"; - public static final String DELETE_MAP_LAYER_DATA_API_PATH = BASE_PATH + DELETE_MAP_LAYER_DATA_API_PATH_FRAGMENT; - - public static final String DELETE_WORLDMAP_TOKEN_PATH_FRAGMENT = "delete-token/"; - public static final String DELETE_WORLDMAP_TOKEN_PATH = BASE_PATH + DELETE_WORLDMAP_TOKEN_PATH_FRAGMENT; - - @EJB - MapLayerMetadataServiceBean mapLayerMetadataService; - - @EJB - DatasetServiceBean datasetService; - - @EJB - DataFileServiceBean dataFileService; - - @EJB - UserNotificationServiceBean userNotificationService; - - @EJB - WorldMapTokenServiceBean tokenServiceBean; - - @EJB - TokenApplicationTypeServiceBean tokenAppServiceBean; - - @EJB - AuthenticationServiceBean dataverseUserService; - - @EJB - PermissionServiceBean permissionService; - - @EJB - SystemConfig systemConfig; - - @Inject - DataverseSession session; - - /** - * Create URL for API call to WorldMapRelatedData.mapDataFile(...) - * - * @param dataFileID - * @param dataverseUserID - * @return - */ - public static String getMapItURL(Long dataFileID, Long dataverseUserID){ - if ((dataverseUserID==null)||(dataFileID==null)){ - return null; - } - //test, see if it gets created - return WorldMapRelatedData.MAP_IT_API_PATH + dataFileID + "/" + dataverseUserID; - } - - - // test call to track down problems - // http://127.0.0.1:8080/api/worldmap/t/ - @GET - @Path("t/{identifier}") - public Response checkWorldMapAPI(@Context HttpServletRequest request - , @PathParam("identifier") int identifier) { - - MapLayerMetadata mapLayerMetadata = this.mapLayerMetadataService.find(new Long(identifier)); - logger.info("mapLayerMetadata retrieved. Try to retrieve image (after 1st time):
    " + mapLayerMetadata.getMapImageLink()); - - try { - this.mapLayerMetadataService.retrieveMapImageForIcon(mapLayerMetadata); - } catch (IOException ex) { - logger.info("IOException. Failed to retrieve image. Error:" + ex); - // Logger.getLogger(WorldMapRelatedData.class.getName()).log(Level.SEVERE, null, ex); - }catch(Exception e2){ - logger.info("Failed to retrieve image. Error:" + e2); - } - - return ok( "Looks good " + identifier + " " + mapLayerMetadata.getLayerName()); - } - - - @GET - @Path( MAP_IT_API_PATH_FRAGMENT + "{datafile_id}/{dvuser_id}") - public Response mapDataFile(@Context HttpServletRequest request - , @PathParam("datafile_id") Long datafile_id - , @PathParam("dvuser_id") Long dvuser_id){ - - return this.mapDataFileTokenOnlyOption(request, datafile_id, dvuser_id, false); - } - - @GET - @Path(MAP_IT_API_TOKEN_ONLY_PATH_FRAGMENT + "{datafile_id}/{dvuser_id}") - public Response getMapDataFileToken(@Context HttpServletRequest request - , @PathParam("datafile_id") Long datafile_id - , @PathParam("dvuser_id") Long dvuser_id){ - - //request. - return this.mapDataFileTokenOnlyOption(request, datafile_id, dvuser_id, true); - } - - - /* - Link used within Dataverse for MapIt button - Sends file link to GeoConnect using a Redirect - - */ - //@GET - //@Path( MAP_IT_API_PATH_FRAGMENT + "token-option/{datafile_id}/{dvuser_id}/{token_only}") - private Response mapDataFileTokenOnlyOption(@Context HttpServletRequest request - , Long datafile_id - , Long dvuser_id - , boolean tokenOnly - ){ - - logger.log(Level.INFO, "mapDataFile datafile_id: {0}", datafile_id); - logger.log(Level.INFO, "mapDataFile dvuser_id: {0}", dvuser_id); - - AuthenticatedUser user = null; - - if (session != null) { - if (session.getUser() != null) { - if (session.getUser().isAuthenticated()) { - user = (AuthenticatedUser) session.getUser(); - } - } - } - if (user==null){ - return error(Response.Status.FORBIDDEN, "Not logged in"); - } - - - if (true){ - //return okResponse( "Looks good " + datafile_id); - //tokenAppServiceBean.getGeoConnectApplication(); - //return okResponse("Currently deactivated (mapDataFile)"); - } - - // Check if the user exists - AuthenticatedUser dvUser = dataverseUserService.findByID(dvuser_id); - if ( dvUser == null ){ - return error(Response.Status.FORBIDDEN, "Invalid user"); - } - - // Check if this file exists - DataFile dfile = dataFileService.find(datafile_id); - if (dfile==null){ - return error(Response.Status.NOT_FOUND, "DataFile not found for id: " + datafile_id); - } - - /* - Is the dataset public? - */ - if (!dfile.getOwner().isReleased()){ - return error(Response.Status.FORBIDDEN, "Mapping is only permitted for public datasets/files"); - - } - - // Does this user have permission to edit metadata for this file? - if (!permissionService.request(createDataverseRequest(dvUser)).on(dfile.getOwner()).has(Permission.EditDataset)){ - String errMsg = "The user does not have permission to edit metadata for this file."; - return error(Response.Status.FORBIDDEN, errMsg); - } - - WorldMapToken token = tokenServiceBean.getNewToken(dfile, dvUser); - - if (tokenOnly){ - // Return only the token in a JSON object - final JsonObjectBuilder jsonInfo = Json.createObjectBuilder(); - jsonInfo.add(WorldMapToken.GEOCONNECT_TOKEN_KEY, token.getToken()); - return ok(jsonInfo); - } - - // Redirect to geoconnect url - String callback_url = systemConfig.getDataverseSiteUrl() + GET_WORLDMAP_DATAFILE_API_PATH; - String redirect_url_str = token.getApplication().getMapitLink() + "/" + token.getToken() + "/?cb=" + URLEncoder.encode(callback_url); - - URI redirect_uri; - - try { - redirect_uri = new URI(redirect_url_str); - } catch (URISyntaxException ex) { - return error(Response.Status.NOT_FOUND, "Faile to create URI from: " + redirect_url_str); - } -// Response. - return Response.seeOther(redirect_uri).build(); - - } - - @Deprecated - private String getServerNamePort(HttpServletRequest request){ - if (request == null){ - return ""; - } - String serverName = request.getServerName(); - if (serverName==null){ - return ""; - } - int portNumber = request.getServerPort(); - - String http_prefix = "https://"; - if (serverName.contains("localhost")){ - http_prefix = "http://"; - }else if(serverName.contains("127.0.0.1")){ - http_prefix = "http://"; - } - if (portNumber==80){ - return http_prefix + serverName; - } - return http_prefix + serverName + ":" + portNumber; - - } - - /** - * Parse json looking for the GEOCONNECT_TOKEN_KEY. - * Make sure that the string itself is not null and 64 chars - * - * @param jsonTokenInfo - * @return - */ - private String retrieveTokenValueFromJson(JsonObject jsonTokenInfo){ - if (jsonTokenInfo==null){ - return null; - } - logger.info("retrieveTokenValueFromJson.jsonTokenInfo:"+ jsonTokenInfo); - logger.info("token keys: " + jsonTokenInfo.keySet()); - logger.info("key looked for: " + WorldMapToken.GEOCONNECT_TOKEN_KEY); - if (!jsonTokenInfo.containsKey(WorldMapToken.GEOCONNECT_TOKEN_KEY)){ - logger.warning("Token not found. Permission denied."); - return null; - //return errorResponse( Response.Status.BAD_REQUEST, "Permission denied"); - } - - - //String worldmapTokenParam = worldmapTokenObject.toString(); - String worldmapTokenParam = jsonTokenInfo.getString(WorldMapToken.GEOCONNECT_TOKEN_KEY); - if (worldmapTokenParam==null){ // shouldn't happen - logger.warning("worldmapTokenParam is null when .toString() called. Permission denied."); - return null; - //return errorResponse(Response.Status.UNAUTHORIZED, "No access."); - } - //logger.info("worldmapTokenParam:"+ worldmapTokenParam); - - //logger.info("worldmapTokenParam length:"+ worldmapTokenParam.length()); - if (!(worldmapTokenParam.length()==64)){ - logger.warning("worldmapTokenParam not length 64. Permission denied."); - return null; - // return errorResponse(Response.Status.UNAUTHORIZED, "No access."); - } - return worldmapTokenParam; - } - - - /** - * Retrieve FileMetadata for Use by WorldMap. - * This includes information about the DataFile, Dataset, DatasetVersion, and Dataverse - * - * @param jsonTokenData - * @param request - * @return - */ - @POST - @Path(GET_WORLDMAP_DATAFILE_API_PATH_FRAGMENT)// + "{worldmap_token}") - public Response getWorldMapDatafileInfo(String jsonTokenData, @Context HttpServletRequest request){//, @PathParam("worldmap_token") String worldmapTokenParam) { - if (true){ - //return okResponse("Currently deactivated"); - - // return okResponse("remote server: " + request.getRemoteAddr()); - } - logger.info("API call: getWorldMapDatafileInfo"); - //---------------------------------- - // Auth check: Parse the json message and check for a valid GEOCONNECT_TOKEN_KEY and GEOCONNECT_TOKEN_VALUE - // -- For testing, the GEOCONNECT_TOKEN_VALUE will be dynamic, found in the db - //---------------------------------- - logger.info("(1) jsonTokenData: " + jsonTokenData); - // Parse JSON - JsonObject jsonTokenInfo; - try ( StringReader rdr = new StringReader(jsonTokenData) ) { - jsonTokenInfo = Json.createReader(rdr).readObject(); - } catch ( JsonParsingException jpe ) { - logger.log(Level.SEVERE, "Json: " + jsonTokenData); - return error( Response.Status.BAD_REQUEST, "Error parsing Json: " + jpe.getMessage() ); - } - logger.info("(1a) jsonTokenInfo: " + jsonTokenInfo); - - // Retrieve token string - String worldmapTokenParam = this.retrieveTokenValueFromJson(jsonTokenInfo); - logger.info("(1b) token from JSON: " + worldmapTokenParam); - if (worldmapTokenParam==null){ - return error(Response.Status.BAD_REQUEST, "Token not found in JSON request."); - } - - // Retrieve WorldMapToken and make sure it is valid - // - WorldMapToken wmToken = tokenServiceBean.retrieveAndRefreshValidToken(worldmapTokenParam); - logger.info("(2) token retrieved from db: " + wmToken); - - if (wmToken==null){ - return error(Response.Status.UNAUTHORIZED, "No access. Invalid token."); - } - - // Make sure the token's User still has permissions to access the file - // - logger.info("(3) check permissions"); - if (!(tokenServiceBean.canTokenUserEditFile(wmToken))){ - tokenServiceBean.expireToken(wmToken); - return error(Response.Status.UNAUTHORIZED, "No access. Invalid token."); - } - - - // (1) Retrieve token connected data: DataverseUser, DataFile - // - // Make sure token user and file are still available - // - AuthenticatedUser dvUser = wmToken.getDataverseUser(); - if (dvUser == null) { - return error(Response.Status.NOT_FOUND, "DataverseUser not found for token"); - } - DataFile dfile = wmToken.getDatafile(); - if (dfile == null) { - return error(Response.Status.NOT_FOUND, "DataFile not found for token"); - } - - - // (1a) Retrieve FileMetadata - FileMetadata dfile_meta = dfile.getFileMetadata(); - if (dfile_meta==null){ - return error(Response.Status.NOT_FOUND, "FileMetadata not found"); - } - - // (2) Now get the dataset and the latest DatasetVersion - Dataset dset = dfile.getOwner(); - if (dset==null){ - return error(Response.Status.NOT_FOUND, "Owning Dataset for this DataFile not found"); - } - - // (2a) latest DatasetVersion - // !! How do you check if the lastest version has this specific file? - // - DatasetVersion dset_version = dset.getLatestVersion(); - if (dset_version==null){ - return error(Response.Status.NOT_FOUND, "Latest DatasetVersion for this DataFile not found"); - } - - // (3) get Dataverse - Dataverse dverse = dset.getOwner(); - if (dverse==null){ - return error(Response.Status.NOT_FOUND, "Dataverse for this DataFile's Dataset not found"); - } - - // (4) Roll it all up in a JSON response - final JsonObjectBuilder jsonData = Json.createObjectBuilder(); - - //------------------------------------ - // Type of file, currently: - // - shapefile or - // - tabular file (.tab) with geospatial tag - //------------------------------------ - if (dfile.isShapefileType()){ - jsonData.add("mapping_type", "shapefile"); - }else if (dfile.isTabularData()){ - jsonData.add("mapping_type", "tabular"); - }else{ - logger.log(Level.SEVERE, "This was neither a Shapefile nor a Tabular data file. DataFile id: " + dfile.getId()); - return error( Response.Status.BAD_REQUEST, "Sorry! This file does not have mapping data. Please contact the Dataverse administrator. DataFile id: " + dfile.getId()); - } - - - //------------------------------------ - // DataverseUser Info - //------------------------------------ - jsonData.add("dv_user_id", dvUser.getId()); - jsonData.add("dv_username", dvUser.getUserIdentifier()); - jsonData.add("dv_user_email", dvUser.getEmail()); - - //------------------------------------ - // Dataverse URLs to this server - //------------------------------------ - String serverName = systemConfig.getDataverseSiteUrl(); - jsonData.add("return_to_dataverse_url", dset_version.getReturnToFilePageURL(serverName, dset, dfile)); - jsonData.add("datafile_download_url", dfile.getMapItFileDownloadURL(serverName)); - - //------------------------------------ - // Dataverse - //------------------------------------ - //jsonData.add("dataverse_installation_name", "Harvard Dataverse"); // todo / fix - jsonData.add("dataverse_installation_name", systemConfig.getDataverseSiteUrl()); // is this enough to distinguish a dataverse installation? - - jsonData.add("dataverse_id", dverse.getId()); - jsonData.add("dataverse_name", dverse.getName()); - - String dataverseDesc = dverse.getDescription(); - if (dataverseDesc == null || dataverseDesc.equalsIgnoreCase("")){ - dataverseDesc = ""; - } - jsonData.add("dataverse_description", dataverseDesc); - - //------------------------------------ - // Dataset Info - //------------------------------------ - jsonData.add("dataset_id", dset.getId()); - - //------------------------------------ - // DatasetVersion Info - //------------------------------------ - jsonData.add("dataset_version_id", dset_version.getId()); // database id - jsonData.add("dataset_semantic_version", dset_version.getSemanticVersion()); // major/minor version number, e.g. 3.1 - - jsonData.add("dataset_name", dset_version.getTitle()); - jsonData.add("dataset_citation", dset_version.getCitation(true)); - - jsonData.add("dataset_description", ""); // Need to fix to/do - - jsonData.add("dataset_is_public", dset_version.isReleased()); - - //------------------------------------ - // DataFile/FileMetaData Info - //------------------------------------ - jsonData.add("datafile_id", dfile.getId()); - jsonData.add("datafile_label", dfile_meta.getLabel()); - //jsonData.add("filename", dfile_meta.getLabel()); - jsonData.add("datafile_expected_md5_checksum", dfile.getChecksumValue()); - Long fsize = dfile.getFilesize(); - if (fsize == null){ - fsize= new Long(-1); - } - - jsonData.add("datafile_filesize", fsize); - jsonData.add("datafile_content_type", dfile.getContentType()); - jsonData.add("datafile_create_datetime", dfile.getCreateDate().toString()); - - // restriction status of the DataFile - jsonData.add("datafile_is_restricted", dfile.isRestricted()); - - return ok(jsonData); - - } - - - - /* - For WorldMap/GeoConnect Usage - Create/Updated a MapLayerMetadata object for a given Datafile id - - Example of jsonLayerData String: - { - "layerName": "geonode:boston_census_blocks_zip_cr9" - , "layerLink": "http://localhost:8000/data/geonode:boston_census_blocks_zip_cr9" - , "embedMapLink": "http://localhost:8000/maps/embed/?layer=geonode:boston_census_blocks_zip_cr9" - , "worldmapUsername": "dv_pete" - } - */ - @POST - @Path(UPDATE_MAP_LAYER_DATA_API_PATH_FRAGMENT) - public Response updateWorldMapLayerData(String jsonLayerData){ - - //---------------------------------- - // Auth check: Parse the json message and check for a valid GEOCONNECT_TOKEN_KEY and GEOCONNECT_TOKEN_VALUE - // -- For testing, the GEOCONNECT_TOKEN_VALUE will be dynamic, found in the db - //---------------------------------- - if (jsonLayerData==null){ - logger.log(Level.SEVERE, "jsonLayerData is null"); - return error( Response.Status.BAD_REQUEST, "No JSON data"); - } - - // (1) Parse JSON - // - JsonObject jsonInfo; - try ( StringReader rdr = new StringReader(jsonLayerData) ) { - jsonInfo = Json.createReader(rdr).readObject(); - } catch ( JsonParsingException jpe ) { - logger.log(Level.SEVERE, "Json: " + jsonLayerData); - return error( Response.Status.BAD_REQUEST, "Error parsing Json: " + jpe.getMessage() ); - } - - // Retrieve token string - String worldmapTokenParam = this.retrieveTokenValueFromJson(jsonInfo); - if (worldmapTokenParam==null){ - return error(Response.Status.BAD_REQUEST, "Token not found in JSON request."); - } - - // Retrieve WorldMapToken and make sure it is valid - // - WorldMapToken wmToken = this.tokenServiceBean.retrieveAndRefreshValidToken(worldmapTokenParam); - if (wmToken==null){ - return error(Response.Status.UNAUTHORIZED, "No access. Invalid token."); - } - - // Make sure the token's User still has permissions to access the file - // - if (!(tokenServiceBean.canTokenUserEditFile(wmToken))){ - tokenServiceBean.expireToken(wmToken); - return error(Response.Status.UNAUTHORIZED, "No access. Invalid token."); - } - - - // (2) Make sure the json message has all of the required attributes - // - for (String attr : MapLayerMetadata.MANDATORY_JSON_FIELDS ){ - if (!jsonInfo.containsKey(attr)){ - return error( Response.Status.BAD_REQUEST, "Error parsing Json. Key not found [" + attr + "]\nRequired keys are: " + MapLayerMetadata.MANDATORY_JSON_FIELDS ); - } - } - - // (3) Attempt to retrieve DataverseUser - AuthenticatedUser dvUser = wmToken.getDataverseUser(); - if (dvUser == null) { - return error(Response.Status.NOT_FOUND, "DataverseUser not found for token"); - } - - // (4) Attempt to retrieve DataFile - DataFile dfile = wmToken.getDatafile(); - if (dfile==null){ - return error(Response.Status.NOT_FOUND, "DataFile not found for token"); - } - - // check permissions! - if (!permissionService.request( createDataverseRequest(dvUser) ).on(dfile.getOwner()).has(Permission.EditDataset)){ - String errMsg = "The user does not have permission to edit metadata for this file. (MapLayerMetadata)"; - return error(Response.Status.FORBIDDEN, errMsg); - } - - - // (5) See if a MapLayerMetadata already exists - // - MapLayerMetadata mapLayerMetadata = this.mapLayerMetadataService.findMetadataByDatafile(dfile); - if (mapLayerMetadata==null){ - // Create a new mapLayerMetadata object - mapLayerMetadata = new MapLayerMetadata(); - } - - // Create/Update new MapLayerMetadata object and save it - mapLayerMetadata.setDataFile(dfile); - mapLayerMetadata.setDataset(dfile.getOwner()); - mapLayerMetadata.setLayerName(jsonInfo.getString("layerName")); - mapLayerMetadata.setLayerLink(jsonInfo.getString("layerLink")); - mapLayerMetadata.setEmbedMapLink(jsonInfo.getString("embedMapLink")); - mapLayerMetadata.setWorldmapUsername(jsonInfo.getString("worldmapUsername")); - if (jsonInfo.containsKey("mapImageLink")){ - mapLayerMetadata.setMapImageLink(jsonInfo.getString("mapImageLink")); - } - - // If this was a tabular join set the attributes: - // - isJoinLayer - // - joinDescription - // - String joinDescription = jsonInfo.getString("joinDescription", null); - if ((joinDescription == null) || (joinDescription.equals(""))){ - mapLayerMetadata.setIsJoinLayer(true); - mapLayerMetadata.setJoinDescription(joinDescription); - }else{ - mapLayerMetadata.setIsJoinLayer(false); - mapLayerMetadata.setJoinDescription(null); - } - - // Set the mapLayerLinks - // - String mapLayerLinks = jsonInfo.getString("mapLayerLinks", null); - if ((mapLayerLinks == null) || (mapLayerLinks.equals(""))){ - mapLayerMetadata.setMapLayerLinks(null); - }else{ - mapLayerMetadata.setMapLayerLinks(mapLayerLinks); - } - - - //mapLayer.save(); - MapLayerMetadata savedMapLayerMetadata = mapLayerMetadataService.save(mapLayerMetadata); - if (savedMapLayerMetadata==null){ - logger.log(Level.SEVERE, "Json: " + jsonLayerData); - return error( Response.Status.BAD_REQUEST, "Failed to save map layer! Original JSON: "); - } - - - - // notify user - userNotificationService.sendNotification(dvUser, wmToken.getCurrentTimestamp(), UserNotification.Type.MAPLAYERUPDATED, dfile.getOwner().getLatestVersion().getId()); - - // ------------------------------------------ - // Retrieve a PNG representation from WorldMap - // ------------------------------------------ - try { - logger.info("retrieveMapImageForIcon"); - this.mapLayerMetadataService.retrieveMapImageForIcon(savedMapLayerMetadata); - } catch (IOException ex) { - logger.severe("Failed to retrieve image from WorldMap server"); - Logger.getLogger(WorldMapRelatedData.class.getName()).log(Level.SEVERE, null, ex); - } - - return ok("map layer object saved!"); - - } // end updateWorldMapLayerData - - - - - /* - For WorldMap/GeoConnect Usage - Delete MayLayerMetadata object for a given Datafile - - POST params - { - "GEOCONNECT_TOKEN": "-- some 64 char token which contains a link to the DataFile --" - } - */ - @POST - @Path(DELETE_MAP_LAYER_DATA_API_PATH_FRAGMENT) - public Response deleteWorldMapLayerData(String jsonData){ - - /*---------------------------------- - Parse the json message. - - Auth check: GEOCONNECT_TOKEN - //----------------------------------*/ - if (jsonData==null){ - logger.log(Level.SEVERE, "jsonData is null"); - return error( Response.Status.BAD_REQUEST, "No JSON data"); - } - // (1) Parse JSON - // - JsonObject jsonInfo; - try ( StringReader rdr = new StringReader(jsonData) ) { - jsonInfo = Json.createReader(rdr).readObject(); - } catch ( JsonParsingException jpe ) { - logger.log(Level.SEVERE, "Json: " + jsonData); - return error( Response.Status.BAD_REQUEST, "Error parsing Json: " + jpe.getMessage() ); - } - - // (2) Retrieve token string - String worldmapTokenParam = this.retrieveTokenValueFromJson(jsonInfo); - if (worldmapTokenParam==null){ - return error(Response.Status.BAD_REQUEST, "Token not found in JSON request."); - } - - // (3) Retrieve WorldMapToken and make sure it is valid - // - WorldMapToken wmToken = this.tokenServiceBean.retrieveAndRefreshValidToken(worldmapTokenParam); - if (wmToken==null){ - return error(Response.Status.UNAUTHORIZED, "No access. Invalid token."); - } - - // (4) Make sure the token's User still has permissions to access the file - // - if (!(tokenServiceBean.canTokenUserEditFile(wmToken))){ - tokenServiceBean.expireToken(wmToken); - return error(Response.Status.UNAUTHORIZED, "No access. Invalid token."); - } - - // (5) Attempt to retrieve DataFile and mapLayerMetadata - DataFile dfile = wmToken.getDatafile(); - MapLayerMetadata mapLayerMetadata = this.mapLayerMetadataService.findMetadataByDatafile(dfile); - if (mapLayerMetadata==null){ - return error(Response.Status.EXPECTATION_FAILED, "No map layer metadata found."); - } - - - // (6) Delete the mapLayerMetadata - // (note: permissions checked here for a second time by the mapLayerMetadataService call) - // - if (!(this.mapLayerMetadataService.deleteMapLayerMetadataObject(mapLayerMetadata, wmToken.getDataverseUser()))){ - return error(Response.Status.PRECONDITION_FAILED, "Failed to delete layer"); - }; - - - return ok("Map layer metadata deleted."); - - } // end deleteWorldMapLayerData - - /* - For WorldMap/GeoConnect Usage - Explicitly expire a WorldMap token, removing token from the database - - POST params - { - "GEOCONNECT_TOKEN": "-- some 64 char token which contains a link to the DataFile --" - } - */ - @POST - @Path(DELETE_WORLDMAP_TOKEN_PATH_FRAGMENT) - public Response deleteWorldMapToken(String jsonData){ - - /*---------------------------------- - Parse the json message. - - Auth check: GEOCONNECT_TOKEN - //----------------------------------*/ - if (jsonData==null){ - logger.log(Level.SEVERE, "jsonData is null"); - return error( Response.Status.BAD_REQUEST, "No JSON data"); - } - // (1) Parse JSON - // - JsonObject jsonInfo; - try ( StringReader rdr = new StringReader(jsonData) ) { - jsonInfo = Json.createReader(rdr).readObject(); - } catch ( JsonParsingException jpe ) { - logger.log(Level.SEVERE, "Json: " + jsonData); - return error( Response.Status.BAD_REQUEST, "Error parsing Json: " + jpe.getMessage() ); - } - - // (2) Retrieve token string - String worldmapTokenParam = this.retrieveTokenValueFromJson(jsonInfo); - if (worldmapTokenParam==null){ - return error(Response.Status.BAD_REQUEST, "Token not found in JSON request."); - } - - // (3) Retrieve WorldMapToken - // - WorldMapToken wmToken = this.tokenServiceBean.findByName(worldmapTokenParam); - if (wmToken==null){ - return error(Response.Status.NOT_FOUND, "Token not found."); - } - - // (4) Delete the token - // - tokenServiceBean.deleteToken(wmToken); - return ok("Token has been deleted."); - - } // end deleteWorldMapLayerData - -} // end class diff --git a/src/main/java/edu/harvard/iq/dataverse/authorization/providers/builtin/DataverseUserPage.java b/src/main/java/edu/harvard/iq/dataverse/authorization/providers/builtin/DataverseUserPage.java index f841f2fca74..4f4bf7c8c21 100644 --- a/src/main/java/edu/harvard/iq/dataverse/authorization/providers/builtin/DataverseUserPage.java +++ b/src/main/java/edu/harvard/iq/dataverse/authorization/providers/builtin/DataverseUserPage.java @@ -482,7 +482,6 @@ public void displayNotification() { userNotification.setTheObject(datasetService.find(userNotification.getObjectId())); break; - case MAPLAYERUPDATED: case CREATEDS: case SUBMITTEDDS: case PUBLISHEDDS: @@ -491,10 +490,6 @@ public void displayNotification() { userNotification.setTheObject(datasetVersionService.find(userNotification.getObjectId())); break; - case MAPLAYERDELETEFAILED: - userNotification.setTheObject(fileService.findFileMetadata(userNotification.getObjectId())); - break; - case CREATEACC: userNotification.setTheObject(userNotification.getUser()); break; diff --git a/src/main/java/edu/harvard/iq/dataverse/authorization/users/AuthenticatedUser.java b/src/main/java/edu/harvard/iq/dataverse/authorization/users/AuthenticatedUser.java index 175e5743e93..12161eb1a59 100644 --- a/src/main/java/edu/harvard/iq/dataverse/authorization/users/AuthenticatedUser.java +++ b/src/main/java/edu/harvard/iq/dataverse/authorization/users/AuthenticatedUser.java @@ -12,7 +12,6 @@ import edu.harvard.iq.dataverse.util.BundleUtil; import static edu.harvard.iq.dataverse.util.StringUtil.nonEmpty; import edu.harvard.iq.dataverse.util.json.NullSafeJsonBuilder; -import edu.harvard.iq.dataverse.worldmapauth.WorldMapToken; import java.io.Serializable; import java.sql.Timestamp; import java.util.List; @@ -126,18 +125,7 @@ public class AuthenticatedUser implements User, Serializable { public String getIdentifier() { return IDENTIFIER_PREFIX + userIdentifier; } - - @OneToMany(mappedBy = "dataverseUser", cascade={CascadeType.REMOVE}) - private List worldMapTokens; - - public List getWorldMapTokens() { - return worldMapTokens; - } - public void setWorldMapTokens(List worldMapTokens) { - this.worldMapTokens = worldMapTokens; - } - @OneToMany(mappedBy = "user", cascade={CascadeType.REMOVE}) private List notifications; diff --git a/src/main/java/edu/harvard/iq/dataverse/dataaccess/ImageThumbConverter.java b/src/main/java/edu/harvard/iq/dataverse/dataaccess/ImageThumbConverter.java index ec18f23a5a0..6a69c1f9d77 100644 --- a/src/main/java/edu/harvard/iq/dataverse/dataaccess/ImageThumbConverter.java +++ b/src/main/java/edu/harvard/iq/dataverse/dataaccess/ImageThumbConverter.java @@ -58,7 +58,6 @@ */ public class ImageThumbConverter { public static String THUMBNAIL_SUFFIX = "thumb"; - public static String WORLDMAP_IMAGE_SUFFIX = "img"; public static String THUMBNAIL_MIME_TYPE = "image/png"; public static int DEFAULT_CARDIMAGE_SIZE = 48; @@ -118,8 +117,6 @@ private static boolean isThumbnailAvailable(StorageIO storageIO, int s return generateImageThumbnail(storageIO, size); } else if (file.getContentType().equalsIgnoreCase("application/pdf")) { return generatePDFThumbnail(storageIO, size); - } else if (file.getContentType().equalsIgnoreCase("application/zipped-shapefile") || (file.isTabularData() && file.hasGeospatialTag())) { - return generateWorldMapThumbnail(storageIO, size); } return false; @@ -127,7 +124,7 @@ private static boolean isThumbnailAvailable(StorageIO storageIO, int s } // Note that this method works on ALL file types for which thumbnail - // generation is supported - image/*, pdf, worldmap and geo-tagged tabular; + // generation is supported - image/*, pdf; // not just on images! The type differentiation is handled inside // isThumbnailAvailable(); if the thumbnail is not yet cached, that // method will attempt to generate and cache it. And once it's cached, @@ -283,52 +280,6 @@ private static boolean generateImageThumbnail(StorageIO storageIO, int } - /* - * Note that the "WorldMapThumbnail" generator does the exact same thing as the - * "regular image" thumbnail generator. - * The only difference is that the image generator uses the main file as - * as the source; and the one for the worldmap uses an auxiliary file - * with the ".img" extension (or the swift, etc. equivalent). This file is - * produced and dropped into the Dataset directory (Swift container, etc.) - * the first time the user actually runs WorldMap on the main file. - * Also note that it works the exact same way for tabular-mapped-as-worldmap - * files as well. - */ - private static boolean generateWorldMapThumbnail(StorageIO storageIO, int size) { - - InputStream worldMapImageInputStream = null; - - try { - storageIO.open(); - - Channel worldMapImageChannel = storageIO.openAuxChannel(WORLDMAP_IMAGE_SUFFIX); - if (worldMapImageChannel == null) { - logger.warning("Could not open channel for aux ." + WORLDMAP_IMAGE_SUFFIX + " object; (" + size + ")"); - return false; - } - worldMapImageInputStream = Channels.newInputStream((ReadableByteChannel) worldMapImageChannel); - - long worldMapImageSize = storageIO.getAuxObjectSize(WORLDMAP_IMAGE_SUFFIX); - - if (isImageOverSizeLimit(worldMapImageSize)) { - logger.fine("WorldMap image too large - skipping"); - worldMapImageInputStream.close(); - return false; - } - return generateImageThumbnailFromInputStream(storageIO, size, worldMapImageInputStream); - } catch (FileNotFoundException fnfe) { - logger.fine("No .img file for this worldmap file yet; giving up. Original Error: " + fnfe); - return false; - - } catch (IOException ioex) { - logger.warning("caught IOException trying to open an input stream for worldmap .img file (" + storageIO.getDataFile().getStorageIdentifier() + "). Original Error: " + ioex); - return false; - } finally { - IOUtils.closeQuietly(worldMapImageInputStream); - } - - } - /* * This is the actual workhorse method that does the rescaling of the full * size image: @@ -489,8 +440,6 @@ public static String getImageThumbnailAsBase64(DataFile file, int size) { generated = generateImageThumbnail(storageIO, size); } else if (file.getContentType().equalsIgnoreCase("application/pdf")) { generated = generatePDFThumbnail(storageIO, size); - } else if (file.getContentType().equalsIgnoreCase("application/zipped-shapefile") || (file.isTabularData() && file.hasGeospatialTag())) { - generated = generateWorldMapThumbnail(storageIO, size); } if (generated) { diff --git a/src/main/java/edu/harvard/iq/dataverse/datasetutility/WorldMapPermissionHelper.java b/src/main/java/edu/harvard/iq/dataverse/datasetutility/WorldMapPermissionHelper.java deleted file mode 100644 index 73c0af1d2db..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/datasetutility/WorldMapPermissionHelper.java +++ /dev/null @@ -1,636 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ -package edu.harvard.iq.dataverse.datasetutility; - -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.Dataset; -import edu.harvard.iq.dataverse.DataverseSession; -import edu.harvard.iq.dataverse.DvObject; -import edu.harvard.iq.dataverse.FileMetadata; -import edu.harvard.iq.dataverse.MapLayerMetadata; -import edu.harvard.iq.dataverse.MapLayerMetadataServiceBean; -import edu.harvard.iq.dataverse.PermissionServiceBean; -import edu.harvard.iq.dataverse.SettingsWrapper; -import edu.harvard.iq.dataverse.authorization.Permission; -import edu.harvard.iq.dataverse.authorization.users.GuestUser; -import edu.harvard.iq.dataverse.authorization.users.User; -import edu.harvard.iq.dataverse.settings.SettingsServiceBean; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import javax.faces.view.ViewScoped; -import javax.inject.Inject; -import javax.inject.Named; - -/** - * This class originally encapsulated display logic for the DatasetPage - * - * It allows the following checks without redundantly querying the db to - * check permissions or if MapLayerMetadata exists - * - * - canUserSeeMapDataButton (private) - * - canUserSeeMapDataButtonFromPage (public) - * - canUserSeeMapDataButtonFromAPI (public) - * - * - canSeeMapButtonReminderToPublish (private) - * - canSeeMapButtonReminderToPublishFromPage (public) - * - canSeeMapButtonReminderToPublishFromAPI (public) - * - * - canUserSeeExploreWorldMapButton (private) - * - canUserSeeExploreWorldMapButtonFromPage (public) - * - canUserSeeExploreWorldMapButtonFromAPI (public) - * - * @author rmp553 - */ -@ViewScoped -@Named -public class WorldMapPermissionHelper implements java.io.Serializable { - - @Inject SettingsWrapper settingsWrapper; - @Inject MapLayerMetadataServiceBean mapLayerMetadataService; - @Inject PermissionServiceBean permissionService; - @Inject DataverseSession session; - - - private final Map fileMetadataWorldMapExplore = new HashMap<>(); // { FileMetadata.id : Boolean } - private Map mapLayerMetadataLookup = null; - private final Map datasetPermissionMap = new HashMap<>(); // { Permission human_name : Boolean } - - - public WorldMapPermissionHelper( ){ - - } - - - /** - * Using a DataFile id, retrieve an associated MapLayerMetadata object - * - * The MapLayerMetadata objects have been fetched at page inception by - * "loadMapLayerMetadataLookup()" - */ - public MapLayerMetadata getMapLayerMetadata(DataFile df) { - if (df == null) { - return null; - } - if (mapLayerMetadataLookup == null){ - loadMapLayerMetadataLookup(df.getOwner()); - } - return this.mapLayerMetadataLookup.get(df.getId()); - } - - - /* - * Call this when using the API - * - calls private method canUserSeeExploreWorldMapButton - */ - public boolean canUserSeeExploreWorldMapButtonFromAPI(FileMetadata fm, User user){ - - if (fm == null){ - return false; - } - if (user==null){ - return false; - } - if (!this.permissionService.userOn(user, fm.getDataFile()).has(Permission.DownloadFile)){ - return false; - } - - return this.canUserSeeExploreWorldMapButton(fm, true); - } - - /** - * Call this from a Dataset or File page - * - calls private method canUserSeeExploreWorldMapButton - * - * WARNING: Before calling this, make sure the user has download - * permission for the file!! (See DatasetPage.canDownloadFile()) - * - * @param FileMetadata fm - * @return boolean - */ - public boolean canUserSeeExploreWorldMapButtonFromPage(FileMetadata fm){ - - if (fm==null){ - return false; - } - return this.canUserSeeExploreWorldMapButton(fm, true); - } - - /** - * WARNING: Before calling this, make sure the user has download - * permission for the file!! (See DatasetPage.canDownloadFile()) - * - * Should there be a Explore WorldMap Button for this file? - * See table in: https://github.com/IQSS/dataverse/issues/1618 - * - * (1) Does the file have MapLayerMetadata? - * (2) Are the proper settings in place - * - * @param fm FileMetadata - * @return boolean - */ - private boolean canUserSeeExploreWorldMapButton(FileMetadata fm, boolean permissionsChecked){ - if (fm==null){ - return false; - } - // This is only here to make the public method users think... - if (!permissionsChecked){ - return false; - } - if (this.fileMetadataWorldMapExplore.containsKey(fm.getId())){ - // Yes, return previous answer - //logger.info("using cached result for candownloadfile on filemetadata "+fid); - return this.fileMetadataWorldMapExplore.get(fm.getId()); - } - - /* ----------------------------------------------------- - Does a Map Exist? - ----------------------------------------------------- */ - if (!(this.hasMapLayerMetadata(fm))) { - //See if it does - MapLayerMetadata layer_metadata = mapLayerMetadataService.findMetadataByDatafile(fm.getDataFile()); - if (layer_metadata != null) { - if (mapLayerMetadataLookup == null) { - loadMapLayerMetadataLookup(fm.getDataFile().getOwner()); - } - // yes: keep going... - mapLayerMetadataLookup.put(layer_metadata.getDataFile().getId(), layer_metadata); - } else { - // Nope: no button - this.fileMetadataWorldMapExplore.put(fm.getId(), false); - return false; - } - } - - /* - Is setting for GeoconnectViewMaps true? - Nope? no button - */ - if (!settingsWrapper.isTrueForKey(SettingsServiceBean.Key.GeoconnectViewMaps, false)){ - this.fileMetadataWorldMapExplore.put(fm.getId(), false); - return false; - } - //---------------------------------------------------------------------- - //(0) Before we give it to you - if version is deaccessioned and user - // does not have edit dataset permission then may download - //---------------------------------------------------------------------- - - if (fm.getDatasetVersion().isDeaccessioned()) { - if (this.doesSessionUserHavePermission( Permission.EditDataset, fm)) { - // Yes, save answer and return true - this.fileMetadataWorldMapExplore.put(fm.getId(), true); - return true; - } else { - this.fileMetadataWorldMapExplore.put(fm.getId(), false); - return false; - } - } - //Check for restrictions - - boolean isRestrictedFile = fm.isRestricted(); - - // -------------------------------------------------------------------- - // Is the file Unrestricted ? - // -------------------------------------------------------------------- - if (!isRestrictedFile){ - // Yes, save answer and return true - this.fileMetadataWorldMapExplore.put(fm.getId(), true); - return true; - } - - // -------------------------------------------------------------------- - // Conditions (2) through (4) are for Restricted files - // -------------------------------------------------------------------- - - - if (session.getUser() instanceof GuestUser){ - this.fileMetadataWorldMapExplore.put(fm.getId(), false); - return false; - } - - - // -------------------------------------------------------------------- - // (3) Does the User have DownloadFile Permission at the **Dataset** level - // -------------------------------------------------------------------- - - - if (!this.doesSessionUserHavePermission(Permission.DownloadFile, fm)){ - // Yes, save answer and return true - this.fileMetadataWorldMapExplore.put(fm.getId(), false); - return false; - } - - /* ----------------------------------------------------- - Yes: User can view button! - ----------------------------------------------------- */ - this.fileMetadataWorldMapExplore.put(fm.getId(), true); - return true; - } - - - /* - Check if the FileMetadata.dataFile has an associated MapLayerMetadata object - - The MapLayerMetadata objects have been fetched at page inception by "loadMapLayerMetadataLookup()" - */ - public boolean hasMapLayerMetadata(FileMetadata fm) { - if (fm == null) { - return false; - } - if (fm.getDataFile() == null) { - return false; - } - if (mapLayerMetadataLookup == null) { - loadMapLayerMetadataLookup(fm.getDataFile().getOwner()); - } - return doesDataFileHaveMapLayerMetadata(fm.getDataFile()); - } - - /** - * Check if a DataFile has an associated MapLayerMetadata object - * - * The MapLayerMetadata objects have been fetched at page inception by - * "loadMapLayerMetadataLookup()" - */ - private boolean doesDataFileHaveMapLayerMetadata(DataFile df) { - if (df == null) { - return false; - } - if (df.getId() == null) { - return false; - } - return this.mapLayerMetadataLookup.containsKey(df.getId()); - } - - - /** - * Create a hashmap consisting of { DataFile.id : MapLayerMetadata object} - * - * Very few DataFiles will have associated MapLayerMetadata objects so only - * use 1 query to get them - */ - private void loadMapLayerMetadataLookup(Dataset dataset) { - mapLayerMetadataLookup = new HashMap<>(); - if (dataset == null) { - } - if (dataset.getId() == null) { - return; - } - List mapLayerMetadataList = mapLayerMetadataService.getMapLayerMetadataForDataset(dataset); - if (mapLayerMetadataList == null) { - return; - } - for (MapLayerMetadata layer_metadata : mapLayerMetadataList) { - mapLayerMetadataLookup.put(layer_metadata.getDataFile().getId(), layer_metadata); - } - - }// A DataFile may have a related MapLayerMetadata object - - - /** - * Check if this is a mappable file type. - * - * Currently (2/2016) - * - Shapefile (zipped shapefile) - * - Tabular file with Geospatial Data tag - * - * @param fm - * @return - */ - private boolean isPotentiallyMappableFileType(FileMetadata fm){ - if (fm==null){ - return false; - } - - // Yes, it's a shapefile - // - if (this.isShapefileType(fm)){ - return true; - } - - // Yes, it's tabular with a geospatial tag - // - if (fm.getDataFile().isTabularData()){ - if (fm.getDataFile().hasGeospatialTag()){ - return true; - } - } - return false; - } - - - - public boolean isShapefileType(FileMetadata fm) { - if (fm == null) { - return false; - } - if (fm.getDataFile() == null) { - return false; - } - - return fm.getDataFile().isShapefileType(); - } - - - /** - * Call this from a Dataset or File page - * - calls private method canSeeMapButtonReminderToPublish - * - * WARNING: Assumes user isAuthenicated AND has Permission.EditDataset - * - These checks should be made on the DatasetPage or FilePage which calls this method - * - * - * @param FileMetadata fm - * @return boolean - */ - public boolean canSeeMapButtonReminderToPublishFromPage(FileMetadata fm){ - if (fm == null){ - return false; - } - - if (mapLayerMetadataLookup == null){ - loadMapLayerMetadataLookup(fm.getDatasetVersion().getDataset()); - } - - return this.canSeeMapButtonReminderToPublish(fm, true); - - } - - - /** - * Call this when using the API - * - calls private method canSeeMapButtonReminderToPublish - * - * @param fm - * @param user - * @return - */ - public boolean canSeeMapButtonReminderToPublishFromAPI(FileMetadata fm, User user){ - if (fm == null){ - return false; - } - if (user==null){ - return false; - } - - if (!this.permissionService.userOn(user, fm.getDataFile().getOwner()).has(Permission.EditDataset)){ - return false; - } - - return this.canSeeMapButtonReminderToPublish(fm, true); - - } - - - - /** - * Assumes permissions have been checked!! - * - * See table in: https://github.com/IQSS/dataverse/issues/1618 - * - * Can the user see a reminder to publish button? - * (1) Is the view GeoconnectViewMaps - * (2) Is this file a Shapefile or a Tabular file tagged as Geospatial? - * (3) Is this DataFile released? Yes, don't need reminder - * (4) Does a map already exist? Yes, don't need reminder - */ - private boolean canSeeMapButtonReminderToPublish(FileMetadata fm, boolean permissionsChecked){ - if (fm==null){ - return false; - } - - // Is this user authenticated with EditDataset permission? - // - if (!(isUserAuthenticatedWithEditDatasetPermission(fm))){ - return false; - } - - // This is only here as a reminder to the public method users - if (!permissionsChecked){ - return false; - } - - // (1) Is the view GeoconnectViewMaps - if (!settingsWrapper.isTrueForKey(SettingsServiceBean.Key.GeoconnectCreateEditMaps, false)){ - return false; - } - - - // (2) Is this file a Shapefile or a Tabular file tagged as Geospatial? - // - if (!(this.isPotentiallyMappableFileType(fm))){ - return false; - } - - // (3) Is this DataFile released? Yes, don't need reminder - // - if (fm.getDataFile().isReleased()){ - return false; - } - - // (4) Does a map already exist? Yes, don't need reminder - // - if (this.hasMapLayerMetadata(fm)){ - return false; - } - - // Looks good - // - return true; - } - - /** - * - * WARNING: Assumes user isAuthenicated AND has Permission.EditDataset - * - These checks are made on the DatasetPage which calls this method - * - */ - public boolean canUserSeeMapDataButtonFromPage(FileMetadata fm){ - - if (fm==null){ - return false; - } - - // Is this user authenticated with EditDataset permission? - // - if (!(isUserAuthenticatedWithEditDatasetPermission(fm))){ - return false; - } - if (mapLayerMetadataLookup == null){ - loadMapLayerMetadataLookup(fm.getDatasetVersion().getDataset()); - } - return this.canUserSeeMapDataButton(fm, true); - } - - - - /** - * Call this when using the API - * - calls private method canUserSeeMapDataButton - * - * @param fm - * @param user - * @return - */ - public boolean canUserSeeMapDataButtonFromAPI(FileMetadata fm, User user){ - if (fm == null){ - return false; - } - if (user==null){ - return false; - } - - if (!this.permissionService.userOn(user, fm.getDataFile().getOwner()).has(Permission.EditDataset)){ - return false; - } - - return this.canUserSeeMapDataButton(fm, true); - - } - - /** - * - * WARNING: Assumes user isAuthenicated AND has Permission.EditDataset - * - These checks are made on the DatasetPage which calls this method - * - * Should there be a Map Data Button for this file? - * see table in: https://github.com/IQSS/dataverse/issues/1618 - * (1) Is the user logged in? - * (2) Is this file a Shapefile or a Tabular file tagged as Geospatial? - * (3) Does the logged in user have permission to edit the Dataset to which this FileMetadata belongs? - * (4) Is the create Edit Maps flag set to true? - * (5) Any of these conditions: - * 9a) File Published - * (b) Draft: File Previously published - * @param fm FileMetadata - * @return boolean - */ - private boolean canUserSeeMapDataButton(FileMetadata fm, boolean permissionsChecked){ - if (fm==null){ - return false; - } - - // This is only here as a reminder to the public method users - if (!permissionsChecked){ - - return false; - } - - // (1) Is this file a Shapefile or a Tabular file tagged as Geospatial? - // TO DO: EXPAND FOR TABULAR FILES TAGGED AS GEOSPATIAL! - // - if (!(this.isPotentiallyMappableFileType(fm))){ - - return false; - } - - - // (2) Is the view GeoconnectViewMaps - if (!settingsWrapper.isTrueForKey(SettingsServiceBean.Key.GeoconnectCreateEditMaps, false)){ - - return false; - } - - - // (3) Is File restricted? if Yes - no button. - if (fm.isRestricted() || fm.getDataFile().isRestricted()){ - - return false; - } - - - - // (4) Is File released? - // - if (fm.getDataFile().isReleased()){ - - return true; - } - - // Nope - return false; - } - - private boolean isUserAuthenticatedWithEditDatasetPermission( FileMetadata fm){ - - // Is the user authenticated? - // - if (!(isSessionUserAuthenticated())){ - return false; - } - - // If so, can the logged in user edit the Dataset to which this FileMetadata belongs? - // - if (!this.doesSessionUserHavePermission(Permission.EditDataset, fm)){ - return false; - } - - return true; - } - - public boolean isSessionUserAuthenticated() { - - - if (session == null) { - return false; - } - - if (session.getUser() == null) { - return false; - } - - return session.getUser().isAuthenticated(); - - } - - private boolean doesSessionUserHavePermission(Permission permissionToCheck, FileMetadata fileMetadata){ - if (permissionToCheck == null){ - return false; - } - - DvObject objectToCheck = null; - - if (permissionToCheck.equals(Permission.EditDataset)){ - objectToCheck = fileMetadata.getDatasetVersion().getDataset(); - } else if (permissionToCheck.equals(Permission.DownloadFile)){ - objectToCheck = fileMetadata.getDataFile(); - } - - if (objectToCheck == null){ - return false; - } - - if (this.session.getUser() == null){ - return false; - } - - if (this.permissionService == null){ - return false; - } - - String permName = permissionToCheck.getHumanName(); - - // Has this check already been done? - // - if (this.datasetPermissionMap.containsKey(permName)){ - // Yes, return previous answer - return this.datasetPermissionMap.get(permName); - } - - // Check the permission - // - - boolean hasPermission = this.permissionService.userOn(this.session.getUser(), objectToCheck).has(permissionToCheck); - - // Save the permission - this.datasetPermissionMap.put(permName, hasPermission); - - // return true/false - return hasPermission; - } - - -} diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/CommandContext.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/CommandContext.java index cf0d6e781b0..8e555d5f7a2 100644 --- a/src/main/java/edu/harvard/iq/dataverse/engine/command/CommandContext.java +++ b/src/main/java/edu/harvard/iq/dataverse/engine/command/CommandContext.java @@ -18,7 +18,6 @@ import edu.harvard.iq.dataverse.FileDownloadServiceBean; import edu.harvard.iq.dataverse.GuestbookResponseServiceBean; import edu.harvard.iq.dataverse.GuestbookServiceBean; -import edu.harvard.iq.dataverse.MapLayerMetadataServiceBean; import edu.harvard.iq.dataverse.search.IndexServiceBean; import edu.harvard.iq.dataverse.PermissionServiceBean; import edu.harvard.iq.dataverse.RoleAssigneeServiceBean; @@ -134,8 +133,6 @@ public interface CommandContext { public WorkflowServiceBean workflows(); - public MapLayerMetadataServiceBean mapLayerMetadata(); - public DataCaptureModuleServiceBean dataCaptureModule(); public FileDownloadServiceBean fileDownload(); diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/DeleteDataFileCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/DeleteDataFileCommand.java index bc2ce4286f7..3a7bfb984d3 100644 --- a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/DeleteDataFileCommand.java +++ b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/DeleteDataFileCommand.java @@ -213,31 +213,7 @@ public FileVisitResult postVisitDirectory(final Path dir, final IOException e) } catch (Exception e) { logger.log(Level.WARNING, "Identifier deletion was not successfull:", e.getMessage()); } - - // If there is a Map Layer associated with this file, we may need to - // try and remove the layer data on the WorldMap side. - if (ctxt.mapLayerMetadata().findMetadataByDatafile(doomed) != null) { - // (We need an AuthenticatedUser in order to produce a WorldMap token!) - String id = getUser().getIdentifier(); - id = id.startsWith("@") ? id.substring(1) : id; - AuthenticatedUser authenticatedUser = ctxt.authentication().getAuthenticatedUser(id); - try { - ctxt.mapLayerMetadata().deleteMapLayerFromWorldMap(doomed, authenticatedUser); - // We have the dedicatd command DeleteMapLayerMetadataCommand, but - // there's no need to use it explicitly, since the Dataverse-side - // MapLayerMetadata entity will be deleted by the database - // cascade on the DataFile. -- L.A. Apr. 2020 - - } catch (Exception ex) { - // We are not going to treat it as a fatal condition and bail out, - // but we will send a notification to the user, warning them - // there may still be some data associated with the mapped layer, - // on the WorldMap side, un-deleted: - ctxt.notifications().sendNotification(authenticatedUser, new Timestamp(new Date().getTime()), UserNotification.Type.MAPLAYERDELETEFAILED, doomed.getFileMetadata().getId()); - } - - } DataFile doomedAndMerged = ctxt.em().merge(doomed); ctxt.em().remove(doomedAndMerged); /** diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/DeleteMapLayerMetadataCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/DeleteMapLayerMetadataCommand.java deleted file mode 100644 index e4fd4c18031..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/DeleteMapLayerMetadataCommand.java +++ /dev/null @@ -1,40 +0,0 @@ -package edu.harvard.iq.dataverse.engine.command.impl; - -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.MapLayerMetadata; -import edu.harvard.iq.dataverse.authorization.Permission; -import edu.harvard.iq.dataverse.engine.command.AbstractCommand; -import edu.harvard.iq.dataverse.engine.command.CommandContext; -import edu.harvard.iq.dataverse.engine.command.DataverseRequest; -import edu.harvard.iq.dataverse.engine.command.RequiredPermissions; -import edu.harvard.iq.dataverse.engine.command.exception.CommandException; -import java.util.logging.Logger; - -@RequiredPermissions(Permission.EditDataset) -public class DeleteMapLayerMetadataCommand extends AbstractCommand { - - private static final Logger logger = Logger.getLogger(DeleteMapLayerMetadataCommand.class.getCanonicalName()); - - private final DataFile dataFile; - - public DeleteMapLayerMetadataCommand(DataverseRequest aRequest, DataFile datafile) { - super(aRequest, datafile); - this.dataFile = datafile; - } - - @Override - public Boolean execute(CommandContext ctxt) throws CommandException { - if (dataFile == null) { - return false; - } - MapLayerMetadata mapLayerMetadata = ctxt.mapLayerMetadata().findMetadataByDatafile(dataFile); - if (mapLayerMetadata == null) { - return false; - } - boolean mapDeleted = ctxt.mapLayerMetadata().deleteMapLayerMetadataObject(mapLayerMetadata, getUser()); - logger.info("Boolean returned from deleteMapLayerMetadataObject: " + mapDeleted); - return mapDeleted; - - } - -} diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/FinalizeDatasetPublicationCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/FinalizeDatasetPublicationCommand.java index c062070f566..7a32f515d36 100644 --- a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/FinalizeDatasetPublicationCommand.java +++ b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/FinalizeDatasetPublicationCommand.java @@ -408,45 +408,11 @@ private void updateFiles(Timestamp updateTime, CommandContext ctxt) throws Comma if (dataFile.isRestricted()) { - // A couple things need to happen if the file has been restricted: - // 1. If there's a map layer associated with this shape file, or - // tabular-with-geo-tag file, all that map layer data (that - // includes most of the actual data in the file!) need to be - // removed from WorldMap and GeoConnect, since anyone can get - // download the data from there; - // 2. If this (image) file has been assigned as the dedicated + // If the file has been restricted: + // If this (image) file has been assigned as the dedicated // thumbnail for the dataset, we need to remove that assignment, // now that the file is restricted. - - // Map layer: - - if (ctxt.mapLayerMetadata().findMetadataByDatafile(dataFile) != null) { - // (We need an AuthenticatedUser in order to produce a WorldMap token!) - String id = getUser().getIdentifier(); - id = id.startsWith("@") ? id.substring(1) : id; - AuthenticatedUser authenticatedUser = ctxt.authentication().getAuthenticatedUser(id); - try { - ctxt.mapLayerMetadata().deleteMapLayerFromWorldMap(dataFile, authenticatedUser); - - // If that was successful, delete the layer on the Dataverse side as well: - //SEK 4/20/2017 - //Command to delete from Dataverse side - ctxt.engine().submit(new DeleteMapLayerMetadataCommand(this.getRequest(), dataFile)); - - // RP - Bit of hack, update the datafile here b/c the reference to the datafile - // is not being passed all the way up/down the chain. - // - dataFile.setPreviewImageAvailable(false); - - } catch (IOException ioex) { - // We are not going to treat it as a fatal condition and bail out, - // but we will send a notification to the user, warning them about - // the layer still being out there, un-deleted: - ctxt.notifications().sendNotification(authenticatedUser, getTimestamp(), UserNotification.Type.MAPLAYERDELETEFAILED, dataFile.getFileMetadata().getId()); - } - - } - + // Dataset thumbnail assignment: if (dataFile.equals(getDataset().getThumbnailFile())) { diff --git a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UningestFileCommand.java b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UningestFileCommand.java index 5690e58c3cc..29180f65e36 100644 --- a/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UningestFileCommand.java +++ b/src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UningestFileCommand.java @@ -10,7 +10,6 @@ import edu.harvard.iq.dataverse.DataTable; import edu.harvard.iq.dataverse.DatasetVersion; import edu.harvard.iq.dataverse.FileMetadata; -import edu.harvard.iq.dataverse.MapLayerMetadata; import edu.harvard.iq.dataverse.authorization.Permission; import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; import edu.harvard.iq.dataverse.dataaccess.DataAccess; @@ -165,19 +164,6 @@ protected void executeImpl(CommandContext ctxt) throws CommandException { ctxt.datasetVersion().fixMissingUnf(dv.getId().toString(), true); } - MapLayerMetadata mapLayerMetadata = ctxt.mapLayerMetadata().findMetadataByDatafile(uningest); - if (mapLayerMetadata != null) { - try { - String id = getUser().getIdentifier(); - id = id.startsWith("@") ? id.substring(1) : id; - AuthenticatedUser authenticatedUser = ctxt.authentication().getAuthenticatedUser(id); - ctxt.mapLayerMetadata().deleteMapLayerFromWorldMap(uningest, authenticatedUser); - } catch (Exception e) { - logger.warning("Unable to delete WorldMap file - may not have existed. Data File id: " + uningest.getId()); - } - ctxt.mapLayerMetadata().deleteMapLayerMetadataObject(mapLayerMetadata, getUser()); - } - try{ dataAccess.deleteAllAuxObjects(); } catch (IOException e){ diff --git a/src/main/java/edu/harvard/iq/dataverse/externaltools/ExternalTool.java b/src/main/java/edu/harvard/iq/dataverse/externaltools/ExternalTool.java index 0ae8c9c496d..c996e332bdb 100644 --- a/src/main/java/edu/harvard/iq/dataverse/externaltools/ExternalTool.java +++ b/src/main/java/edu/harvard/iq/dataverse/externaltools/ExternalTool.java @@ -97,17 +97,6 @@ public class ExternalTool implements Serializable { @Column(nullable = true, columnDefinition = "TEXT") private String contentType; - @Transient - private boolean worldMapTool; - - public boolean isWorldMapTool() { - return worldMapTool; - } - - public void setWorldMapTool(boolean worldMapTool) { - this.worldMapTool = worldMapTool; - } - /** * This default constructor is only here to prevent this error at * deployment: diff --git a/src/main/java/edu/harvard/iq/dataverse/settings/SettingsServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/settings/SettingsServiceBean.java index e292ee39722..1f53281b2fa 100644 --- a/src/main/java/edu/harvard/iq/dataverse/settings/SettingsServiceBean.java +++ b/src/main/java/edu/harvard/iq/dataverse/settings/SettingsServiceBean.java @@ -228,20 +228,6 @@ public enum Key { SPSS/sav format, "RData" for R, etc. for example: :TabularIngestSizeLimit:RData */ TabularIngestSizeLimit, - /** - Whether to allow user to create GeoConnect Maps - This boolean effects whether the user sees the map button on - the dataset page and if the ingest will create a shape file - Default is false - */ - GeoconnectCreateEditMaps, - /** - Whether to allow a user to view existing maps - This boolean effects whether a user may see the - Explore World Map Button - Default is false; - */ - GeoconnectViewMaps, /** The message added to a popup upon dataset publish * diff --git a/src/main/java/edu/harvard/iq/dataverse/util/MailUtil.java b/src/main/java/edu/harvard/iq/dataverse/util/MailUtil.java index 88f0b9a61c5..37667d16b55 100644 --- a/src/main/java/edu/harvard/iq/dataverse/util/MailUtil.java +++ b/src/main/java/edu/harvard/iq/dataverse/util/MailUtil.java @@ -44,10 +44,6 @@ public static String getSubjectTextBasedOnNotification(UserNotification userNoti return BundleUtil.getStringFromBundle("notification.email.grant.file.access.subject", rootDvNameAsList); case REJECTFILEACCESS: return BundleUtil.getStringFromBundle("notification.email.rejected.file.access.subject", rootDvNameAsList); - case MAPLAYERUPDATED: - return BundleUtil.getStringFromBundle("notification.email.update.maplayer", rootDvNameAsList); - case MAPLAYERDELETEFAILED: - return BundleUtil.getStringFromBundle("notification.email.maplayer.deletefailed.subject", rootDvNameAsList); case CREATEDS: return BundleUtil.getStringFromBundle("notification.email.create.dataset.subject", rootDvNameAsList); case SUBMITTEDDS: diff --git a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/TokenApplicationType.java b/src/main/java/edu/harvard/iq/dataverse/worldmapauth/TokenApplicationType.java deleted file mode 100644 index 2530f9b1285..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/TokenApplicationType.java +++ /dev/null @@ -1,268 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ - -package edu.harvard.iq.dataverse.worldmapauth; - -import java.sql.Timestamp; -import java.util.Date; -import java.util.logging.Logger; -import javax.persistence.Column; -import javax.persistence.Entity; -import javax.persistence.GeneratedValue; -import javax.persistence.GenerationType; -import javax.persistence.Id; -import javax.persistence.Index; -import javax.persistence.Table; - -/** - * - * @author raprasad - */ -@Entity -@Table(name="worldmapauth_tokentype", indexes = {@Index(name = "application_name", columnList="name", unique = true)}) -public class TokenApplicationType implements java.io.Serializable { - - private static final Logger logger = Logger.getLogger(TokenApplicationType.class.getCanonicalName()); - public static final String DEFAULT_GEOCONNECT_APPLICATION_NAME = "GEOCONNECT"; - public static final int DEFAULT_TOKEN_TIME_LIMIT_MINUTES = 30; - public static final String LOCAL_DEV_MAPIT_LINK = "http://127.0.0.1:8070/shapefile/map-it"; // local GeoConnect - public static final String DEV_MAPIT_LINK = "http://geoconnect.datascience.iq.harvard.edu/shapefile/map-it"; // dev GeoConnect - - - - @Id - @GeneratedValue(strategy = GenerationType.IDENTITY) - private Long id; - - @Column(nullable=false) - private String name; - - private String contactEmail; - - @Column(nullable=true) - private String hostname; - - @Column(nullable=true) - private String ipAddress; - - - @Column(nullable=false) - private String mapitLink; - - //api_permissions = models.ManyToManyField(APIPermission, blank=True, null=True) - @Column(nullable=false, name="timeLimitMinutes", columnDefinition="int default 30") - private int timeLimitMinutes;// = models.IntegerField(default=30, help_text='in minutes') - - @Column(nullable=false, name="timeLimitSeconds", columnDefinition="bigint default 1800") - private long timeLimitSeconds; - - @Column(nullable=false) - private String md5; - - @Column(nullable=false) - private Timestamp created; - - @Column(nullable=false) - private Timestamp modified; - - public TokenApplicationType(){ - - this.name = DEFAULT_GEOCONNECT_APPLICATION_NAME; - this.timeLimitMinutes = DEFAULT_TOKEN_TIME_LIMIT_MINUTES; - this.setCreated(); - this.setModified(); - } - - /** - * Get property id. - * @return Long, value of property id. - */ - public Long getId() { - return this.id; - } - - /** - * Set property id. - * @param id new value of property id. - */ - public void setId(Long id) { - this.id = id; - } - - - /** - * Get property name. - * @return String, value of property name. - */ - public String getName() { - return this.name; - } - - /** - * Set property name. - * @param name new value of property name. - */ - public void setName(String name) { - this.name = name; - } - - - /** - * Get property contactEmail. - * @return String, value of property contactEmail. - */ - public String getContactEmail() { - return this.contactEmail; - } - - /** - * Set property contactEmail. - * @param contactEmail new value of property contactEmail. - */ - public void setContactEmail(String contactEmail) { - this.contactEmail = contactEmail; - } - - - /** - * Get property hostname. - * @return String, value of property hostname. - */ - public String getHostname() { - return this.hostname; - } - - /** - * Set property hostname. - * @param hostname new value of property hostname. - */ - public void setHostname(String hostname) { - this.hostname = hostname; - } - - - /** - * Get property ipAddress. - * @return String, value of property ipAddress. - */ - public String getIpAddress() { - return this.ipAddress; - } - - /** - * Set property ipAddress. - * @param ipAddress new value of property ipAddress. - */ - public void setIpAddress(String ipAddress) { - this.ipAddress = ipAddress; - } - - - /** - * Get property mapitLink. - * @return String, value of property mapitLink. - */ - public String getMapitLink() { - return this.mapitLink; - } - - /** - * Set property mapitLink. - * @param mapitLink new value of property mapitLink. - */ - public void setMapitLink(String mapitLink) { - this.mapitLink = mapitLink; - } - - - /** - * Get property timeLimitMinutes. - * @return int, value of property timeLimitMinutes. - */ - public int getTimeLimitMinutes() { - return this.timeLimitMinutes; - } - - /** - * Set property timeLimitMinutes. - * @param timeLimitMinutes new value of property timeLimitMinutes. Also sets timeLimitSeconds - */ - public void setTimeLimitMinutes(int timeLimitMinutes) { - if (timeLimitMinutes < 1){ - logger.warning("Cannot set Token time limit to less than 1 minute"); - return; - } - this.timeLimitMinutes = timeLimitMinutes; - this.timeLimitSeconds = timeLimitMinutes * 60; - } - - - /** - * Get property timeLimitSeconds. - * @return long, value of property timeLimitSeconds. - */ - public long getTimeLimitSeconds() { - return this.timeLimitSeconds; - } - - - /** - * Get property md5. - * @return String, value of property md5. - */ - public String getMd5() { - return this.md5; - } - - /** - * Set property md5. - * @param md5 new value of property md5. - */ - public void setMd5(String md5) { - this.md5 = md5; - } - - - /** - * Get property modified. - * @return Timestamp, value of property modified. - */ - public Timestamp getModified() { - return this.modified; - } - - /** - * Set property modified. - */ - public void setModified() { - this.modified = new Timestamp(new Date().getTime()); - } - - - /** - * Get property created. - * @return Timestamp, value of property created. - */ - public Timestamp getCreated() { - return this.created; - } - - /** - * Set property created. - */ - public void setCreated() { - this.created = new Timestamp(new Date().getTime()); - } - - - @Override - public String toString() { - return "edu.harvard.iq.dataverse.worldmapauth.TokenApplicationType[id=" + this.id + "]"; - - //return "WorldMap Layer: " + this.layerName + " for DataFile: " + this.dataFile.toString(); - } - -} diff --git a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/TokenApplicationTypeServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/worldmapauth/TokenApplicationTypeServiceBean.java deleted file mode 100644 index 24c2f0c1c1e..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/TokenApplicationTypeServiceBean.java +++ /dev/null @@ -1,153 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ - -package edu.harvard.iq.dataverse.worldmapauth; - -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.util.List; -import java.util.logging.Logger; -import javax.ejb.Stateless; -import javax.inject.Named; -import javax.persistence.EntityManager; -import javax.persistence.NoResultException; -import javax.persistence.PersistenceContext; -import javax.persistence.TypedQuery; - -/** - * - * @author raprasad - */ -@Stateless -@Named -public class TokenApplicationTypeServiceBean { - - private static final Logger logger = Logger.getLogger(TokenApplicationTypeServiceBean.class.getCanonicalName()); - - @PersistenceContext(unitName = "VDCNet-ejbPU") - private EntityManager em; - - public TokenApplicationType getGeoConnectApplication(){ - logger.info("--getGeoConnectApplication--"); - TokenApplicationType tat = this.findByName(TokenApplicationType.DEFAULT_GEOCONNECT_APPLICATION_NAME); - if (tat != null){ - logger.info("-- Got it!!"); - return tat; - } - // Make a default application for GeoConnect - tat = new TokenApplicationType(); - tat.setName(TokenApplicationType.DEFAULT_GEOCONNECT_APPLICATION_NAME); - tat.setContactEmail("info@iq.harvard.edu"); - tat.setHostname("geoconnect.datascience.iq.harvard.edu"); - tat.setIpAddress("140.247.115.127"); - tat.setTimeLimitMinutes(TokenApplicationType.DEFAULT_TOKEN_TIME_LIMIT_MINUTES); - //tat.setMapitLink(TokenApplicationType.LOCAL_DEV_MAPIT_LINK); - tat.setMapitLink(TokenApplicationType.DEV_MAPIT_LINK); - - return this.save(tat); - - //return null; - } - public TokenApplicationType find(Object pk) { - if (pk==null){ - return null; - } - return em.find(TokenApplicationType.class, pk); - } - - /** - * - * Convert string to md5 hash - * - import hashlib - m = hashlib.md5() - m.update("Give me python or give me...more time, more time -- c.mena") - m.hexdigest() #'266cf94160a22fe1ef118c907379cd60' - * @param stringToHash - * @return - */ - public String getMD5Hash(String stringToHash){ - if (stringToHash==null){ - return null; - } - MessageDigest md; - try { - md = MessageDigest.getInstance("MD5"); - } catch (NoSuchAlgorithmException ex) { - logger.severe("Failed to set TokenApplicationType for 'Map It' request!!!"); - return null; - } - md.update(stringToHash.getBytes()); - - byte[] mdbytes = md.digest(); - StringBuilder sb = new StringBuilder(""); - for (int i = 0; i < mdbytes.length; i++) { - sb.append(Integer.toString((mdbytes[i] & 0xff) + 0x100, 16).substring(1)); - } - return sb.toString(); - - - } - - public TokenApplicationType save( TokenApplicationType tokenApp ) { - - if (tokenApp==null){ - return null; - } - - if (tokenApp.getName()==null){ - tokenApp.setName(TokenApplicationType.DEFAULT_GEOCONNECT_APPLICATION_NAME); - } - - if (tokenApp.getMapitLink()==null){ - logger.warning("mapitLink is missing for tokenApp"); - return null; - } - - // Set time limit minutes - Integer time_limit_minutes = tokenApp.getTimeLimitMinutes(); - if (time_limit_minutes == null){ - tokenApp.setTimeLimitMinutes(TokenApplicationType.DEFAULT_TOKEN_TIME_LIMIT_MINUTES); - // (also sets the time limit seconds) - } - - // set md5 - tokenApp.setMd5(this.getMD5Hash(tokenApp.getName())); - - if ( tokenApp.getId() == null ) { - tokenApp.setCreated(); - em.persist(tokenApp); - logger.fine("New tokenApp saved"); - return tokenApp; - } else { - tokenApp.setModified(); - logger.fine("Existing tokenApp saved"); - return em.merge( tokenApp ); - } - } - - - public TokenApplicationType findByName(String name){ - if (name == null){ - return null; - } - try{ - return em.createQuery("select m from TokenApplicationType m WHERE m.name=:name", TokenApplicationType.class) - .setParameter("name", name) - .getSingleResult(); - } catch ( NoResultException nre ) { - return null; - } - } - - - public List getAllTokenApplicationTypes(){ - String qr = "select object(o) from TokenApplicationType order by o.modified desc"; - TypedQuery query = em.createQuery(qr, TokenApplicationType.class); - return query.getResultList(); - } - -} \ No newline at end of file diff --git a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapToken.java b/src/main/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapToken.java deleted file mode 100644 index ccae1d1a920..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapToken.java +++ /dev/null @@ -1,349 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ - -package edu.harvard.iq.dataverse.worldmapauth; - -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.sql.Timestamp; -import java.util.Date; -import java.util.Random; -import java.util.logging.Logger; -import javax.persistence.Column; -import javax.persistence.Entity; -import javax.persistence.GeneratedValue; -import javax.persistence.GenerationType; -import javax.persistence.Id; -import javax.persistence.Index; -import javax.persistence.JoinColumn; -import javax.persistence.ManyToOne; -import javax.persistence.Table; -import javax.persistence.Transient; - -/** - * - * @author raprasad - */ -@Entity -@Table(name="worldmapauth_token" - , indexes = {@Index(name = "token_value", columnList="token", unique = true) - , @Index(columnList="application_id") - , @Index(columnList="datafile_id") - , @Index(columnList="dataverseuser_id") - }) -public class WorldMapToken implements java.io.Serializable { - - @Transient - public static final String GEOCONNECT_TOKEN_KEY = "GEOCONNECT_TOKEN"; - @Transient - public static final long MAX_HOURS_TOKEN_CAN_BE_USED = 10; - - private static final Logger logger = Logger.getLogger(WorldMapToken.class.getCanonicalName()); - - @Id - @GeneratedValue(strategy = GenerationType.IDENTITY) - private Long id; - - @Column(nullable=true) - private String token; // save it to get an id, then set token - - @ManyToOne - @JoinColumn(nullable=false) - private TokenApplicationType application; - - @ManyToOne - @JoinColumn(nullable=false) - private AuthenticatedUser dataverseUser; - - @ManyToOne - @JoinColumn(nullable=false) - private DataFile dataFile; - - - @Column(nullable=false) - private boolean hasExpired; - - @Column(nullable=false) - private Timestamp lastRefreshTime; - - @Column(nullable=false) - private Timestamp modified; - - @Column(nullable=false) - private Timestamp created; - - public WorldMapToken(){ - this.setLastRefreshTime(this.getCurrentTimestamp()); - this.setCreated(); - this.setModified(); - } - /** - * Get property token. - * @return String, value of property token. - */ - public String getToken() { - return this.token; - } - - public Long getId(){ - return this.id; - } - /** - * Set property token. - * @param token new value of property token. - */ - public boolean setToken(){ - if (this.token != null){ - return false; - } - if ((this.dataFile==null)||(this.dataverseUser==null)){ - return false; - } - MessageDigest md; - try { - md = MessageDigest.getInstance("SHA-256"); - } catch (NoSuchAlgorithmException ex) { - logger.severe("Failed to set token for 'Map It' request!!!"); - return false; - } - md.update(this.getCurrentTimestamp().toString().getBytes()); - md.update(this.dataFile.toString().getBytes()); - md.update(this.dataverseUser.toString().getBytes()); - Random rand = new Random(); - Integer rnd_int = rand.nextInt(); - md.update(rnd_int.byteValue()); - - byte[] mdbytes = md.digest(); - StringBuilder sb = new StringBuilder(""); - for (int i = 0; i < mdbytes.length; i++) { - sb.append(Integer.toString((mdbytes[i] & 0xff) + 0x100, 16).substring(1)); - } - this.token = sb.toString(); - //this.token = md.toString(); - return true; - } - - - /** - * Get property application. - * @return TokenApplicationType, value of property application. - */ - public TokenApplicationType getApplication() { - return this.application; - } - - /** - * Set property application. - * @param application new value of property application. - */ - public void setApplication(TokenApplicationType application) { - this.application = application; - } - - - /** - * Get property dataverseUser. - * @return DataverseUser, value of property dataverseUser. - */ - public AuthenticatedUser getDataverseUser() { - return this.dataverseUser; - } - - /** - * Set property dataverseUser. - * @param dataverseUser new value of property dataverseUser. - */ - public void setDataverseUser(AuthenticatedUser dataverseUser) { - this.dataverseUser = dataverseUser; - } - - - /** - * Get property datafile. - * @return DataFile, value of property datafile. - */ - public DataFile getDatafile() { - return this.dataFile; - } - - /** - * Set property datafile. - * @param datafile new value of property datafile. - */ - public void setDatafile(DataFile dataFile) { - this.dataFile = dataFile; - } - - - /** - * Get property hasExpired. - * @return boolean, value of property hasExpired. - */ - public boolean getHasExpired() { - return this.hasExpired; - } - - /** - * Set property hasExpired. - * @param hasExpired new value of property hasExpired. - */ - public void setHasExpired(boolean hasExpired) { - this.hasExpired = hasExpired; - } - - - /** - * Get property lastRefreshTime. - * @return Timestamp, value of property lastRefreshTime. - */ - public Timestamp getLastRefreshTime() { - return this.lastRefreshTime; - } - - /** - * Set property lastRefreshTime. - * @param lastRefreshTime new value of property lastRefreshTime. - */ - public void setLastRefreshTime(Timestamp lastRefreshTime) { - this.lastRefreshTime = this.getCurrentTimestamp(); - } - - - /** - * Get property modified. - * @return Timestamp, value of property modified. - */ - public Timestamp getModified() { - return this.modified; - } - - /** - * Set property modified. - */ - public void setModified() { - this.modified = this.getCurrentTimestamp(); - } - - - /** - * Get property created. - * @return Timestamp, value of property created. - */ - public Timestamp getCreated() { - return this.created; - } - - - /** - * Set property created. - */ - public void setCreated() { - this.created = this.getCurrentTimestamp(); - } - - public boolean hasTokenExpired(){ - logger.fine("hasTokenExpired"); - // long currentTime = this.getCurrentTimestamp();//new Date().getTime(); - return this.hasTokenExpired(this.getCurrentTimestamp()); - } - - - private long getElapsedHours(Timestamp currentTime, Timestamp oldTime){ - - // If values are null, send back an elapsed time - if ((currentTime==null)||(oldTime==null)){ - return this.application.getTimeLimitSeconds() + 100000; - } - - long milliseconds1 = oldTime.getTime(); - long milliseconds2 = currentTime.getTime(); - - long diff = milliseconds2 - milliseconds1; - //long diffSeconds = diff / 1000; - //long diffMinutes = diff / (60 * 1000); - long diffHours = diff / (60 * 60 * 1000); - //long diffDays = diff / (24 * 60 * 60 * 1000); - - return diffHours; - } - - private long getElapsedSeconds(Timestamp currentTime, Timestamp oldTime){ - - // If values are null, send back an elapsed time - if ((currentTime==null)||(oldTime==null)){ - return this.application.getTimeLimitSeconds() + 100000; - } - - long milliseconds1 = oldTime.getTime(); - long milliseconds2 = currentTime.getTime(); - - long diff = milliseconds2 - milliseconds1; - long diffSeconds = diff / 1000; - //long diffMinutes = diff / (60 * 1000); - //long diffHours = diff / (60 * 60 * 1000); - //long diffDays = diff / (24 * 60 * 60 * 1000); - - return diffSeconds; - } - - public boolean hasTokenExpired(Timestamp currentTimestamp){ - logger.fine("hasTokenExpired (w/timestamp)"); - - if (this.getHasExpired()){ - return true; - } - - if ((currentTimestamp==null)||(this.lastRefreshTime==null)){ - this.expireToken(); - return true; - } - logger.fine("currentTimestamp: " + currentTimestamp); - logger.fine("lastRefreshTime: " + lastRefreshTime); - - //System.out.println(" ..pre diff: "+ currentTimestamp); - long hours = this.getElapsedHours(currentTimestamp, this.created); - logger.fine("elapsed hours: " + hours); - - if (hours > MAX_HOURS_TOKEN_CAN_BE_USED){ - this.expireToken(); - return true; - } - - long diffSeconds = this.getElapsedSeconds(currentTimestamp, this.lastRefreshTime); - //System.out.println(" ..diffSeconds: "+ diffSeconds); - //logger.fine("this.application.getTimeLimitSeconds: "+ this.application.getTimeLimitSeconds()); - if (diffSeconds > this.application.getTimeLimitSeconds()){ - this.expireToken(); - return true; - } - return false; - - } - - public void expireToken(){ - this.setHasExpired(true); - } - - - public Timestamp getCurrentTimestamp(){ - return new Timestamp(new Date().getTime()); - } - - public boolean refreshToken(){ - - if (this.getHasExpired()){ - return false; - } - Timestamp currentTime = this.getCurrentTimestamp(); - if (this.hasTokenExpired(currentTime)){ - return false; - } - this.setLastRefreshTime(currentTime); - return true; - } -} diff --git a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenServiceBean.java b/src/main/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenServiceBean.java deleted file mode 100644 index 7ca5c62b1f3..00000000000 --- a/src/main/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenServiceBean.java +++ /dev/null @@ -1,314 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ - -package edu.harvard.iq.dataverse.worldmapauth; - -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.PermissionServiceBean; -import edu.harvard.iq.dataverse.authorization.Permission; -import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; -import java.util.List; -import java.util.logging.Level; -import java.util.logging.Logger; -import javax.ejb.EJB; -import javax.ejb.Stateless; -import javax.ejb.TransactionAttribute; -import static javax.ejb.TransactionAttributeType.REQUIRES_NEW; -import javax.inject.Named; -import javax.persistence.EntityManager; -import javax.persistence.NoResultException; -import javax.persistence.PersistenceContext; -import javax.persistence.TypedQuery; -import javax.ws.rs.WebApplicationException; -import javax.ws.rs.core.Response; - -/** - * - * @author raprasad - */ -@Stateless -@Named -public class WorldMapTokenServiceBean { - - @EJB - PermissionServiceBean permissionService; - - @EJB - TokenApplicationTypeServiceBean tokenApplicationService; - - private static final Logger logger = Logger.getLogger(TokenApplicationTypeServiceBean.class.getCanonicalName()); - - @PersistenceContext(unitName = "VDCNet-ejbPU") - private EntityManager em; - - /** - * - * Retrieve a fresh token to map a Dataverse file on GeoConnect - * - * @param dataFile - * @param dvUser - * @return WorldMapToken - */ - @TransactionAttribute(REQUIRES_NEW) - public WorldMapToken getNewToken(DataFile dataFile, AuthenticatedUser dvUser){ - if ((dataFile==null)||(dvUser==null)){ - logger.severe("dataFile or dvUser is null"); - return null; - } - WorldMapToken token = new WorldMapToken(); - token.setApplication(tokenApplicationService.getGeoConnectApplication()); - token.setDatafile(dataFile); - token.setDataverseUser(dvUser); - token.setModified(); - token.setToken(); - return this.save(token); - } - - public WorldMapToken find(Object pk) { - if (pk==null){ - return null; - } - return em.find(WorldMapToken.class, pk); - } - - - /** - * Expire the token, set the "hasExpired" flag to True - * - * @param wmToken - */ - public void expireToken(WorldMapToken wmToken){ - if (wmToken==null){ - return; - } - wmToken.expireToken(); - em.merge(wmToken); - } - - /** - * Expire and then Delete the token - * (The expire is a bit extraneous) - * - * @param wmToken - */ - public void deleteToken(WorldMapToken wmToken){ - - if (wmToken==null){ - return; - } - em.remove(em.merge(wmToken)); - } - - /* - Remove expired tokens from the database - */ - public void deleteExpiredTokens(){ - - TypedQuery query = em.createQuery("select object(w) from WorldMapToken as w where w.hasExpired IS TRUE", WorldMapToken.class);// order by o.name"); - List tokenList = query.getResultList(); - for (WorldMapToken wmToken : tokenList) { - // em.remove(token); - em.remove(em.merge(wmToken)); - } - } - - public WorldMapToken save( WorldMapToken dvToken ) { - - if (dvToken==null){ - return null; - } - - if (dvToken.getToken()==null){ - dvToken.setToken(); - } - - if (dvToken.getApplication().getMapitLink()==null){ - logger.warning("mapitLink is missing for token_application"); - return null; - } - - TokenApplicationType dat = dvToken.getApplication(); - if (dat==null){ - logger.warning("TokenApplicationType is null for WorldMapToken"); - return null; - } - - if ( dvToken.getId()== null ) { - em.persist(dvToken); - logger.fine("New token_application saved"); - return dvToken; - } else { - logger.fine("Existing token_application saved"); - return em.merge( dvToken ); - } - } - - - public WorldMapToken findByName(String token){ - if (token == null){ - return null; - } - - try{ - return em.createQuery("select m from WorldMapToken m WHERE m.token=:token", WorldMapToken.class) - .setParameter("token", token) - .getSingleResult(); - } catch ( NoResultException nre ) { - return null; - } - } - - - - /** - * Given a string token, retrieve the related WorldMapToken object - * - * @param worldmapTokenParam - * @return WorldMapToken object (if it hasn't expired) - */ - public WorldMapToken retrieveAndRefreshValidToken(String worldmapTokenParam){ - if (worldmapTokenParam==null){ - logger.warning("worldmapTokenParam is null. Permission denied."); - return null; - } - WorldMapToken wmToken = this.findByName(worldmapTokenParam); - if (wmToken==null){ - logger.warning("WorldMapToken not found for '" + worldmapTokenParam + "'. Permission denied."); - return null; - } - if (wmToken.hasTokenExpired()){ - em.remove(em.merge(wmToken)); // Delete expired token from the database. - logger.warning("WorldMapToken has expired. Permission denied."); - return null; - } - wmToken.refreshToken(); - - logger.info("WorldMapToken refreshed."); - this.save(wmToken); - - return wmToken; - } - - - /* - Can the user connected to the WorldMapToken still - edit the dataset for the file connected to the token? - */ - public boolean canTokenUserEditFile(WorldMapToken wmToken){ - if (wmToken==null){ - return false; - } - if (permissionService.userOn(wmToken.getDataverseUser(), wmToken.getDatafile()).has(Permission.EditDataset)) { - logger.info("WorldMap token-based auth: Token's User is still authorized to edit the dataset for the datafile."); - return true; - } - return false; - - } - - /* - Can the user connected to the WorldMapToken still - download the file connected to the token? - */ - private boolean canTokenUserDownloadFile(WorldMapToken wmToken){ - if (wmToken==null){ - return false; - } - if (permissionService.userOn(wmToken.getDataverseUser(), wmToken.getDatafile()).has(Permission.DownloadFile)) { - logger.info("WorldMap token-based auth: Token's User is still authorized to download the datafile."); - return true; - } - return false; - - } - - - /* - Given a string for a WorldMapToken and DataFile, check: - - (1) Is this token valid? - (2) Does the token correspond to this DataFile? - - @return boolean if token is valid and corresponds to the given DataFile - - */ - public boolean isWorldMapTokenAuthorizedForDataFileDownload(String worldmapTokenParam, DataFile df){ - logger.info("-- isWorldMapTokenAuthorizedForDataFileworldmapTokenParam " + worldmapTokenParam); - - if ((worldmapTokenParam == null)||(df == null)){ - logger.info("nope: worldmapTokenParam or data file is null"); - return false; - } - - // Check 1: Is this a valid WorldMap token? - // - WorldMapToken token = this.retrieveAndRefreshValidToken(worldmapTokenParam); - if (token==null){ - logger.info("WorldMap token-based auth: Token is not invalid."); - return false; - } - - - // Check 2: Does this WorldMap token's datafile match the requested datafile? - // - if (!(token.getDatafile().getId().equals(df.getId()))){ - logger.info("WorldMap token-based auth: Token's datafile does not match the requested datafile."); - return false; - } - - // Check 3: Does this WorldMap token's user have permission for the requested datafile? - // - if (!(this.canTokenUserDownloadFile(token))){ - logger.info("WorldMap token-based auth: Token's User is not authorized for the requested datafile."); - return false; - } - - - logger.info("WorldMap token-based auth: Token is valid for the requested datafile"); - return true; - - } - - - /* - Given a string for a WorldMapToken, check: - - (1) Is this token valid? - (2) Re-verify that can edit the Dataset connected to the token's DataFile - - @return boolean if token is valid and corresponds to the given DataFile - - */ - public boolean isWorldMapTokenAuthorizedForMetadataRetrievalAndUpdates(String worldmapTokenParam){ - logger.info("-- isWorldMapTokenAuthorizedForDataFile?"); - if (worldmapTokenParam == null){ - logger.info("nope: worldmapTokenParam or data file is null"); - return false; - } - - // Check 1: Is this a valid WorldMap token? - // - WorldMapToken token = this.retrieveAndRefreshValidToken(worldmapTokenParam); - if (token==null){ - logger.info("WorldMap token-based auth: Token is not invalid."); - return false; - } - - - - // Check 2: Does this WorldMap token's user still have edit permission for the requested datafile? - // - if (!(this.canTokenUserEditFile(token))){ - logger.info("WorldMap token-based auth: Token's User is not authorized for the requested datafile."); - return false; - } - - - logger.info("WorldMap token-based auth: Token is valid for the requested datafile"); - return true; - - } -} // end of class diff --git a/src/main/java/propertyFiles/Bundle.properties b/src/main/java/propertyFiles/Bundle.properties index 9c3abafc77d..78c8e8ed875 100755 --- a/src/main/java/propertyFiles/Bundle.properties +++ b/src/main/java/propertyFiles/Bundle.properties @@ -176,7 +176,6 @@ wasPublished=, was published in wasReturnedByReviewer=, was returned by the curator of # TODO: Confirm that "toReview" can be deleted. toReview=Don't forget to publish it or send it back to the contributor! -worldMap.added=dataset had a WorldMap layer data added to it. # Bundle file editors, please note that "notification.welcome" is used in a unit test. notification.welcome=Welcome to {0}! Get started by adding or finding data. Have questions? Check out the {1}. Want to test out Dataverse features? Use our {2}. Also, check for your welcome email to verify your address. notification.demoSite=Demo Site @@ -193,8 +192,6 @@ notification.wasPublished={0} was published in {1}. notification.publishFailedPidReg={0} in {1} could not be published due to a failure to register, or update the Global Identifier for the dataset or one of the files in it. Contact support if this continues to happen. notification.ingestCompleted=Dataset {1} ingest has successfully finished. notification.ingestCompletedWithErrors=Dataset {1} ingest has finished with errors. -notification.worldMap.added={0}, dataset had WorldMap layer data added to it. -notification.maplayer.deletefailed=Failed to delete the map layer associated with the restricted file {0} from WorldMap. Please try again, or contact WorldMap and/or Dataverse support. (Dataset: {1}) notification.generic.objectDeleted=The dataverse, dataset, or file for this notification has been deleted. notification.access.granted.dataverse=You have been granted the {0} role for {1}. notification.access.granted.dataset=You have been granted the {0} role for {1}. @@ -643,9 +640,6 @@ notification.email.create.dataset.subject={0}: Your dataset has been created notification.email.request.file.access.subject={0}: Access has been requested for a restricted file notification.email.grant.file.access.subject={0}: You have been granted access to a restricted file notification.email.rejected.file.access.subject={0}: Your request for access to a restricted file has been rejected -notification.email.update.maplayer={0}: WorldMap layer added to dataset -notification.email.maplayer.deletefailed.subject={0}: Failed to delete WorldMap layer -notification.email.maplayer.deletefailed.text=We failed to delete the WorldMap layer associated with the restricted file {0}, and any related data that may still be publicly available on the WorldMap site. Please try again, or contact WorldMap and/or Dataverse support. (Dataset: {1}) notification.email.submit.dataset.subject={0}: Your dataset has been submitted for review notification.email.publish.dataset.subject={0}: Your dataset has been published notification.email.publishFailure.dataset.subject={0}: Failed to publish your dataset @@ -671,7 +665,6 @@ notification.email.wasSubmittedForReview={0} (view at {1}) was submitted for rev notification.email.wasReturnedByReviewer={0} (view at {1}) was returned by the curator of {2} (view at {3}). notification.email.wasPublished={0} (view at {1}) was published in {2} (view at {3}). notification.email.publishFailedPidReg={0} (view at {1}) in {2} (view at {3}) could not be published due to a failure to register, or update the Global Identifier for the dataset or one of the files in it. Contact support if this continues to happen. -notification.email.worldMap.added={0} (view at {1}) had WorldMap layer data added to it. notification.email.closing=\n\nYou may contact us for support at {0}.\n\nThank you,\n{1} notification.email.assignRole=You are now {0} for the {1} "{2}" (view at {3}). notification.email.revokeRole=One of your roles for the {0} "{1}" has been revoked (view at {2}). @@ -1355,7 +1348,6 @@ dataset.compute.computeBtn=Compute dataset.compute.computeBatchListHeader=Compute Batch dataset.compute.computeBatchRestricted=This dataset contains restricted files you may not compute on because you have not been granted access. dataset.delete.error=Could not deaccession the dataset because the {0} update failed. -dataset.publish.worldMap.deleteConfirm=Please note that your data and map on WorldMap will be removed due to restricted file access changes in this dataset version which you are publishing. Do you want to continue? dataset.publish.workflow.message=Publish in Progress dataset.publish.workflow.inprogress=This dataset is locked until publication. dataset.pidRegister.workflow.inprogress=The dataset is locked while the persistent identifiers are being registered or updated, and/or the physical files are being validated. @@ -1595,11 +1587,6 @@ file.spss-porExtraLabels.title=Upload an additional text file with extra variabl file.spss-porExtraLabels.selectToAddBtn=Select File to Add file.ingestFailed.header=Upload Completed with Errors file.ingestFailed.message=Tabular data ingest failed. -file.map=Map -file.mapData=Map Data -file.mapData.worldMap=WorldMap -file.mapData.unpublished.header=Data Not Published -file.mapData.unpublished.message=In order to map your data with WorldMap, your data must be published. Please publish this dataset, then retry the Map Data feature. file.downloadBtn.format.all=All File Formats + Information file.downloadBtn.format.tab=Tab-Delimited file.downloadBtn.format.original=Original File Format ({0}) diff --git a/src/main/java/propertyFiles/MimeTypeDisplay.properties b/src/main/java/propertyFiles/MimeTypeDisplay.properties index 29407ccda40..614ecb9b12a 100644 --- a/src/main/java/propertyFiles/MimeTypeDisplay.properties +++ b/src/main/java/propertyFiles/MimeTypeDisplay.properties @@ -5,6 +5,7 @@ application/pdf=Adobe PDF image/pdf=Adobe PDF text/pdf=Adobe PDF application/x-pdf=Adobe PDF +application/cnt=Windows Help Contents File application/msword=MS Word application/vnd.ms-excel=MS Excel Spreadsheet application/vnd.openxmlformats-officedocument.spreadsheetml.sheet=MS Excel Spreadsheet @@ -12,8 +13,10 @@ application/vnd.ms-powerpoint=MS Powerpoint application/vnd.openxmlformats-officedocument.presentationml.presentation=MS Powerpoint application/vnd.openxmlformats-officedocument.wordprocessingml.document=MS Word application/vnd.oasis.opendocument.spreadsheet=OpenOffice Spreadsheet +application/vnd.ms-excel.sheet.macroenabled.12=MS Excel Spreadsheet # Text text/plain=Plain Text +text/x-log=Application Log text/html=HTML application/x-tex=LaTeX text/x-tex=LaTeX @@ -21,6 +24,7 @@ text/markdown=Markdown Text text/x-markdown=Markdown Text text/x-r-markdown=R Markdown Text application/rtf=Rich Text Format +text/x-rst=reStructuredText text/rtf=Rich Text Format text/richtext=Rich Text Format text/turtle=Turtle RDF @@ -28,16 +32,26 @@ application/xml=XML text/xml=XML # Code text/x-c=C++ Source +text/x-c++src=C++ Source text/css=Cascading Style Sheet +text/x-fortran=Fortran Source Code +application/java-vm=Java Class +text/x-java-source=Java Source Code text/javascript=Javascript Code application/javascript=Javascript Code application/x-javascript=Javascript Code text/x-matlab=MATLAB Source Code text/x-mathematica=Mathematica Input +text/x-objcsrc=Objective-C Source Code +text/x-pascal=Pascal Source Code +text/x-perl=Perl Script +text/x-perl-script=Perl Script text/php=PHP Source Code +application/postscript=Postscript text/x-python=Python Source Code text/x-python-script=Python Source Code text/x-r-source=R Source Code +text/x-sh=Shell Script application/x-sh=Shell Script application/x-shellscript=Shell Script application/x-sql=SQL Code @@ -57,13 +71,18 @@ application/x-spss-sps=SPSS Script Syntax text/x-sas-syntax=SAS Syntax application/x-sas-syntax=SAS Syntax type/x-r-syntax=R Syntax +application/vnd.wolfram.mathematica.package=Wolfram Mathematica Code +application/vnd.wolfram.mathematica=Wolfram Mathematica Code # Ingested Tabular Data text/tab-separated-values=Tab-Delimited # RawData text/tsv=Tab-Separated Values text/comma-separated-values=Comma Separated Values +text/x-comma-separated-values=Comma Separated Values text/csv=Comma Separated Values text/x-fixed-field=Fixed Field Text Data +application/vnd.flographit=FloGraphIt Media +application/x-r-data=R Data application/x-rlang-transport=R Data application/x-R-2=R Binary application/x-stata=Stata Binary @@ -94,6 +113,7 @@ application/matlab-mat=MATLAB Data application/x-matlab-data=MATLAB Data application/x-matlab-figure=MATLAB Figure application/x-matlab-workspace=MATLAB Workspace +text/x-vcard=Virtual Contact File application/x-xfig=MATLAB Figure application/x-msaccess=MS Access application/netcdf=Network Common Data Form @@ -101,6 +121,10 @@ application/x-netcdf=Network Common Data Form application/vnd.lotus-notes=Notes Storage Facility application/x-nsdstat=NSDstat application/vnd.realvnc.bed=PLINK Binary +application/vnd.ms-pki.stl=STL Format +application/vnd.isac.fcs=FCS Data +application/java-serialized-object=Java Serialized Object +chemical/x-xyz=Co-Ordinate Animation # FITS image/fits=FITS application/fits=FITS @@ -112,7 +136,9 @@ application/sbn=ESRI Spatial Index application/sbx=ESRI Spatial Index application/shp=Shape application/shx=Shape -application/zipped-shapefile=Shape +application/x-esri-shape=ESRI Shapefile +application/vnd.google-earth.kml+xml=Keyhole Markup Language +application/zipped-shapefile=Zipped Shapefiles # Archive application/zip=ZIP Archive application/x-zip-compressed=ZIP Archive @@ -122,15 +148,18 @@ application/x-bzip2=Bzip Archive application/vnd.google-earth.kmz=Google Earth Archive application/gzip=Gzip Archive application/x-gzip=Gzip Archive +application/x-gzip-compressed=Gzip Archive application/rar=RAR Archive application/x-rar=RAR Archive application/x-rar-compressed=RAR Archive application/tar=TAR Archive application/x-tar=TAR Archive +application/x-compressed=Compressed Archive application/x-compressed-tar=TAR Archive application/x-7z-compressed=7Z Archive application/x-xz=XZ Archive application/warc=Web Archive +application/x-iso9660-image=Optical Disc Image # Image image/gif=GIF Image image/jpeg=JPEG Image @@ -151,8 +180,13 @@ image/tiff=TIFF Image image/bmp=Bitmap Image image/x-xbitmap=Bitmap Image image/RAW=Bitmap Image +image/raw=Bitmap Image +application/x-tgif=TGIF File image/x-xpixmap=Pixmap Image image/x-xwindowdump=X Windows Dump +application/photoshop=Photoshop Image +image/vnd.adobe.photoshop=Photoshop Image +application/x-photoshop=Photoshop Image # Audio audio/x-aiff=AIFF Audio audio/mp3=MP3 Audio @@ -177,4 +211,4 @@ text/xml-graphml=GraphML Network Data # Other application/octet-stream=Unknown # Dataverse-specific -application/vnd.dataverse.file-package=Dataverse Package \ No newline at end of file +application/vnd.dataverse.file-package=Dataverse Package diff --git a/src/main/java/propertyFiles/MimeTypeFacets.properties b/src/main/java/propertyFiles/MimeTypeFacets.properties index 54c5e01d317..2f91ca042b4 100644 --- a/src/main/java/propertyFiles/MimeTypeFacets.properties +++ b/src/main/java/propertyFiles/MimeTypeFacets.properties @@ -5,6 +5,7 @@ application/pdf=Document image/pdf=Document text/pdf=Document application/x-pdf=Document +application/cnt=Document application/msword=Document application/vnd.ms-excel=Document application/vnd.openxmlformats-officedocument.spreadsheetml.sheet=Document @@ -12,14 +13,17 @@ application/vnd.ms-powerpoint=Document application/vnd.openxmlformats-officedocument.presentationml.presentation=Document application/vnd.openxmlformats-officedocument.wordprocessingml.document=Document application/vnd.oasis.opendocument.spreadsheet=Document +application/vnd.ms-excel.sheet.macroenabled.12=Document # Text text/plain=Text +text/x-log=Text text/html=Text application/x-tex=Text text/x-tex=Text text/markdown=Text text/x-markdown=Text text/x-r-markdown=Text +text/x-rst=Text application/rtf=Text text/rtf=Text text/richtext=Text @@ -28,16 +32,24 @@ application/xml=Text text/xml=Text # Code text/x-c=Code +text/x-c++src=Code text/css=Code +text/x-objcsrc=Code +application/java-vm=Code +text/x-java-source=Code text/javascript=Code application/javascript=Code application/x-javascript=Code +text/x-perl-script=Code text/x-matlab=Code text/x-mathematica=Code text/php=Code +text/x-fortran=Code +text/x-pascal=Code text/x-python=Code text/x-python-script=Code text/x-r-source=Code +text/x-sh=Code application/x-sh=Code application/x-shellscript=Code application/x-sql=Code @@ -53,12 +65,17 @@ application/x-spss-syntax=Code text/x-sas-syntax=Code application/x-sas-syntax=Code type/x-r-syntax=Code +application/postscript=Code +application/vnd.wolfram.mathematica.package=Code +application/vnd.wolfram.mathematica=Code # Ingested text/tab-separated-values=Tabular Data # Data text/tsv=Data text/comma-separated-values=Data +text/x-comma-separated-values=Data text/csv=Data +text/x-vcard=Data text/x-fixed-field=Data application/x-rlang-transport=Data application/x-R-2=Data @@ -81,6 +98,7 @@ application/x-sas-data=Data application/x-sas-catalog=Data application/x-sas-log=Data application/x-sas-output=Data +application/x-r-data=Data application/softgrid-do=Data application/x-dvn-csvspss-zip=Data application/x-dvn-tabddi-zip=Data @@ -100,7 +118,12 @@ application/netcdf=Data application/x-netcdf=Data application/vnd.lotus-notes=Data application/x-nsdstat=Data +application/vnd.flographit=Data application/vnd.realvnc.bed=Data +application/vnd.ms-pki.stl=Data +application/vnd.isac.fcs=Data +application/java-serialized-object=Data +chemical/x-xyz=Data # FITS image/fits=FITS application/fits=FITS @@ -112,6 +135,8 @@ application/sbn=Shape application/sbx=Shape application/shp=Shape application/shx=Shape +application/x-esri-shape=Shape +application/vnd.google-earth.kml+xml=Shape application/zipped-shapefile=Shape # Archive application/zip=Archive @@ -122,15 +147,18 @@ application/x-bzip2=Archive application/vnd.google-earth.kmz=Archive application/gzip=Archive application/x-gzip=Archive +application/x-gzip-compressed=Archive application/rar=Archive application/x-rar=Archive application/x-rar-compressed=Archive application/tar=Archive application/x-tar=Archive +application/x-compressed=Archive application/x-compressed-tar=Archive application/x-7z-compressed=Archive application/x-xz=Archive application/warc=Archive +application/x-iso9660-image=Archive # Image image/gif=Image image/jpeg=Image @@ -151,8 +179,13 @@ image/tiff=Image image/bmp=Image image/x-xbitmap=Image image/RAW=Image +image/raw=Image +application/x-tgif=Image image/x-xpixmap=Image image/x-xwindowdump=Image +application/photoshop=Image +image/vnd.adobe.photoshop=Image +application/x-photoshop=Image # (anything else that looks like image/* will also be indexed as facet type "Image") # Audio audio/x-aiff=Audio diff --git a/src/main/resources/db/migration/V5.3.0.1__7409-remove-worldmap-geoconnect.sql b/src/main/resources/db/migration/V5.3.0.1__7409-remove-worldmap-geoconnect.sql new file mode 100644 index 00000000000..c42a9e3c081 --- /dev/null +++ b/src/main/resources/db/migration/V5.3.0.1__7409-remove-worldmap-geoconnect.sql @@ -0,0 +1,6 @@ +-- psql:src/main/resources/db/migration/V5.3.0.1__7409-remove-worldmap-geoconnect.sql:5: ERROR: cannot drop table worldmapauth_tokentype because other objects depend on it +-- DETAIL: constraint fk_worldmapauth_token_application_id on table worldmapauth_token depends on table worldmapauth_tokentype +-- HINT: Use DROP ... CASCADE to drop the dependent objects too. +drop table if exists worldmapauth_tokentype cascade; +drop table if exists worldmapauth_token; +drop table if exists maplayermetadata; diff --git a/src/main/resources/db/migration/afterMigrate__1-7256-upsert-referenceData.sql b/src/main/resources/db/migration/afterMigrate__1-7256-upsert-referenceData.sql index d45c03bc03b..07e9b2c6266 100644 --- a/src/main/resources/db/migration/afterMigrate__1-7256-upsert-referenceData.sql +++ b/src/main/resources/db/migration/afterMigrate__1-7256-upsert-referenceData.sql @@ -39,13 +39,3 @@ INSERT INTO foreignmetadatafieldmapping (id, foreignfieldxpath, datasetfieldname INSERT INTO guestbook (emailrequired, enabled, institutionrequired, createtime, name, namerequired, positionrequired, dataverse_id) SELECT false, true, false, now(), 'Default', false, false, null WHERE NOT EXISTS (SELECT id FROM guestbook); - --- Simple trick: WHERE NOT EXISTS (SELECT id FROM table) is only true if the table is empty. -INSERT INTO worldmapauth_tokentype - (name, created, contactemail, hostname, - ipaddress, mapitlink, - md5, modified, timelimitminutes) - SELECT 'GEOCONNECT', current_timestamp, 'support@dataverse.org', 'geoconnect.datascience.iq.harvard.edu', - '140.247.115.127', 'http://geoconnect.datascience.iq.harvard.edu/shapefile/map-it', - '38c0a931b2d582a5c43fc79405b30c22', current_timestamp, 30 - WHERE NOT EXISTS (SELECT id from worldmapauth_tokentype); diff --git a/src/main/webapp/dataverseuser.xhtml b/src/main/webapp/dataverseuser.xhtml index 84d0e9888a0..893b1e7e643 100644 --- a/src/main/webapp/dataverseuser.xhtml +++ b/src/main/webapp/dataverseuser.xhtml @@ -190,23 +190,6 @@ - - - - - #{item.theObject.getDataset().getDisplayName()} - - - - - - - - - #{item.theObject.getDatasetVersion().getDataset().getDisplayName()} - - - diff --git a/src/main/webapp/file-download-button-fragment.xhtml b/src/main/webapp/file-download-button-fragment.xhtml index da88d1ae64d..85fe60863b4 100644 --- a/src/main/webapp/file-download-button-fragment.xhtml +++ b/src/main/webapp/file-download-button-fragment.xhtml @@ -220,9 +220,8 @@ - + -
  • #{bundle['dataset.accessBtn.header.explore']}
    • @@ -246,26 +245,6 @@ - -
  • - - #{bundle['file.mapData.worldMap']} - - - #{bundle['file.mapData.worldMap']} - -
    • -     diff --git a/src/main/webapp/file-download-popup-fragment.xhtml b/src/main/webapp/file-download-popup-fragment.xhtml index 1283c3a6e3c..b9a8c6e0425 100644 --- a/src/main/webapp/file-download-popup-fragment.xhtml +++ b/src/main/webapp/file-download-popup-fragment.xhtml @@ -164,7 +164,6 @@ - - - -
    - - - - - - - - - - - - - - - - - - -
    diff --git a/src/test/java/edu/harvard/iq/dataverse/api/GeoconnectIT.java b/src/test/java/edu/harvard/iq/dataverse/api/GeoconnectIT.java deleted file mode 100644 index 2b18e3283d7..00000000000 --- a/src/test/java/edu/harvard/iq/dataverse/api/GeoconnectIT.java +++ /dev/null @@ -1,53 +0,0 @@ -package edu.harvard.iq.dataverse.api; - -import com.jayway.restassured.RestAssured; -import com.jayway.restassured.response.Response; -import static javax.ws.rs.core.Response.Status.OK; -import org.junit.BeforeClass; -import org.junit.Test; - -public class GeoconnectIT { - - @BeforeClass - public static void setUp() { - RestAssured.baseURI = UtilIT.getRestAssuredBaseUri(); - } - -// @Ignore - @Test - public void checkMapLayerMetadatas() { - Response createUser = UtilIT.createRandomUser(); - String username = UtilIT.getUsernameFromResponse(createUser); - String apiToken = UtilIT.getApiTokenFromResponse(createUser); - - Response checkMapLayerMetadatas = UtilIT.checkMapLayerMetadatas(apiToken); - checkMapLayerMetadatas.prettyPrint(); - checkMapLayerMetadatas.then().assertThat() - .statusCode(OK.getStatusCode()); - } - -// @Ignore - @Test - public void checkSingleMapLayerMetadata() { - Response createUser = UtilIT.createRandomUser(); - String username = UtilIT.getUsernameFromResponse(createUser); - String apiToken = UtilIT.getApiTokenFromResponse(createUser); - - Response checkMapLayerMetadatas = UtilIT.checkMapLayerMetadatas(1l, apiToken); - checkMapLayerMetadatas.prettyPrint(); - checkMapLayerMetadatas.then().assertThat() - .statusCode(OK.getStatusCode()); - } - -// @Ignore - @Test - public void deleteMapFromFile() { - String apiToken = "b5a144d8-b3a3-452b-9543-131360884281"; - long fileId = 2661; - Response deleteMapFromFile = UtilIT.deleteMapFromFile(fileId, apiToken); - deleteMapFromFile.prettyPrint(); - deleteMapFromFile.then().assertThat() - .statusCode(OK.getStatusCode()); - } - -} diff --git a/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java b/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java index 97afd7e060e..27b34aba5e1 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java @@ -1845,40 +1845,6 @@ private static byte[] inputStreamToBytes(InputStream inputStream) { } } - static Response listMapLayerMetadatas() { - return given().get("/api/admin/geoconnect/mapLayerMetadatas"); - } - - static Response checkMapLayerMetadatas(String apiToken) { - return given() - .header(API_TOKEN_HTTP_HEADER, apiToken) - .post("/api/admin/geoconnect/mapLayerMetadatas/check"); - } - - static Response checkMapLayerMetadatas(Long mapLayerMetadataId, String apiToken) { - return given() - .header(API_TOKEN_HTTP_HEADER, apiToken) - .post("/api/admin/geoconnect/mapLayerMetadatas/check/" + mapLayerMetadataId); - } - - static Response getMapFromFile(long fileId, String apiToken) { - return given() - .header(API_TOKEN_HTTP_HEADER, apiToken) - .get("/api/files/" + fileId + "/map"); - } - - static Response checkMapFromFile(long fileId, String apiToken) { - return given() - .header(API_TOKEN_HTTP_HEADER, apiToken) - .get("/api/files/" + fileId + "/map/check"); - } - - static Response deleteMapFromFile(long fileId, String apiToken) { - return given() - .header(API_TOKEN_HTTP_HEADER, apiToken) - .delete("/api/files/" + fileId + "/map?key=" + apiToken); - } - static Response getRsyncScript(String datasetPersistentId, String apiToken) { RequestSpecification requestSpecification = given(); if (apiToken != null) { diff --git a/src/test/java/edu/harvard/iq/dataverse/engine/TestCommandContext.java b/src/test/java/edu/harvard/iq/dataverse/engine/TestCommandContext.java index 329f8509ec6..f0ddcf4c81c 100644 --- a/src/test/java/edu/harvard/iq/dataverse/engine/TestCommandContext.java +++ b/src/test/java/edu/harvard/iq/dataverse/engine/TestCommandContext.java @@ -220,11 +220,6 @@ public WorkflowServiceBean workflows() { return null; } - @Override - public MapLayerMetadataServiceBean mapLayerMetadata() { - return null; - } - @Override public DataCaptureModuleServiceBean dataCaptureModule() { return null; diff --git a/src/test/java/edu/harvard/iq/dataverse/util/MailUtilTest.java b/src/test/java/edu/harvard/iq/dataverse/util/MailUtilTest.java index 03a9f1f3a20..af0c414d356 100644 --- a/src/test/java/edu/harvard/iq/dataverse/util/MailUtilTest.java +++ b/src/test/java/edu/harvard/iq/dataverse/util/MailUtilTest.java @@ -71,19 +71,7 @@ public void testSubjectRejectFileAccess() { userNotification.setType(UserNotification.Type.REJECTFILEACCESS); assertEquals("LibraScholar: Your request for access to a restricted file has been rejected", MailUtil.getSubjectTextBasedOnNotification(userNotification, rootDataverseName, null)); } - - @Test - public void testSubjectMapLayerUpdated() { - userNotification.setType(UserNotification.Type.MAPLAYERUPDATED); - assertEquals("LibraScholar: WorldMap layer added to dataset", MailUtil.getSubjectTextBasedOnNotification(userNotification, rootDataverseName, null)); - } - - @Test - public void testSubjectMapLayerDeleteFailed() { - userNotification.setType(UserNotification.Type.MAPLAYERDELETEFAILED); - assertEquals("LibraScholar: Failed to delete WorldMap layer", MailUtil.getSubjectTextBasedOnNotification(userNotification, rootDataverseName, null)); - } - + @Test public void testSubjectCreateDataset() { userNotification.setType(UserNotification.Type.CREATEDS); diff --git a/src/test/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenServiceBeanTest.java b/src/test/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenServiceBeanTest.java deleted file mode 100644 index e2d392249d7..00000000000 --- a/src/test/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenServiceBeanTest.java +++ /dev/null @@ -1,76 +0,0 @@ -package edu.harvard.iq.dataverse.worldmapauth; - -import static org.junit.Assert.assertFalse; -import static org.junit.Assert.assertTrue; -import static org.mockito.Mockito.mock; - -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.mockito.Mockito; - -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.PermissionServiceBean; -import edu.harvard.iq.dataverse.PermissionServiceBean.StaticPermissionQuery; -import edu.harvard.iq.dataverse.authorization.Permission; -import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; - -public class WorldMapTokenServiceBeanTest { - - private WorldMapTokenServiceBean worldMapTokenServiceBean; - - @Before - public void setUp() { - this.worldMapTokenServiceBean = new WorldMapTokenServiceBean(); - this.worldMapTokenServiceBean.permissionService = mock(PermissionServiceBean.class); - } - - @After - public void tearDown() { - this.worldMapTokenServiceBean = null; - } - - private WorldMapToken createWorldMapToken(AuthenticatedUser user, DataFile file) { - WorldMapToken worldMapToken = new WorldMapToken(); - worldMapToken.setDataverseUser(user); - worldMapToken.setDatafile(file); - - return worldMapToken; - } - - private void mockPermissionServiceCallForEditDataset(AuthenticatedUser user, DataFile file, - boolean isUserPermitted) { - StaticPermissionQuery query = mock(StaticPermissionQuery.class); - Mockito.when(this.worldMapTokenServiceBean.permissionService.userOn(user, file)).thenReturn(query); - Mockito.when(query.has(Permission.EditDataset)).thenReturn(isUserPermitted); - } - - @Test - public void testCanTokenUserEditFileWithUndefinedWorldMapToken() { - WorldMapToken worldMapToken = null; - - assertFalse(this.worldMapTokenServiceBean.canTokenUserEditFile(worldMapToken)); - } - - @Test - public void testCanTokenUserEditFileWithPermission() { - AuthenticatedUser user = new AuthenticatedUser(); - DataFile file = new DataFile(); - WorldMapToken worldMapToken = this.createWorldMapToken(user, file); - - this.mockPermissionServiceCallForEditDataset(user, file, true); - - assertTrue(this.worldMapTokenServiceBean.canTokenUserEditFile(worldMapToken)); - } - - @Test - public void testCanTokenUserEditFileWithoutPermission() { - AuthenticatedUser user = new AuthenticatedUser(); - DataFile file = new DataFile(); - WorldMapToken worldMapToken = this.createWorldMapToken(user, file); - - this.mockPermissionServiceCallForEditDataset(user, file, false); - - assertFalse(this.worldMapTokenServiceBean.canTokenUserEditFile(worldMapToken)); - } -} \ No newline at end of file diff --git a/src/test/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenTest.java b/src/test/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenTest.java deleted file mode 100644 index 94380d95651..00000000000 --- a/src/test/java/edu/harvard/iq/dataverse/worldmapauth/WorldMapTokenTest.java +++ /dev/null @@ -1,152 +0,0 @@ -/* - * To change this license header, choose License Headers in Project Properties. - * To change this template file, choose Tools | Templates - * and open the template in the editor. - */ - -package edu.harvard.iq.dataverse.worldmapauth; - -import edu.harvard.iq.dataverse.DataFile; -import edu.harvard.iq.dataverse.NonEssentialTests; -import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; -import java.sql.Timestamp; -import java.util.Date; -import javax.ejb.embeddable.EJBContainer; -import static org.junit.Assert.assertEquals; -import org.junit.Test; -import org.junit.experimental.categories.Category; - -/** - * - * @author raprasad - */ -public class WorldMapTokenTest { - - private static EJBContainer c; - - - public void msg(String s){ - System.out.println(s); - } - - public void msgt(String s){ - msg("------------------------------------------------------------"); - msg(s); - msg("------------------------------------------------------------"); - } - - - private TokenApplicationType makeTokenApplicationType(int timeLimitMinutes){ - TokenApplicationType tat = new TokenApplicationType(); - tat.setName("GeoConnect"); - tat.setContactEmail("info@iq.harvard.edu"); - tat.setHostname("geoconnect.datascience.iq.harvard.edu"); - tat.setIpAddress("127.0.0.1"); - tat.setTimeLimitMinutes(timeLimitMinutes); - return tat; - } - private WorldMapToken makeNewToken(TokenApplicationType tat){ - WorldMapToken token; - token = new WorldMapToken(); - token.setApplication(tat); - token.setDatafile(new DataFile()); - token.setDataverseUser(new AuthenticatedUser()); - token.refreshToken(); - token.setToken(); - return token; - } - - @Category(NonEssentialTests.class) - @Test - public void testTokenValues(){ - msgt("WorldMapTokenTest!"); - TokenApplicationType tat = this.makeTokenApplicationType(30); - - WorldMapToken token = this.makeNewToken(tat); - String token_str1 = token.getToken(); - - // Should only be able to set token value once--it doesn't "reset" - token.setToken(); - assertEquals(token.getToken().equalsIgnoreCase(token_str1), true); - - - WorldMapToken token2 = this.makeNewToken(tat); - WorldMapToken token3 = this.makeNewToken(tat); - assertEquals(token2.getToken().equalsIgnoreCase(token.getToken()), false); - assertEquals(token2.getToken().equalsIgnoreCase(token3.getToken()), false); - } - - @Category(NonEssentialTests.class) - @Test - public void testTokenTimes(){ - msgt("testTokenTimes"); - - TokenApplicationType tat = this.makeTokenApplicationType(30); - - assertEquals(30*60, tat.getTimeLimitSeconds()); - msg("time limit seconds: " + tat.getTimeLimitSeconds()); - tat.setTimeLimitMinutes(1); - msg("time limit seconds (2): " + tat.getTimeLimitSeconds()); - assertEquals(1*60, tat.getTimeLimitSeconds()); - - tat.setTimeLimitMinutes(30); - WorldMapToken token = this.makeNewToken(tat); - assertEquals(token.hasTokenExpired(), false); - - //msg("Future token time (31 min): " + getFutureTimeStamp(31)); - msg("token time limit (minutes): " + token.getApplication().getTimeLimitMinutes()); - msg("Did token expire in 10 minutes? (should be no)"); - //msg("expired? " + token.hasTokenExpired(getFutureTimeStamp(10))); - - assertEquals(token.hasTokenExpired(getFutureTimeStamp(10)), false); - - msg("Did token expire? (automatically check current time)"); - assertEquals(token.hasTokenExpired(), false); - - msg("Did token expire at 30 minutes? (should be no)"); - assertEquals(token.hasTokenExpired(getFutureTimeStamp(30)), false); - - msg("Did token expire in 31 minutes? (should be yes)"); - assertEquals(token.hasTokenExpired(getFutureTimeStamp(31)), true); - - msg("Did token expire in 45 minutes? (should be yes)"); - assertEquals(token.hasTokenExpired(getFutureTimeStamp(45)), true); - - msg("token time limit (minutes): 10 minutes"); - token.getApplication().setTimeLimitMinutes(10); - assertEquals(token.getApplication().getTimeLimitMinutes(), 10); - - msg("Did token expire if null sent? (should be yes)"); - assertEquals(token.hasTokenExpired(null), true); - - msg("Refresh token (but fails b/c sending null automatically expires it"); - token.refreshToken(); - msg("Did token expire in 5 minutes? (should be no)"); - assertEquals(token.hasTokenExpired(getFutureTimeStamp(5)), true); - msg("Did token expire? (auto-check current time)"); - assertEquals(token.hasTokenExpired(), true); - - msgt("Get a new Token"); - WorldMapToken token2 = this.makeNewToken(tat); - - - msg("Did token expire? (automatically check current time)"); - assertEquals(token2.hasTokenExpired(), false); - - msg("Manually expire token: setHasExpired(true)"); - token2.setHasExpired(true); - msg("Did token expire in 1 minute? (should be yes--b/c manually expired)"); - assertEquals(token2.hasTokenExpired(getFutureTimeStamp(1)), true); - - - } - - private Timestamp getFutureTimeStamp(int futuremMinutes){ - - long ONE_MINUTE_IN_MILLIS=60000;//millisecs - long currentTimeMillisec = new Date().getTime(); - long inFutureMinutesMillisec = currentTimeMillisec + (futuremMinutes * ONE_MINUTE_IN_MILLIS); - return new Timestamp(new Date(inFutureMinutesMillisec).getTime()); - - } -}