-
Notifications
You must be signed in to change notification settings - Fork 501
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support deletion of saved searches #10198
Changes from 8 commits
2dc84db
ac80e0a
a50f963
98f7b53
4fabfec
0dc2ea9
b6f4b7f
614da60
6836366
633cd6c
92e61fc
4061c08
5f2de4a
008507b
8acfdc9
4022c9f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
### Saved search deletion | ||
|
||
Saved search can now be removed using API `/api/admin/savedsearches/$id`. See PR #10198. | ||
This is reflected in the [Saved Search Native API section](https://guides.dataverse.org/en/latest/api/native-api.html#saved-search) of the Guide. | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5745,8 +5745,7 @@ The ``$identifier`` should start with an ``@`` if it's a user. Groups start with | |
Saved Search | ||
~~~~~~~~~~~~ | ||
|
||
The Saved Search, Linked Dataverses, and Linked Datasets features shipped with Dataverse 4.0, but as a "`superuser-only <https://github.com/IQSS/dataverse/issues/90#issuecomment-86094663>`_" because they are **experimental** (see `#1364 <https://github.com/IQSS/dataverse/issues/1364>`_, `#1813 <https://github.com/IQSS/dataverse/issues/1813>`_, `#1840 <https://github.com/IQSS/dataverse/issues/1840>`_, `#1890 <https://github.com/IQSS/dataverse/issues/1890>`_, `#1939 <https://github.com/IQSS/dataverse/issues/1939>`_, `#2167 <https://github.com/IQSS/dataverse/issues/2167>`_, `#2186 <https://github.com/IQSS/dataverse/issues/2186>`_, `#2053 <https://github.com/IQSS/dataverse/issues/2053>`_, and `#2543 <https://github.com/IQSS/dataverse/issues/2543>`_). The following API endpoints were added to help people with access to the "admin" API make use of these features in their current form. There is a known issue (`#1364 <https://github.com/IQSS/dataverse/issues/1364>`_) that once a link to a Dataverse collection or dataset is created, it cannot be removed (apart from database manipulation and reindexing) which is why a ``DELETE`` endpoint for saved searches is neither documented nor functional. The Linked Dataverse collections feature is `powered by Saved Search <https://github.com/IQSS/dataverse/issues/1852>`_ and therefore requires that the "makelinks" endpoint be executed on a periodic basis as well. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There's a conversation going on in #10628 about how these docs used to explain to people that they might have to wait up to 24 hours for their saved search to show results. Should we add that back as part of this pull request? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated with The update of all saved search is run by a timer once a week (See :ref: |
||
|
||
The Saved Search, Linked Dataverses, and Linked Datasets features are only accessible to superusers except for Linking a dataset. The following API endpoints were added to help people with access to the “admin” API make use of these features in their current form, keep in mind that they are partially experimental. | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
List all saved searches. :: | ||
|
||
GET http://$SERVER/api/admin/savedsearches/list | ||
|
@@ -5755,6 +5754,10 @@ List a saved search by database id. :: | |
|
||
GET http://$SERVER/api/admin/savedsearches/$id | ||
|
||
Delete a saved search by database id. The ``unlink=true`` query parameter unlink links (Linked dataset or Dataverse collection) related to the deleted saved search. This parameter should be well considered as you cannot know if the saved search created the links or if someone else did via other API. Also, it may be followed ``/makelinks/all`` depending on the need if other saved searches could recreate some deleted links or by reindexing some Dataverse or Dataset. :: | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
DELETE http://$SERVER/api/admin/savedsearches/$id?unlink=true | ||
|
||
Execute a saved search by database id and make links to Dataverse collections and datasets that are found. The JSON response indicates which Dataverse collections and datasets were newly linked versus already linked. The ``debug=true`` query parameter adds to the JSON response extra information about the saved search being executed (which you could also get by listing the saved search). :: | ||
|
||
PUT http://$SERVER/api/admin/savedsearches/makelinks/$id?debug=true | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,7 @@ | ||
package edu.harvard.iq.dataverse.api; | ||
|
||
import edu.harvard.iq.dataverse.Dataverse; | ||
import edu.harvard.iq.dataverse.api.auth.AuthRequired; | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; | ||
import edu.harvard.iq.dataverse.engine.command.DataverseRequest; | ||
import edu.harvard.iq.dataverse.engine.command.exception.CommandException; | ||
|
@@ -24,6 +25,8 @@ | |
import jakarta.ws.rs.Path; | ||
import jakarta.ws.rs.PathParam; | ||
import jakarta.ws.rs.QueryParam; | ||
import jakarta.ws.rs.container.ContainerRequestContext; | ||
import jakarta.ws.rs.core.Context; | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
import jakarta.ws.rs.core.Response; | ||
import static jakarta.ws.rs.core.Response.Status.BAD_REQUEST; | ||
import static jakarta.ws.rs.core.Response.Status.INTERNAL_SERVER_ERROR; | ||
|
@@ -46,22 +49,46 @@ public Response meta() { | |
} | ||
|
||
@GET | ||
@AuthRequired | ||
@Path("list") | ||
public Response list() { | ||
public Response list(@Context ContainerRequestContext crc) { | ||
|
||
AuthenticatedUser superuser = null; | ||
try { | ||
superuser = getRequestAuthenticatedUserOrDie(crc); | ||
} catch (WrappedResponse wr) { | ||
return wr.getResponse(); | ||
} | ||
if (superuser == null || !superuser.isSuperuser()) { | ||
return error(Response.Status.UNAUTHORIZED, "Superusers only."); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need this backward-incompatible change? I'm only going to leave this comment once but it applies to all endpoints that formerly were not superuser-only. @luddaniel I know we talked about this a bit at https://dataverse.zulipchat.com/#narrow/stream/379673-dev/topic/admin.20API.20and.20superuser.20authorization/near/443079852 but my understanding was that we were talking theoretically about the future. I didn't realize you were thinking about making backward-incompatible changes now, in the name of consistency. We should probably pause and get rough consensus from more of the core team and decide on the timing for introducing these changes, if at all. |
||
} | ||
|
||
JsonArrayBuilder savedSearchesBuilder = Json.createArrayBuilder(); | ||
List<SavedSearch> savedSearches = savedSearchSvc.findAll(); | ||
for (SavedSearch savedSearch : savedSearches) { | ||
JsonObjectBuilder thisSavedSearch = toJson(savedSearch); | ||
savedSearchesBuilder.add(thisSavedSearch); | ||
} | ||
JsonObjectBuilder response = Json.createObjectBuilder(); | ||
response.add("saved searches", savedSearchesBuilder); | ||
response.add("savedSearches", savedSearchesBuilder); | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
return ok(response); | ||
} | ||
|
||
@GET | ||
@AuthRequired | ||
@Path("{id}") | ||
public Response show(@PathParam("id") long id) { | ||
public Response show(@Context ContainerRequestContext crc, @PathParam("id") long id) { | ||
|
||
AuthenticatedUser superuser = null; | ||
try { | ||
superuser = getRequestAuthenticatedUserOrDie(crc); | ||
} catch (WrappedResponse wr) { | ||
return wr.getResponse(); | ||
} | ||
if (superuser == null || !superuser.isSuperuser()) { | ||
return error(Response.Status.UNAUTHORIZED, "Superusers only."); | ||
} | ||
|
||
SavedSearch savedSearch = savedSearchSvc.find(id); | ||
if (savedSearch != null) { | ||
JsonObjectBuilder response = toJson(savedSearch); | ||
|
@@ -89,7 +116,18 @@ private JsonObjectBuilder toJson(SavedSearch savedSearch) { | |
} | ||
|
||
@POST | ||
public Response add(JsonObject body) { | ||
@AuthRequired | ||
public Response add(@Context ContainerRequestContext crc, JsonObject body) { | ||
|
||
AuthenticatedUser superuser = null; | ||
try { | ||
superuser = getRequestAuthenticatedUserOrDie(crc); | ||
} catch (WrappedResponse wr) { | ||
return wr.getResponse(); | ||
} | ||
if (superuser == null || !superuser.isSuperuser()) { | ||
return error(Response.Status.UNAUTHORIZED, "Superusers only."); | ||
} | ||
|
||
if (body == null) { | ||
return error(BAD_REQUEST, "JSON is expected."); | ||
|
@@ -159,7 +197,7 @@ public Response add(JsonObject body) { | |
|
||
try { | ||
SavedSearch persistedSavedSearch = savedSearchSvc.add(toPersist); | ||
return ok("Added: " + persistedSavedSearch); | ||
return ok("Added: " + persistedSavedSearch, Json.createObjectBuilder().add("id", persistedSavedSearch.getId())); | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} catch (EJBException ex) { | ||
StringBuilder errors = new StringBuilder(); | ||
Throwable throwable = ex.getCause(); | ||
|
@@ -172,17 +210,31 @@ public Response add(JsonObject body) { | |
} | ||
|
||
@DELETE | ||
@AuthRequired | ||
@Path("{id}") | ||
public Response delete(@PathParam("id") long doomedId) { | ||
boolean disabled = true; | ||
if (disabled) { | ||
return error(BAD_REQUEST, "Saved Searches can not safely be deleted because links can not safely be deleted. See https://github.com/IQSS/dataverse/issues/1364 for details."); | ||
public Response delete(@Context ContainerRequestContext crc, @PathParam("id") long doomedId, @QueryParam("unlink") boolean unlink) { | ||
|
||
AuthenticatedUser superuser = null; | ||
try { | ||
superuser = getRequestAuthenticatedUserOrDie(crc); | ||
} catch (WrappedResponse wr) { | ||
return wr.getResponse(); | ||
} | ||
if (superuser == null || !superuser.isSuperuser()) { | ||
return error(Response.Status.UNAUTHORIZED, "Superusers only."); | ||
} | ||
|
||
SavedSearch doomed = savedSearchSvc.find(doomedId); | ||
if (doomed == null) { | ||
return error(NOT_FOUND, "Could not find saved search id " + doomedId); | ||
} | ||
boolean wasDeleted = savedSearchSvc.delete(doomedId); | ||
boolean wasDeleted; | ||
try { | ||
wasDeleted = savedSearchSvc.delete(doomedId, unlink); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want to put this in a command? There's a "CreateSavedSearchCommand" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hey @sekmiller, if my understanding is correct I note that It could be prematured to create a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One reason that I suggested the command was because commands also check for permissions. I didn't notice that this is a superuser only feature - so that permission check wouldn't be available in the command. |
||
} catch (Exception e) { | ||
return error(INTERNAL_SERVER_ERROR, "Problem while trying to unlink links of saved search id " + doomedId); | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
if (wasDeleted) { | ||
return ok(Json.createObjectBuilder().add("Deleted", doomedId)); | ||
} else { | ||
|
@@ -191,8 +243,20 @@ public Response delete(@PathParam("id") long doomedId) { | |
} | ||
|
||
@PUT | ||
@AuthRequired | ||
@Path("makelinks/all") | ||
public Response makeLinksForAllSavedSearches(@QueryParam("debug") boolean debug) { | ||
public Response makeLinksForAllSavedSearches(@Context ContainerRequestContext crc, @QueryParam("debug") boolean debug) { | ||
|
||
AuthenticatedUser superuser = null; | ||
try { | ||
superuser = getRequestAuthenticatedUserOrDie(crc); | ||
} catch (WrappedResponse wr) { | ||
return wr.getResponse(); | ||
} | ||
if (superuser == null || !superuser.isSuperuser()) { | ||
return error(Response.Status.UNAUTHORIZED, "Superusers only."); | ||
} | ||
|
||
JsonObjectBuilder makeLinksResponse; | ||
try { | ||
makeLinksResponse = savedSearchSvc.makeLinksForAllSavedSearches(debug); | ||
|
@@ -205,8 +269,20 @@ public Response makeLinksForAllSavedSearches(@QueryParam("debug") boolean debug) | |
} | ||
|
||
@PUT | ||
@AuthRequired | ||
@Path("makelinks/{id}") | ||
public Response makeLinksForSingleSavedSearch(@PathParam("id") long savedSearchIdToLookUp, @QueryParam("debug") boolean debug) { | ||
public Response makeLinksForSingleSavedSearch(@Context ContainerRequestContext crc, @PathParam("id") long savedSearchIdToLookUp, @QueryParam("debug") boolean debug) { | ||
|
||
AuthenticatedUser superuser = null; | ||
try { | ||
superuser = getRequestAuthenticatedUserOrDie(crc); | ||
} catch (WrappedResponse wr) { | ||
return wr.getResponse(); | ||
} | ||
if (superuser == null || !superuser.isSuperuser()) { | ||
return error(Response.Status.UNAUTHORIZED, "Superusers only."); | ||
} | ||
|
||
SavedSearch savedSearchToMakeLinksFor = savedSearchSvc.find(savedSearchIdToLookUp); | ||
if (savedSearchToMakeLinksFor == null) { | ||
return error(BAD_REQUEST, "Count not find saved search id " + savedSearchIdToLookUp); | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,29 +2,28 @@ | |
|
||
import edu.harvard.iq.dataverse.Dataset; | ||
import edu.harvard.iq.dataverse.DatasetLinkingDataverse; | ||
import edu.harvard.iq.dataverse.DatasetLinkingServiceBean; | ||
import edu.harvard.iq.dataverse.Dataverse; | ||
import edu.harvard.iq.dataverse.DataverseLinkingDataverse; | ||
import edu.harvard.iq.dataverse.DataverseLinkingServiceBean; | ||
import edu.harvard.iq.dataverse.DvObject; | ||
import edu.harvard.iq.dataverse.DvObjectServiceBean; | ||
import edu.harvard.iq.dataverse.EjbDataverseEngine; | ||
import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser; | ||
import edu.harvard.iq.dataverse.authorization.users.GuestUser; | ||
import edu.harvard.iq.dataverse.engine.command.DataverseRequest; | ||
import edu.harvard.iq.dataverse.search.SearchServiceBean; | ||
import edu.harvard.iq.dataverse.search.SolrQueryResponse; | ||
import edu.harvard.iq.dataverse.search.SolrSearchResult; | ||
import edu.harvard.iq.dataverse.engine.command.exception.CommandException; | ||
import edu.harvard.iq.dataverse.engine.command.impl.DeleteDatasetLinkingDataverseCommand; | ||
import edu.harvard.iq.dataverse.engine.command.impl.DeleteDataverseLinkingDataverseCommand; | ||
import edu.harvard.iq.dataverse.engine.command.impl.LinkDatasetCommand; | ||
import edu.harvard.iq.dataverse.engine.command.impl.LinkDataverseCommand; | ||
import edu.harvard.iq.dataverse.search.SearchException; | ||
import edu.harvard.iq.dataverse.search.SearchFields; | ||
import edu.harvard.iq.dataverse.search.SearchServiceBean; | ||
import edu.harvard.iq.dataverse.search.SolrQueryResponse; | ||
import edu.harvard.iq.dataverse.search.SolrSearchResult; | ||
import edu.harvard.iq.dataverse.search.SortBy; | ||
import edu.harvard.iq.dataverse.util.SystemConfig; | ||
import java.util.ArrayList; | ||
import java.util.Date; | ||
import java.util.List; | ||
import java.util.logging.Level; | ||
import java.util.logging.Logger; | ||
import jakarta.ejb.EJB; | ||
import jakarta.ejb.Schedule; | ||
import jakarta.ejb.Stateless; | ||
|
@@ -39,6 +38,12 @@ | |
import jakarta.persistence.TypedQuery; | ||
import jakarta.servlet.http.HttpServletRequest; | ||
|
||
import java.util.ArrayList; | ||
import java.util.Date; | ||
import java.util.List; | ||
import java.util.logging.Level; | ||
import java.util.logging.Logger; | ||
|
||
@Stateless | ||
@Named | ||
public class SavedSearchServiceBean { | ||
|
@@ -50,6 +55,10 @@ public class SavedSearchServiceBean { | |
@EJB | ||
DvObjectServiceBean dvObjectService; | ||
@EJB | ||
protected DatasetLinkingServiceBean dsLinkingService; | ||
@EJB | ||
protected DataverseLinkingServiceBean dvLinkingService; | ||
@EJB | ||
EjbDataverseEngine commandEngine; | ||
@EJB | ||
SystemConfig systemConfig; | ||
|
@@ -101,11 +110,15 @@ public SavedSearch add(SavedSearch toPersist) { | |
return persisted; | ||
} | ||
|
||
public boolean delete(long id) { | ||
public boolean delete(long id, boolean unlink) throws SearchException, CommandException { | ||
SavedSearch doomed = find(id); | ||
boolean wasDeleted = false; | ||
if (doomed != null) { | ||
System.out.println("deleting saved search id " + doomed.getId()); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. While we're in this part of the code, should we delete this println? Or maybe convert it to our regular logger and set the level to "fine"? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done! |
||
if(unlink) { | ||
DataverseRequest dataverseRequest = new DataverseRequest(doomed.getCreator(), getHttpServletRequest()); | ||
unLinksForSingleSavedSearch(dataverseRequest, doomed); | ||
} | ||
em.remove(doomed); | ||
em.flush(); | ||
wasDeleted = true; | ||
|
@@ -240,6 +253,37 @@ public JsonObjectBuilder makeLinksForSingleSavedSearch(DataverseRequest dvReq, S | |
return response; | ||
} | ||
|
||
public void unLinksForSingleSavedSearch(DataverseRequest dvReq, SavedSearch savedSearch) throws SearchException, CommandException { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't love this method name. I'm not even sure we need "single" in there. I probably added it back in the day for a reason but now it seems verbose. 🤷 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was too focused on mimicry the |
||
logger.info("UNLINK SAVED SEARCH (" + savedSearch.getId() + ") START search and unlink process"); | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Date start = new Date(); | ||
Dataverse linkingDataverse = savedSearch.getDefinitionPoint(); | ||
|
||
SolrQueryResponse queryResponse = findHits(savedSearch); | ||
for (SolrSearchResult solrSearchResult : queryResponse.getSolrSearchResults()) { | ||
|
||
DvObject dvObjectThatDefinitionPointWillLinkTo = dvObjectService.findDvObject(solrSearchResult.getEntityId()); | ||
if (dvObjectThatDefinitionPointWillLinkTo == null) { | ||
continue; | ||
} | ||
|
||
if (dvObjectThatDefinitionPointWillLinkTo.isInstanceofDataverse()) { | ||
Dataverse linkedDataverse = (Dataverse) dvObjectThatDefinitionPointWillLinkTo; | ||
DataverseLinkingDataverse dvld = dvLinkingService.findDataverseLinkingDataverse(linkedDataverse.getId(), linkingDataverse.getId()); | ||
if(dvld != null) { | ||
Dataverse dv = commandEngine.submitInNewTransaction(new DeleteDataverseLinkingDataverseCommand(dvReq, linkingDataverse, dvld, true)); | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
} else if (dvObjectThatDefinitionPointWillLinkTo.isInstanceofDataset()) { | ||
Dataset linkedDataset = (Dataset) dvObjectThatDefinitionPointWillLinkTo; | ||
DatasetLinkingDataverse dsld = dsLinkingService.findDatasetLinkingDataverse(linkedDataset.getId(), linkingDataverse.getId()); | ||
if(dsld != null) { | ||
Dataset ds = commandEngine.submitInNewTransaction(new DeleteDatasetLinkingDataverseCommand(dvReq, linkedDataset, dsld, true)); | ||
} | ||
} | ||
} | ||
|
||
logger.info("UNLINK SAVED SEARCH (" + savedSearch.getId() + ") total time in ms: " + (new Date().getTime() - start.getTime())); | ||
luddaniel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
private SolrQueryResponse findHits(SavedSearch savedSearch) throws SearchException { | ||
String sortField = SearchFields.TYPE; // first return dataverses, then datasets | ||
String sortOrder = SortBy.DESCENDING; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to leave comments below about backward incompatible changes. I'm pretty sure my preference is to NOT introduce any backward-incompatible changes but if we do the should be listed in
doc/sphinx-guides/source/api/changelog.rst
and the release notes should give an overview of backward-incompatible changes and tell people to check that changelog.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, I removed every authentication requirement for
/api/admin/savedsearches