-
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add testing interface, including a set of capabilities to tests for #6687
Merged
koppor
merged 46 commits into
JabRef:master
from
DominikVoigt:feat/add-capability-tests-for-fetchers
Aug 2, 2020
Merged
Changes from 12 commits
Commits
Show all changes
46 commits
Select commit
Hold shift + click to select a range
a60bdec
Add testing interface, including a set of capabilities to tests for
DominikVoigt 4881f7b
Merge branch 'feat/add-capability-tests-for-fetchers' of https://gith…
koppor 7c13275
Remove all interfaces
koppor c7795e9
Squashed 'src/main/resources/csl-styles/' changes from a995c63a0a..bf…
a89db08
Merge commit 'c7795e9124d55908b8f074c475d62229972a4edc'
5ec7ef6
Merge branch 'feat/add-capability-tests-for-fetchers' of https://gith…
DominikVoigt ea4b67f
Remove empty lines in tests
DominikVoigt 0950cb6
Correct SpringerFetcherTest
DominikVoigt 6bffc68
Rename AdvancedFetcher and AdvancedSearchConfig.java accordingly to c…
DominikVoigt f5f1b5b
Add documentation to the default field
DominikVoigt 7e81cca
Modify AdvancedSearchBasedParserFetcher interface to be in line with …
DominikVoigt db19e60
Adapt test of getAdvancedURL
DominikVoigt 4065fa5
Integrate change requests.
DominikVoigt 7b9e85f
Integrate change requests.
DominikVoigt d2a79fd
Merge remote-tracking branch 'upstream/master' into feat/add-capabili…
DominikVoigt 9673b6c
Start Working on format conversion
DominikVoigt e4b1ad1
Continue integrate format conversion
DominikVoigt de62312
Continue integrate format conversion.
DominikVoigt bb089f4
Merge remote-tracking branch 'upstream/master' into feat/add-capabili…
DominikVoigt 988a275
Get it to build. Start correcting tests.
DominikVoigt a2a26e2
Work on correcting tests.
DominikVoigt 6f482a4
Finalize format conversion changes.
DominikVoigt a746197
Merge remote-tracking branch 'upstream/master' into feat/add-capabili…
DominikVoigt f675e13
Add ADR.
DominikVoigt 9054011
Remove comment.
DominikVoigt c8ead6b
Revert "Squashed 'src/main/resources/csl-styles/' changes from a995c6…
DominikVoigt 1eddd44
Revert "Revert "Squashed 'src/main/resources/csl-styles/' changes fro…
DominikVoigt 92c05be
Update ADRs
DominikVoigt 84a9b53
Update ADR. Remove import.
DominikVoigt 8fa03d5
Remove unused import.
DominikVoigt b86f34e
Merge remote-tracking branch 'upstream/master' into feat/add-capabili…
DominikVoigt c694bc1
Remove whitespaces and redundant paragraph from ADR.
DominikVoigt e1b0bf8
Move format conversion from Fetchers into ImportCleanup.
DominikVoigt d424f88
Revert incorrect change.
DominikVoigt d6eb086
Remove debugging print statement.
DominikVoigt f08184e
Integrate requested changes.
DominikVoigt 8f3fe0f
Integrate requested changes.
DominikVoigt 056cc93
Change test to use ImportCleanup.
DominikVoigt e0fc10a
Merge remote-tracking branch 'upstream/master' into feat/add-capabili…
DominikVoigt 1c87c1d
Remove unused bibentry format
DominikVoigt 190414f
Integrate requested changes
DominikVoigt a37d29e
Inline variable
DominikVoigt 4b6b54b
Modify ADR
DominikVoigt b5d9369
Fix compile error
DominikVoigt cf2ff45
Change ADR title
DominikVoigt 37293c6
Inline some variables
DominikVoigt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
38 changes: 38 additions & 0 deletions
38
src/main/java/org/jabref/logic/importer/AdvancedSearchBasedParserFetcher.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
package org.jabref.logic.importer; | ||
|
||
import java.io.IOException; | ||
import java.io.InputStream; | ||
import java.net.URL; | ||
import java.util.List; | ||
|
||
import org.jabref.logic.importer.fetcher.ComplexSearchQuery; | ||
import org.jabref.model.entry.BibEntry; | ||
|
||
/** | ||
* This interface allows SearchBasedParserFetcher fetchers to test their corresponding | ||
* library APIs for their advanced search options, e.g. search in the "title" field. | ||
*/ | ||
public interface AdvancedSearchBasedParserFetcher extends SearchBasedParserFetcher { | ||
|
||
/** | ||
* This method is used to send queries with advanced URL parameters. | ||
* This method is necessary as the performSearch method does not support certain URL parameters that are used for | ||
* fielded search, such as a title, author, or year parameter. | ||
* | ||
* @param complexSearchQuery the search query defining all fielded search parameters | ||
*/ | ||
default List<BibEntry> performComplexSearchQuery(ComplexSearchQuery complexSearchQuery) throws FetcherException { | ||
try (InputStream stream = getUrlDownload(getComplexQueryURL(complexSearchQuery)).asInputStream()) { | ||
List<BibEntry> fetchedEntries = getParser().parseEntries(stream); | ||
fetchedEntries.forEach(this::doPostCleanup); | ||
return fetchedEntries; | ||
} catch (IOException e) { | ||
// TODO: Catch HTTP Response 401/403 errors and report that user has no rights to access resource | ||
throw new FetcherException("A network error occurred", e); | ||
} catch (ParseException e) { | ||
throw new FetcherException("An internal parser error occurred", e); | ||
} | ||
} | ||
|
||
URL getComplexQueryURL(ComplexSearchQuery complexSearchQuery); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
126 changes: 126 additions & 0 deletions
126
src/main/java/org/jabref/logic/importer/fetcher/ComplexSearchQuery.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
package org.jabref.logic.importer.fetcher; | ||
|
||
import java.util.Objects; | ||
import java.util.Optional; | ||
|
||
import org.jabref.model.strings.StringUtil; | ||
|
||
public class ComplexSearchQuery { | ||
// Field for non-fielded search | ||
private final String defaultField; | ||
private final String author; | ||
private final String title; | ||
private final Integer fromYear; | ||
private final Integer toYear; | ||
private final String journal; | ||
|
||
private ComplexSearchQuery(String defaultField, String author, String title, Integer fromYear, Integer toYear, String journal) { | ||
this.defaultField = defaultField; | ||
this.author = author; | ||
this.title = title; | ||
this.fromYear = fromYear; | ||
this.toYear = toYear; | ||
this.journal = journal; | ||
} | ||
|
||
public Optional<String> getDefaultField() { | ||
return Optional.ofNullable(defaultField); | ||
} | ||
|
||
public Optional<String> getAuthor() { | ||
return Optional.ofNullable(author); | ||
} | ||
|
||
public Optional<String> getTitle() { | ||
return Optional.ofNullable(title); | ||
} | ||
|
||
public Optional<Integer> getFromYear() { | ||
return Optional.ofNullable(fromYear); | ||
} | ||
|
||
public Optional<Integer> getToYear() { | ||
return Optional.ofNullable(toYear); | ||
} | ||
|
||
public Optional<String> getJournal() { | ||
return Optional.ofNullable(journal); | ||
} | ||
|
||
public static AdvancedSearchConfigBuilder builder() { | ||
return new AdvancedSearchConfigBuilder(); | ||
} | ||
|
||
public static class AdvancedSearchConfigBuilder { | ||
private String defaultField; | ||
private String author; | ||
private String title; | ||
private String journal; | ||
private Integer fromYear; | ||
private Integer toYear; | ||
|
||
public AdvancedSearchConfigBuilder() { | ||
} | ||
|
||
public AdvancedSearchConfigBuilder defaultField(String defaultField) { | ||
if (Objects.requireNonNull(defaultField).isBlank()) { | ||
throw new IllegalArgumentException("Parameter must not be blank"); | ||
} | ||
this.defaultField = defaultField; | ||
return this; | ||
} | ||
|
||
public AdvancedSearchConfigBuilder author(String author) { | ||
if (Objects.requireNonNull(author).isBlank()) { | ||
throw new IllegalArgumentException("Parameter must not be blank"); | ||
} | ||
this.author = author; | ||
return this; | ||
} | ||
|
||
public AdvancedSearchConfigBuilder title(String title) { | ||
if (Objects.requireNonNull(title).isBlank()) { | ||
throw new IllegalArgumentException("Parameter must not be blank"); | ||
} | ||
this.title = title; | ||
return this; | ||
} | ||
|
||
public AdvancedSearchConfigBuilder fromYear(Integer fromYear) { | ||
this.fromYear = Objects.requireNonNull(fromYear); | ||
return this; | ||
} | ||
|
||
public AdvancedSearchConfigBuilder toYear(Integer toYear) { | ||
this.toYear = Objects.requireNonNull(toYear); | ||
return this; | ||
} | ||
|
||
public AdvancedSearchConfigBuilder journal(String journal) { | ||
if (Objects.requireNonNull(journal).isBlank()) { | ||
throw new IllegalArgumentException("Parameter must not be blank"); | ||
} | ||
this.journal = journal; | ||
return this; | ||
} | ||
|
||
/** | ||
* Instantiates the AdvancesSearchConfig from the provided Builder parameters | ||
* If all text fields are empty an empty optional is returned | ||
* | ||
* @return AdvancedSearchConfig instance with the fields set to the values defined in the building instance. | ||
* @throws IllegalStateException An IllegalStateException is thrown in case all text search fields are empty. | ||
* See: https://softwareengineering.stackexchange.com/questions/241309/builder-pattern-when-to-fail/241320#241320 | ||
*/ | ||
public ComplexSearchQuery build() throws IllegalStateException { | ||
if (textSearchFieldsAreEmpty()) { | ||
throw new IllegalStateException("At least one text field has to be set"); | ||
} | ||
return new ComplexSearchQuery(defaultField, author, title, fromYear, toYear, journal); | ||
} | ||
|
||
private boolean textSearchFieldsAreEmpty() { | ||
return StringUtil.isBlank(defaultField) && StringUtil.isBlank(title) && StringUtil.isBlank(author) && StringUtil.isBlank(journal); | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it really a problem if the user searches for an empty string, e.g.
author = ""
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey :)
IMHO allowing empty strings for any query parameter does not really make much sense semantically.
I do not see the semantics such an empty string should have:
Did I overlook a use case where giving an empty string for a query parameter is useful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good! I just wanted to double check.
(We should keep this in the back of our heads when the user interface is implemented. There you would like to show a helping message instead of an error dialog).