Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix journal abbbrev checker for curly braces #9504

Merged
merged 45 commits into from
Jan 2, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
2628b59
Fix journal abbbrev checker for curly braces
Siedlerchr Dec 28, 2022
79e2531
refactor abbreviation checker to latex free entry checker
Siedlerchr Dec 28, 2022
28915bb
adjust test cases
Siedlerchr Dec 28, 2022
b8d72ce
checkstyle
Siedlerchr Dec 28, 2022
e4bf0af
checkstyle
Siedlerchr Dec 28, 2022
6bb8db4
Merge branch 'main' into fixJournalAbbrevChecker
Siedlerchr Dec 31, 2022
1a5c777
Streamline tests and add tests for 9475 and 9503
koppor Jan 1, 2023
c2c590b
Merge remote-tracking branch 'origin/main' into fixJournalAbbrevChecker
koppor Jan 1, 2023
54fcbd4
Rename Medline to Dotless and add translations for the menu actions
koppor Jan 1, 2023
2779e18
On our way
koppor Jan 1, 2023
630c772
Add misisng test
koppor Jan 1, 2023
9399277
Try to use full MVMap in groovy script
koppor Jan 1, 2023
5801e92
Fix path
koppor Jan 1, 2023
626e41a
Streamline action
koppor Jan 1, 2023
b6beabe
Try to commit changes
koppor Jan 1, 2023
e649c5e
Revert "Streamline action"
koppor Jan 1, 2023
155617e
Try to fix action
koppor Jan 1, 2023
ac82d8e
Debug: show branch
koppor Jan 1, 2023
f6d743f
Stay on current branch
koppor Jan 1, 2023
acf5b91
Fix action
koppor Jan 1, 2023
0e37d5d
Remove bad files
koppor Jan 1, 2023
16e0bd8
Serizalizabe version 1
koppor Jan 1, 2023
d8fd007
Fix duplicate handling
koppor Jan 1, 2023
46d8027
Add trigger on braches
koppor Jan 1, 2023
a45c505
fix
koppor Jan 1, 2023
b5fcfad
Fix wrong comment
koppor Jan 1, 2023
e8a6124
Some debug maybe?
koppor Jan 1, 2023
6d0e9b8
debug
koppor Jan 1, 2023
f6f6aa8
Add warning
koppor Jan 1, 2023
623514e
Try to use a single MV store only
koppor Jan 2, 2023
38bcf84
Fix import
koppor Jan 2, 2023
c7b4267
CamleCase to dash-case
koppor Jan 2, 2023
1b0e431
Fix brace
koppor Jan 2, 2023
540b26a
Update journal abbrev list
koppor Jan 2, 2023
ba04c0a
Remove debug output
koppor Jan 2, 2023
90f7825
Internalize loading
koppor Jan 2, 2023
28bf41d
Disable workflow
koppor Jan 2, 2023
25f517e
Update CHANGELOG.md
koppor Jan 2, 2023
b012a08
Remove initialization hack
koppor Jan 2, 2023
5c66561
Update journal abbrev list
koppor Jan 2, 2023
f579568
No need to fetch
koppor Jan 2, 2023
e61c492
Use right WoS file
koppor Jan 2, 2023
3b23a41
Update journal abbrev list
koppor Jan 2, 2023
70a804f
fix l10n and reorder
Siedlerchr Jan 2, 2023
a6c791d
Merge remote-tracking branch 'origin/main' into fixJournalAbbrevChecker
koppor Jan 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Note that this project **does not** adhere to [Semantic Versioning](http://semve
- The tab "deprecated fields" is shown in biblatex-mode only. [#7757](https://github.com/JabRef/jabref/issues/7757)
- We fixed an issue where the last opened libraries were not remembered when a new unsaved libray was open as well [#9190](https://github.com/JabRef/jabref/issues/9190)
- We fixed an issue where no context menu for the group "All entries" was present [forum#3682](https://discourse.jabref.org/t/how-sort-groups-a-z-not-subgroups/3682)
- We fixed an issue where extra curly braces in some fields would trigger an exception when selecting the entry or doing an integrity check [#9475](https://github.com/JabRef/jabref/issues/9475), [#9503](https://github.com/JabRef/jabref/issues/9503)

### Removed

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ public boolean unabbreviate(BibDatabase database, BibEntry entry, Field field, C
return true;
}

String text = entry.getField(field).get();
String text = entry.getLatexFreeField(field).get();
String origText = text;
if (database != null) {
text = database.resolveForStrings(text);
Expand Down
26 changes: 15 additions & 11 deletions src/main/java/org/jabref/logic/integrity/AbbreviationChecker.java
Original file line number Diff line number Diff line change
@@ -1,29 +1,33 @@
package org.jabref.logic.integrity;

import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import java.util.Set;

import org.jabref.logic.journals.JournalAbbreviationRepository;
import org.jabref.logic.l10n.Localization;
import org.jabref.model.strings.StringUtil;
import org.jabref.model.entry.BibEntry;
import org.jabref.model.entry.field.Field;
import org.jabref.model.entry.field.FieldFactory;

public class AbbreviationChecker implements ValueChecker {
public class AbbreviationChecker implements EntryChecker {

private final JournalAbbreviationRepository abbreviationRepository;
private final Set<Field> fields = FieldFactory.getBookNameFields();

public AbbreviationChecker(JournalAbbreviationRepository abbreviationRepository) {
this.abbreviationRepository = abbreviationRepository;
}

@Override
public Optional<String> checkValue(String value) {
if (StringUtil.isBlank(value)) {
return Optional.empty();
public List<IntegrityMessage> check(BibEntry entry) {
Siedlerchr marked this conversation as resolved.
Show resolved Hide resolved
List<IntegrityMessage> messages = new ArrayList<>();
for (Field field : fields) {
Optional<String> value = entry.getLatexFreeField(field);
value.filter(abbreviationRepository::isAbbreviatedName)
.ifPresent(val -> messages.add(new IntegrityMessage(Localization.lang("abbreviation detected"), entry, field)));
}

if (abbreviationRepository.isAbbreviatedName(value)) {
return Optional.of(Localization.lang("abbreviation detected"));
}

return Optional.empty();
return messages;
}
}
3 changes: 0 additions & 3 deletions src/main/java/org/jabref/logic/integrity/FieldCheckers.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,6 @@ public FieldCheckers(BibDatabaseContext databaseContext, FilePreferences filePre
private static Multimap<Field, ValueChecker> getAllMap(BibDatabaseContext databaseContext, FilePreferences filePreferences, JournalAbbreviationRepository abbreviationRepository, boolean allowIntegerEdition) {
ArrayListMultimap<Field, ValueChecker> fieldCheckers = ArrayListMultimap.create(50, 10);

for (Field field : FieldFactory.getBookNameFields()) {
fieldCheckers.put(field, new AbbreviationChecker(abbreviationRepository));
}
for (Field field : FieldFactory.getPersonNameFields()) {
fieldCheckers.put(field, new PersonNamesChecker(databaseContext));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ public JournalInAbbreviationListChecker(Field field, JournalAbbreviationReposito

@Override
public List<IntegrityMessage> check(BibEntry entry) {
Optional<String> value = entry.getField(field);
Optional<String> value = entry.getLatexFreeField(field);
if (value.isEmpty()) {
return Collections.emptyList();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.regex.PatternSyntaxException;
import java.util.stream.Collectors;

import org.h2.mvstore.MVMap;
Expand Down Expand Up @@ -65,7 +66,7 @@ public boolean isKnownName(String journalName) {
}

return fullToAbbreviation.containsKey(journal) || abbreviationToFull.containsKey(journal)
|| findDottedAbbrFromDotless(journal).length() > 0;
|| (findDottedAbbrFromDotless(journal).length() > 0);
}

/**
Expand All @@ -80,7 +81,7 @@ public boolean isAbbreviatedName(String journalName) {

return customAbbreviations.stream().anyMatch(abbreviation -> isMatchedAbbreviated(journal, abbreviation))
|| abbreviationToFull.containsKey(journal)
|| (isMoreThanTwoWords && findDottedAbbrFromDotless(journal).length() > 0);
|| (isMoreThanTwoWords && (findDottedAbbrFromDotless(journal).length() > 0));
}

public String findDottedAbbrFromDotless(String journalName) {
Expand All @@ -94,6 +95,7 @@ public String findDottedAbbrFromDotless(String journalName) {
// check for a dot-less abbreviation
if (!DOT.matcher(journalName).find()) {
// use dot-less abbr to find full name using regex

String[] journalSplit = journalName.split(" ");

for (int i = 0; i < journalSplit.length; i++) {
Expand All @@ -104,7 +106,15 @@ public String findDottedAbbrFromDotless(String journalName) {
String joined = String.join("", journalSplit);

foundKey = abbreviationToFull.keySet().stream()
.filter(s -> Pattern.compile(joined).matcher(s).find())
.filter(s -> {
Siedlerchr marked this conversation as resolved.
Show resolved Hide resolved
try {
return Pattern.compile(joined).matcher(s).find();
}
Siedlerchr marked this conversation as resolved.
Show resolved Hide resolved
catch (PatternSyntaxException ignored){
Siedlerchr marked this conversation as resolved.
Show resolved Hide resolved
// if for some reason the latex free field still contains illegal chars we ignore the exception
return false;
}
})
.collect(Collectors.joining());
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
package org.jabref.logic.integrity;

import java.util.Optional;
import java.util.Collections;

import org.jabref.logic.journals.Abbreviation;
import org.jabref.logic.journals.JournalAbbreviationLoader;
import org.jabref.logic.journals.JournalAbbreviationRepository;
import org.jabref.model.entry.BibEntry;
import org.jabref.model.entry.field.StandardField;
import org.jabref.model.entry.types.StandardEntryType;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
Expand All @@ -16,32 +19,49 @@ class AbbreviationCheckerTest {

private JournalAbbreviationRepository abbreviationRepository;
private AbbreviationChecker checker;
private BibEntry entry;

@BeforeEach
void setUp() {
abbreviationRepository = JournalAbbreviationLoader.loadBuiltInRepository();
abbreviationRepository.addCustomAbbreviation(new Abbreviation("Test Journal", "T. J."));
entry = new BibEntry(StandardEntryType.InProceedings);
checker = new AbbreviationChecker(abbreviationRepository);
}

@Test
void checkValueComplainsAboutAbbreviatedJournalName() {
assertNotEquals(Optional.empty(), checker.checkValue("T. J."));
void checkEntryComplainsAboutAbbreviatedJournalName() {
entry.setField(StandardField.BOOKTITLE, "T. J.");
assertNotEquals(Collections.emptyList(), checker.check(entry));
}

@Test
void checkValueDoesNotComplainAboutJournalNameThatHasSameAbbreviation() {
void checkEntryDoesNotComplainAboutJournalNameThatHasSameAbbreviation() {
entry.setField(StandardField.BOOKTITLE, "Journal");
abbreviationRepository.addCustomAbbreviation(new Abbreviation("Journal", "Journal"));
assertEquals(Optional.empty(), checker.checkValue("Journal"));
assertEquals(Collections.emptyList(), checker.check(entry));
}

@Test
void checkValueDoesNotComplainAboutJournalNameThatHasΝοAbbreviation() {
assertEquals(Optional.empty(), checker.checkValue("IEEE Software"));
void checkEntryDoesNotComplainAboutJournalNameThatHasΝοAbbreviation() {
entry.setField(StandardField.BOOKTITLE, "IEEE Software");
assertEquals(Collections.emptyList(), checker.check(entry));
}

@Test
void checkValueDoesNotComplainAboutJournalNameThatHasΝοInput() {
assertEquals(Optional.empty(), checker.checkValue(""));
void checkEntryDoesNotComplainAboutJournalNameThatHasΝοInput() {
assertEquals(Collections.emptyList(), checker.check(entry));
}

@Test
void checkEntryWorksForLaTeXField() {
entry.setField(StandardField.BOOKTITLE, "Reducing Complexity and Power of Digital Multibit Error-Feedback $\\Delta$$\\Sigma$ Modulators");
assertEquals(Collections.emptyList(), checker.check(entry));
}

@Test
void checkEntryWorksForLaTeXFieldStilContainingIllegalChars() {
entry.setField(StandardField.BOOKTITLE, "Proceedings of the 5\\({}^{\\mbox{th}}\\) Central-European Workshop on Services and their Composition, Rostock, Germany, February 21-22, 2013");
assertEquals(Collections.emptyList(), checker.check(entry));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -267,4 +267,19 @@ void testJournalDotlessAbbreviation() {
.withField(StandardField.JOURNAL, "ACS Applied Materials & Interfaces");
assertEquals(expectedAbbreviatedJournalEntry, abbreviatedJournalEntry);
}

@Test
void testJournalDotlessAbbreviationWithCurlyBraces() {
BibDatabase bibDatabase = new BibDatabase();
JournalAbbreviationRepository journalAbbreviationRepository = JournalAbbreviationLoader.loadBuiltInRepository();
UndoableUnabbreviator undoableUnabbreviator = new UndoableUnabbreviator(journalAbbreviationRepository);

BibEntry abbreviatedJournalEntry = new BibEntry(StandardEntryType.Article);
abbreviatedJournalEntry.setField(StandardField.JOURNAL, "{ACS Appl Mater Interfaces}");

undoableUnabbreviator.unabbreviate(bibDatabase, abbreviatedJournalEntry, StandardField.JOURNAL, new CompoundEdit());
BibEntry expectedAbbreviatedJournalEntry = new BibEntry(StandardEntryType.Article)
.withField(StandardField.JOURNAL, "ACS Applied Materials & Interfaces");
assertEquals(expectedAbbreviatedJournalEntry, abbreviatedJournalEntry);
}
}