Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search for an entry without triggering group or other fields #7996

Closed
ThiloteE opened this issue Aug 16, 2021 · 5 comments · Fixed by #11542
Closed

Search for an entry without triggering group or other fields #7996

ThiloteE opened this issue Aug 16, 2021 · 5 comments · Fixed by #11542

Comments

@ThiloteE
Copy link
Member

ThiloteE commented Aug 16, 2021

JabRef 5.4--2021-08-15--96061b7
Windows 10 10.0 amd64
Java 16.0.2
JavaFX 16+8

Problem

  • when i search for an entry that is within a group and the groupname is similar to what i search for, the group field triggers and thereby unfortunately shows all entries within that group, even though there are many entries that are not relevant to my search.
  • the problem intensifies when having hundreds of groups and thousands of entries
  • Search commands such as "not group and title" do not work

Example
In the following example, the entry "Bordo" is not wanted
bug 5 4 search for entries without triggering group

Workaround
In the following screenshot i use author=böhme as a workaround to exclude other entries.
bug 5 4 search for entries without triggering group  solution author=xxx

Why post this on github, even though there is a workaround?

  • If there are 10 fields (example for a field: author, range of date, title, group, etc.) and one wants to have a search with 8 out of 10 included, but 2 fields excluded, then this will lead to a long chain of search commands.
  • Typing long chains of search commands takes a long time and is prone to errors
  • Typing long chains of search commands just to exclude ONE single field of entry (in this case 'groups') seems not very efficient.
  • Having to use search commands is not very intuitive for new users of JabRef. I only found out they existed after finding this website: https://docs.jabref.org/finding-sorting-and-cleaning-entries/search, not from within JabRef. Only later i noticed that there is a short explanation when hovering over search for a little while.

Solution

  • Increase visibility of search commands
  • Create new functionality and/or extend existing functionality that allows to exclude certain fields (e.g. Groups) from a search.

i am sure there are many ways to Rome.

Additional Notes

To be clear: i do NOT argue for the removal of to search all fields. (= search everywhere) I think it is a very good start for a search. Also, i do NOT argue for the removal of being able to search for terms that are located within the group field.

@ThiloteE
Copy link
Member Author

suggested tags:

  • enhancement
  • search
  • groups

@ThiloteE
Copy link
Member Author

thank you :) That came as fast as lightning xD

@ThiloteE
Copy link
Member Author

ThiloteE commented Nov 23, 2021

JabRef 5.4--2021-11-15--9955a33
Windows 10 10.0 amd64
Java 16.0.2
JavaFX 17.0.1+1

I checked again today if this works. To make it more clear, why this issue is relevant and not just a hypothetical theoretical plaything, i provide a real world example:

What i want:

  • I want to create one big library(-file) for all my projects, therefore i create groups for each single project. (I know we now have global search again, but still...)
  • I wanted to search within a group with many entries for the author "Keynes" or any entries that mention Keynes in any other field EXCEPT GROUPS, because the groupname is related to Keynes.

What happens:

  • ALL entries within that group are displayed as search result.
    image

  • this stands at odds with more stringent search results. If i remove all the entries from the group, i only find 5 entries that have bibliographic metadata that is related to Keynes in my library:
    image
    Finding only these 5 entries is what is desired.

  • when i attach all the entries to the group again and try to search within this group, but try to exclude the group from being searched with the syntax @koppor used in Fix search: NOT binds more than AND #8241, then it results in NO entries being displayed at all:
    image

All this becomes even messier when entries are part of more than one group.

This issue blocks my aspiration of creating one big library file.

@ThiloteE
Copy link
Member Author

related discussion in the forum: https://discourse.jabref.org/t/can-i-have-mutually-exclusive-groups/3015

@ThiloteE
Copy link
Member Author

ThiloteE commented May 25, 2022

Another workaround via RegEx search:

  1. Have a group called Colistra.

  2. Have two entries:

    @Article{test1,
      author       = {Rita Colistra},
    

    and

    @Article{test2,
      groups = {Colistra},
    }
    
  3. Activate RegEx search

  4. Enter this line into search bar: Colistra and groups != Colistra

Result:

only the entry test1 will show, which is exactly what I want.

Something similar should work out of the box and not just via RegEx.

Additional Info:

For anybody who is willing to work on this:

JabRef maintainers are currently in the middle of adopting Lucene as search syntax, so this is relevant, but adopting Lucene is the main issue. See #8206

@LoayGhreeb LoayGhreeb mentioned this issue Sep 2, 2024
6 tasks
github-merge-queue bot pushed a commit that referenced this issue Sep 5, 2024
* Use pattern matching for cast

Co-authored-by: Christoph <siedlerkiller@gmail.com>

* Fix pattern matching

* Fix merge

* Speed up switches between sorting/filtering modes

* Fixed merge errors

* Fixed small issues

* Removed obsolete tests, fixed some tests

* Fixed merge error in CHANGELOG.md

* Fixed checkstyle

* Fixed more tests

* Removed obsolete tests

* Fixes "Fixed merge error in CHANGELOG.md" by removing duplicate entries

This reverts commit 536ecfa.

* WiP on tests

* Checkstyle

* Checkstyle

* Update Java version

* Refine logging

* Fix compile error

* Add LuceneTest

* Update CHANGELOG.md

* Move search classes to pdf package

* Move search classes to search package

* rewriteRun

* Remove bibEntry from DocumentReader

* Rewrite LuceneIndexer

* Remove IndexingTaskManager

* Separate Bib fields index and LinkedFiles index

* Fix null LuceneManager in ExternalFilesEntryLinker

* Save as action

* Clear linkedFiles indexer when fullText indexing is disabled in preferences

* remove comments

* get indexed files on update

* Add LUCENE_MANAGERS map for accessing managers by databaseContext.getUid

* Move LuceneManager from search.indexing package to search

* Fix wrong order for import

* Move SearchQuery to model package

* Fix issue with opening multiple unsaved libraries

* Pass LuceneManager down to the entry editor

* Improve searching performance

* Change SearchFieldConstants to enum

* More performance improvements for searching

- Read document only one time
- getHighlighterFragments only when the search results tab is opened

* Update FulltextSearchResultsTab.java

* Fix group union, intersection

* Fix backgroundtask

* Fix subscriptions

* Remove lastSearchQueryLogic

* Fix possible NPE

* Fix searchTask check

* Remove sort by score flag

* Fix score column sorting

* Fix modifier buttons listener

* Add search rank column

In floating mode entries will be ranked and sorted by it.
Rank: (1= entry matches group and search, 2= matches group but not search, 3= matches search but not group, 4= matches nothing)

* hide search rank column from preferences

* Add search_rank column to sort order by default

* Update CHANGELOG.md

* fix typo

* Change the order of the rank

1= entry matches group and search, 2= matches search but not group, 3= matches group but not search, 4= matches nothing

* Use NGramAnalyzer for indexing

* Resolve conflicts

* update search matches with lucene

* PreviewViewer highlighting with Lucene

* Delete IndexingTaskManager.java

* SourceTab highlighting with Lucene

* Fix non-ASCII characters

* Extract query terms from search query

* Highlight regex queries

* return js highlight function

* Fix invalid search query throw exception

* Refactor Lucene indexer classes

* Refactor linked files indexer

* Update search matches when entries are added or updated

* Remove preferences from ActionHelper

* checkstyle

* comment out search tests

* OpenRewrite

* Fix Groups Parser/Serializer

* Localization

* Search groups

* Release `IndexSearcher` after completing search task

* Checkstyle

* Correct typo

* Remove GroupSearchQuery

* Remove EventBus from LuceneManager and use BibDatabase eventBus

* Fix number of matched entries in groups

* Fix search groups

* Localization

* Remove bib fields highlighter

* Pass LuceneManager to search groups

* Fix performance issues by caching matched entries

* Update GroupDialogViewModelTest.java

* Update main table matches

* Fix groups icon

* Restore Search.g4 and GrammarBasedSearchRule

* First version of search group migration

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Add groups field to the index

* Remove search rules

* Localization

* Add test cases

* Fix names

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Add some more functionality

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Always add "all" prefix

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Add comment for alternative implementation

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Mark library tab changed after migration

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Add another test for regular expression

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Small fixes

* Fix markBaseChanged

* Fix adding new entries did not update MatchCategory

* Fix searching for Non-ASCII characters

* Fix escaping special characters

Use WhitespaceTokenizer instead of StandardTokenizer
https://stackoverflow.com/a/6119584/21694752

* Fix tests

Co-Authored-By: Oliver Kopp <kopp.dev@gmail.com>

* Add first draft of LatexToUnicodeFoldingFilter

Co-authored-by: Loay Ghreeb <52158423+LoayGhreeb@users.noreply.github.com>

* Fix LatexToUnicodeFoldingFilter

Co-Authored-By: Oliver Kopp <kopp.dev@gmail.com>

* Remove LatexToUnicode from SearchQuery

* Localization

* AllowedToUseLogic

* Update CHANGELOG.md

* Use sentence case for search result heading

* Add CHANGELOG for change in JabRefFrameViewModel

* Add more changes to CHANGELOG.md

* Add ADR-0038

* Rename "SCORE" to "MATCH_SCORE"

* Add link to ADR-0038

* Add another CHANGELOG.md entry

* Add CHANGELOG.md entry

* Revert change of filename

* Add JavaDoc comment

* Trying to find better names

* Discard changes to src/main/resources/tinylog.properties

* Remove commented out code

* Remove obsolete testing class

* Remove obsolete test

* Discard changes to src/test/resources/tinylog-test.properties

* Remove completely disabled code

* Rename "all" to "any"

* Catch thrown exception

Invalid regex queries throws an exception

* Remove groups field from the default field

#7996

* Remove SearchGroupsListener

* Update Benchmarks.java

* Update module-info.java

* Fixes from code review on LibraryTab

* Remove regex button from search bar

* Use BibEntry.getId instead of System.identityHashCode

* Add BibEntry index map

* Readd option

* Add `@ADR` annotation

* Add some comment

* One more annotation

* Add CHANGELOG.md entry

* One more annotation

* Add CHANGELOG.md entry

* Revert "Add BibEntry index map"

This reverts commit 27ed105.

* Use binary search to find the index of the entry

* openrewrite

* Tests for LinkedFilesIndexer

* Fix DatabaseSearcher

* LocalizationConsistencyTest

* DatabaseSearcherWithBibFilesTest

* Fix typo in CHANGELOG.md

* Fix typo

* Use parameterized test for DatabaseSearcherTest

* Fix DatabaseSearcherWithBibFiles tests

* Fix exportMatches test

* Remove regex check box from search groups dialog

* JavaDoc

* Fix SearchGroups test

* Remove closeAndWait methods and use CurrentThreadTaskExecutor

* Fix architecture test

* Allow to use logic

* Add debug logging for search

* Add more logging

* Assert with containsInAnyOrder

* Fix DatabaseSearcher test

* Global search dialog

* Rename method

* Improve code quality

- Maintain a map of BibEntryId to BibEntry.
- Store search results within SearchQuery instead of using the map in StateManager.
- Remove LuceneManager from SearchGroups.
- Use a different Analyzer for PDFs.

* Use non-static preferences variables

* Update CHANGELOG.md

* Delete SearchGroupTest.java

* fix typo

* fix indentation

* Update matchedEntries on the UI thread

matchedEntries should be updated on the UI thread because the size binding of matchedEntries will be reflected in the UI.

* Discard changes to src/main/java/org/jabref/gui/importer/actions/GUIPostOpenAction.java

* Fix LoayGhreeb#12

* Sync search flags between search bar and global search bar

* Move VERSION_6_0_ALPHA const to SearchGroupsMigrationAction

* Refactor LuceneSearcher

* Use linked files analyzer for highlighting full-text results

* Fix line break

* Fix tests

* Use EnglishAnalyzer for indexing/searching linked files

https://github.com/apache/lucene/blob/68cc8734ca28a9db800e4192a636d3b490cfd41a/lucene/analysis/common/src/java/org/apache/lucene/analysis/en/EnglishAnalyzer.java#L101-L110

* Ask to wait for linked files indexing on shutdown

When closing JabRef, only ask users to wait for the linked files indexer to finish. The bib fields indexer is recalculated on startup, so it doesn't need to be completed before shutdown.

* Use EdgeNGram instead of NGram

* Return comment

* Update CHANGELOG.md

---------

Co-authored-by: Benedikt Tutzer <btut@users.noreply.github.com>
Co-authored-by: Christoph <siedlerkiller@gmail.com>
Co-authored-by: Carl Christian Snethlage <50491877+calixtus@users.noreply.github.com>
Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>
LoayGhreeb added a commit that referenced this issue Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging a pull request may close this issue.

3 participants