-
Notifications
You must be signed in to change notification settings - Fork 7.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Branch] - major new UI redesign #2396
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sending mixpanel events. These changes added complexity to the Database code which we're looking to simplify not increase complexity and all info that we care about from mixpanel should be available from the GUI thread. NOTE: This change will not compile by itself. It must be included with subsequent changes made in PR #2302 but is here broken out into a standalone commit to aid in review. Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
…ut of date. Signed-off-by: Adam Treat <treat.adam@gmail.com>
…eaking change that requires re-indexing of all collections. - Adds a general mechanism for handling breaking changes to localdocs database upgrades - Upgrades the database format to version 2 and includes numerous changes to the underlying tables including: a) adds word and token count fields for chunk table b) adds a boolean field indicating whether the chunk has a generated embedding c) adds a last updated time field for collection table which can be displayed in GUI d) adds an embedding model field for collection table e) adds a boolean field for collection table to allow forced re-indexing - Adds several statistics about the collection which can be displayed in GUI a) total documents b) total words c) total tokens d) last updated time e) which document is currently being indexed f) which document is currently being embedded g) which model is being used to generate embeddings for the collection - Simplifies communication between the database thread and the GUI thread removing a lot of extra code and makes clear what methods/signals in database class are soley used for communicating state to the GUI - Provides a pool for sending embedding requests to the embedding thread to increase speed of embedding when the chunks per document are small - Provides a mechanism for resuming indexing/embedding when the app shutsdown or crashes in the middle of indexing/embedding NOTE: This change is part of a larger set of commits, but can be built and compiles up to this point Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
follows the new FIGMA design made by Vincent. It also contains a lot of changes to the way localdocs works. This is now rebased on top of main and conflicts resolved. This is still a monolithic commit that needs to be broken up into smaller pieces in order to aid proper review. I'll be working on that now and force pushing as I do so. Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
5 tasks
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
This comment was marked as outdated.
This comment was marked as outdated.
Operations outside of scanQueue that remove embeddings are now more atomic. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
This makes indexing over 10x faster by syncing the database to disk less often. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
I didn't really understand the code that was here, so I replaced it with something I know works. Key differences: - Always open the latest database that has been *written*, not the latest one that exists on disk. - Never remove databases. We don't have to do it to make this code work, and it's not easy to guarantee that the new db will be available next time before we remove the old one. Plus, we don't want to create a confusing situation if the user downgrades GPT4All (which we do support). Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
The `Q_ASSERT(dir.exists());` assertion in scanDocuments can fail if we `rm -rf` a localdocs collection. Since a folder can be removed at any time, asserting that it exists is fundamentally flawed. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Calling addFolder for an existing folder works, but it is confusing at best. All we actually intend to do is add it to the watch and scan it, so just do that explicitly. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
We were not actually watching subdirectories, because the QDirIterator was missing QDir::Dirs. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
Now we show the release notes link instead of using word 'changelog' and the release notes are also shown in a dialog when you update to a new version. Signed-off-by: Adam Treat <treat.adam@gmail.com>
Pinning this label to the top is confusing, because you may be scrolled into another category, such as "Advanced." Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
This newly added function was returning duplicates and missing a call to `q.next()`, so it was actually returning chunk IDs. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
This button allows the user to rebuild a collection from scratch without having to remove and re-add the folder, in case that is ever necessary. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
This fixes progress display never reaching 100% when deleting a document from a collection while it is being embedded. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Otherwise, empty folders show "INSTALLING" forever. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Adam Treat <treat.adam@gmail.com>
There is already a global setting for n_threads. Use it for embeddings models too. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
Signed-off-by: Jared Van Bortel <jared@nomic.ai>
4e8e038
to
3cce838
Compare
cebtenzzre
approved these changes
Jun 24, 2024
Generated with ❤️ by ellipsis.dev |
cebtenzzre
added a commit
that referenced
this pull request
Jun 25, 2024
This fixes a regression in PR #2396. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
manyoso
pushed a commit
that referenced
this pull request
Jun 25, 2024
This fixes a regression in PR #2396. Signed-off-by: Jared Van Bortel <jared@nomic.ai>
This was referenced Jul 5, 2024
10 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is to track the development of the feature branch for major new UI redesign. This branch will included much more than a UI redesign though including extensive updates and changes to localdocs for example.
The current goal is for this branch is to be merged into main by June 24th. Commits and PR+commits will be merged into this feature branch and this PR will be used to track that progress.
Summary:
Major UI redesign with new QML views and extensive updates to localdocs functionality and backend support.
Key points:
HomeView.qml
,AddCollectionView.qml
,AddModelView.qml
,ModelsView.qml
,LocalDocsView.qml
,SettingsView.qml
ChatView.qml
,ChatDrawer.qml
,Theme.qml
,ApplicationSettings.qml
,ModelSettings.qml
database.cpp
,database.h
,localdocs.cpp
,localdocs.h
Theme.qml
server.cpp
,mysettings.cpp
,mysettings.h
Generated with ❤️ by ellipsis.dev