New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reindexing improvements #288

Open

Southclaws opened this issue Nov 10, 2024 · 1 comment

Owner

Southclaws commented Nov 10, 2024

Off the back of #285:

Reindex should not try to do everything all at once
Each datagraph type should have its own reindex job (I think)
Reindexing should be based on a persisted "last index" date, matched to the datagraph date
Generally, reindexing shouldn't happen that often, it's something for when things go wrong (weaviate was off for a period of time, semdex was enabled some time after the instance was set up, etc)

Indexing in general needs some work:

titles of posts aren't being indexed for some reason
deletions currently aren't handled
visibility is not handled correctly: Draft items show in tags list and other areas #283 only published resources should be indexed, un-publishing should count as deleting from the semdex

Owner Author

Southclaws commented Nov 13, 2024

Left to do:

replies
collections
profiles

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment