Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: wait for index idle before returning data #389

Merged
merged 88 commits into from
Nov 28, 2023
Merged

Conversation

gmaclennan
Copy link
Member

@gmaclennan gmaclennan commented Nov 25, 2023

Stacked on #388 -> #386 -> #362

Depends on new multi-core-indexer being published, but otherwise works. This switches to using file storage in tests, to use memory storage set the environment variable FAST_TESTS=1.

To cleanup before publishing:

  • replace github install of multi-core-indexer with published version
  • delete types/temp.d.ts (temporary types for multi-core-indexer)

For now this simplifies the API (because we are only supporting local
sync, not remote sync over the internet) to:

- `project.$sync.getState()`
- `project.$sync.start()`
- `project.$sync.stop()`
- Events
    - `sync-state`

It's currently not possible to stop local discovery, nor is it possible
to stop sync of the metadata namespaces (auth, config, blobIndex). The
start and stop methods stop the sync of the data and blob namespaces.

Fixes #134. Stacked on #360, #358 and #356.
Fixes Add project method to download auth + config cores #233

Rather than call this inside the `client.addProject()` method, instead I
think it is better for the API consumer to call
`project.$waitForInitialSync()` after adding a project, since this
allows the implementer to give user feedback about what is happening.
This was a big lift, but necessary to be able to debug sync issues since
temporarily adding console.log statements was too much work, and
debugging requires knowing the deviceId associated with each message.
This caused a day of work: a bug from months back
@gmaclennan gmaclennan deleted the fix/await-indexing branch November 28, 2023 12:54
gmaclennan added a commit that referenced this pull request Nov 28, 2023
* main:
  fix: wait for index idle before returning data (#389)
  fix: fix file storage race condition (#388)
  feat: remove project.ready() (#392)
  feat: integrate sync and project invites (#362)
gmaclennan added a commit that referenced this pull request Nov 29, 2023
* main:
  fix: wait for index idle before returning data (#389)
  fix: fix file storage race condition (#388)
  feat: remove project.ready() (#392)
  feat: integrate sync and project invites (#362)
gmaclennan added a commit that referenced this pull request Nov 29, 2023
* main:
  fix: wait for index idle before returning data (#389)
  fix: fix file storage race condition (#388)
  feat: remove project.ready() (#392)
  feat: integrate sync and project invites (#362)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Wait for indexing to be done before returning data
2 participants