Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RemoveDocument API #484

Merged
merged 7 commits into from
Mar 13, 2023
Merged

Add RemoveDocument API #484

merged 7 commits into from
Mar 13, 2023

Conversation

hackerwins
Copy link
Member

@hackerwins hackerwins commented Mar 6, 2023

What this PR does / why we need it:

Add document deletion API.

Document deletion is crucial feature to save resources. Without deletion, accumulated documents and its relevant data will waste storage resources.

The overall document deletion process will look like this:

  1. Client sends document delete API(RemoveDocument) with DocIsRemoved = true in request ChangePack
  2. In PushPull of RemoveDocument(or other APIs), check DocIsRemoved. And if DocIsRemoved = true, simply set RemovedAt date in docInfo to logically(soft) remove document.
  3. Set response ChangePack's DocIsRemoved to true to inform document deletion.
  4. After certain period of time(DocumentRemoveDuration), document is physically removed from database.

To successfully implement this API, we need to seperate our tasks and implement them step by step.

  • 1. Implement basic document deletion API with simply setting RemovedAt date in docInfo
  • 2. Determine whether to include removed documents in ListDocuments API used by Dashboard and CLI(Admin)
  • 3. Implement housekeeping background process to physically remove document(is_removed = true) and its relevant data.

Which issue(s) this PR fixes:

Address #464

Special notes for your reviewer:

Discussion about document deletion API in codepair: https://codepair.yorkie.dev/ylvpas

Previous closed PR for implementing document deletion API: #468

WatchDocuments API also needs to find documents by ID, not by key.

Does this PR introduce a user-facing change?:

adds Client.Remove(document), *RemoveDocument API*

Additional documentation:


Checklist:

  • Added relevant tests or not required
  • Didn't break anything

@krapie krapie force-pushed the remove-document branch 4 times, most recently from d537884 to d8f30ca Compare March 8, 2023 05:33
@codecov
Copy link

codecov bot commented Mar 8, 2023

Codecov Report

Merging #484 (540f841) into main (48821a8) will increase coverage by 1.00%.
The diff coverage is 45.16%.

@@            Coverage Diff             @@
##             main     #484      +/-   ##
==========================================
+ Coverage   47.78%   48.79%   +1.00%     
==========================================
  Files          69       69              
  Lines        5826     5962     +136     
==========================================
+ Hits         2784     2909     +125     
+ Misses       2733     2712      -21     
- Partials      309      341      +32     
Impacted Files Coverage Δ
client/client.go 10.46% <0.00%> (-1.42%) ⬇️
pkg/document/change/pack.go 0.00% <ø> (ø)
pkg/document/document.go 52.11% <0.00%> (-3.95%) ⬇️
server/backend/database/doc_info.go 0.00% <0.00%> (ø)
server/backend/database/mongo/indexes.go 50.00% <ø> (ø)
server/rpc/yorkie_server.go 46.28% <52.23%> (-0.46%) ⬇️
server/backend/database/mongo/client.go 24.90% <54.83%> (+10.52%) ⬆️
pkg/document/internal_document.go 22.03% <66.66%> (ø)
server/backend/database/client_info.go 67.50% <75.00%> (-6.48%) ⬇️
server/backend/database/memory/database.go 52.64% <78.26%> (+0.96%) ⬆️
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@krapie krapie changed the title Add client.Remove API Add document deletion API Mar 8, 2023
@krapie krapie force-pushed the remove-document branch 4 times, most recently from fda566d to 048d3f6 Compare March 9, 2023 09:12
@hackerwins hackerwins marked this pull request as ready for review March 9, 2023 12:32
client/client.go Outdated Show resolved Hide resolved
@hackerwins hackerwins force-pushed the remove-document branch 5 times, most recently from 4cb509b to 53906c6 Compare March 10, 2023 02:00
After implementing document deletion, multiple IDs can be mapped to
the same document key.

Therefore, this commit change to use of the document ID for APIs
except attach API.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
protocol changed 📝 Whether the protocol has changed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants