Skip to content
This repository has been archived by the owner on Jul 12, 2023. It is now read-only.

Purge old realmless users #1135

Merged
merged 13 commits into from
Nov 24, 2020
Merged

Purge old realmless users #1135

merged 13 commits into from
Nov 24, 2020

Conversation

whaught
Copy link
Contributor

@whaught whaught commented Nov 18, 2020

Fixes #606

Proposed Changes

  • Delete users who have been removed from all realms, are not a system admin, and are older than USER_PURGE_MAX_AGE

Release Note

Cleanup for old users who have no realms and have aged out. This clears their DB information, but not their auth.

@google-cla google-cla bot added the cla: yes Auto: added by CLA bot when all committers have signed a CLA. label Nov 18, 2020
@whaught
Copy link
Contributor Author

whaught commented Nov 18, 2020

I tried to do this as

Joins("LEFT JOIN user_realms ON users.id = user_realms.user_id").
Where("user_realms.user_id IS NULL").

but that didn't seem to work

Copy link
Member

@sethvargo sethvargo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/hold

deleteBefore := time.Now().UTC().Add(maxAge)
// Delete users who were created/updated before the expiry time.
rtn := db.db.Unscoped().
Where("users.system_admin = false AND users.created_at < ? AND users.updated_at < ?", deleteBefore, deleteBefore).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this does what you think it does?

  • Realm membership changes don't change updated_at, so you could end up immediately deleting someone
  • That column is updated anytime the direct user record changes, we can happen at login with the revoke check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latter isn't a problem - it's okay to not-delete users who are logging in to keep fresh.

I suppose we can ensure users is bumped on realm-remove

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we can ensure users is bumped on realm-remove

I couldn't figure out a good way to do that. If we can "touch" the update_at on the user when realm membership is changed, I think this is fine (maybe add a comment).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added TestRemoveRealmUpdatesTime and I can't seem to repro your claim that modifying the realm relationship isn't bumping time

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting... what happens if the user comes serialized from the cache?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the reason is that we have wrapped calls with SaveUser here:

which always saves both the User entry and the associations (even if the table fields haven't changed)

pkg/database/user_test.go Outdated Show resolved Hide resolved
deleteBefore := time.Now().UTC().Add(maxAge)
// Delete users who were created/updated before the expiry time.
rtn := db.db.Unscoped().
Where("users.system_admin = false AND users.created_at < ? AND users.updated_at < ?", deleteBefore, deleteBefore).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting... what happens if the user comes serialized from the cache?

@mikehelmick
Copy link
Contributor

not sure if we can use it - but firebase stores the last login date of a user.

Copy link
Member

@sethvargo sethvargo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/hold

Can you update the release note to warn about this and note that it does not delete them from the upstream auth provider?

@google-oss-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sethvargo, whaught

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-robot
Copy link

New changes are detected. LGTM label has been removed.

@sethvargo
Copy link
Member

Hmm - you might need to rebase @whaught

@sethvargo sethvargo added the lgtm label Nov 24, 2020
@whaught
Copy link
Contributor Author

whaught commented Nov 24, 2020

/retest

@sethvargo
Copy link
Member

/retest

@google-oss-robot google-oss-robot merged commit 101be00 into google:main Nov 24, 2020
@whaught whaught deleted the purge-users branch November 24, 2020 22:57
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 25, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla: yes Auto: added by CLA bot when all committers have signed a CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Delete User if removed from all realms
4 participants