Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvements for large orgs #2692

Merged
merged 17 commits into from
Jun 30, 2024
Merged

Performance improvements for large orgs #2692

merged 17 commits into from
Jun 30, 2024

Conversation

jdorn
Copy link
Member

@jdorn jdorn commented Jun 26, 2024

Features and Changes

Large orgs with many feature flags, experiments, and/or users can experience long load times for the GrowthBook app.

During investigation, I came up with a few key areas for improvement:

  1. We're sending unnecessary data down to the front-end (e.g. archived experiments)
  2. We're doing too many API calls during app startup
  3. We're doing too many expensive Mongo queries during key API requests

This PR includes a number of fixes for the above, plus some more minor performance issues:

  • Exclude archived features and experiments throughout app unless they are explicitly requested (e.g. by clicking "Show Archived" on the feature list)
  • In-memory cache of user id -> email / display name map
  • Remove duplicate API calls to /user, /organization, and /subscription/quote
  • Remove /user/watching route and return the data as part of GET /organization instead
  • Add missing Mongo indexes for teams (queried on every request), apikeys (queried on every page load), and features (queried on almost every page load)
  • Return cache header for /init endpoint (called on every page load)
  • Switch TeamModel, OrganizationModel, UserModel, ApiKeyModel, and WatchModel to use the native MongoDB driver instead of Mongoose for reading
  • Skip API calls to /usage/features if realtime graphs are not being displayed
  • Update swr library and add options to the useApi hook to selectively optimize API calls
  • Remove lots of unnecessary findOrganizationById calls in the back-end
  • Optimize font loading on front-end

Future, Bigger Changes

  • Migrate all db models to new BaseModel class (uses native MongoDB under the hood instead of Mongoose)
  • Use simplified interfaces for definitions on the front-end (metrics, data sources, etc.). Just the bare minimum to display throughout the app.
  • Use simplified interfaces for features and experiments in the front-end
  • Move authentication and org init to server-side rendering

Potential regressions / breaking changes

  • If a user updates their display name, it may now take up to 15 minutes to be reflected on other's accounts (e.g. in comment avatars). The change will be immediately reflected on the user's own account though.
  • If important environment variables change (e.g. API_HOST), you now may need to do a hard browser refresh to see the changes. Most browsers are smart enough where just refreshing the page will work fine.
  • GET /api/v1/feature-keys now excludes archived feature flags. GET /api/v1/features is unchanged and continues to return archived features
  • We are no longer returning archived features or experiments to the front-end unless they are explicitly asked for. This may cause some potential broken link edge cases (common ones were tested and work fine).
  • The entire initial app boostrap process was re-written, which might introduce new bugs or edge cases. Common scenarios were tested and work fine.

@jdorn jdorn marked this pull request as ready for review June 28, 2024 13:16
Copy link

github-actions bot commented Jun 28, 2024

Your preview environment pr-2692-bttf has been deployed with errors.

@jdorn jdorn merged commit 94cf235 into main Jun 30, 2024
2 of 3 checks passed
@jdorn jdorn deleted the jd/perf-improvements branch June 30, 2024 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant