-
-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mongoose takes a very long time to save large Maps with subdocuments to the database #13191
Closed
2 tasks done
Labels
Milestone
Comments
2 tasks
vkarpov15
added a commit
that referenced
this issue
Apr 16, 2023
vkarpov15
added a commit
that referenced
this issue
Apr 16, 2023
vkarpov15
added a commit
that referenced
this issue
Apr 18, 2023
vkarpov15
added a commit
that referenced
this issue
Apr 27, 2023
perf: speed up creating maps of subdocuments
vkarpov15
added a commit
that referenced
this issue
Jun 5, 2023
vkarpov15
added a commit
that referenced
this issue
Jun 10, 2023
vkarpov15
added a commit
that referenced
this issue
Jul 15, 2023
vkarpov15
added a commit
that referenced
this issue
Jul 17, 2023
perf: speed up mapOfSubdocs benchmark by 4x by avoiding unnecessary O(n^2) loop in getPathsToValidate()
At last, closure! Thanks Valeri! |
Thanks for your patience, took a lot of debugging but the bottleneck should be gone now 👍 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Prerequisites
Mongoose version
7.0.2
Node.js version
16.9.0
MongoDB server version
latest
Typescript version (if applicable)
No response
Description
Mongoose takes an absurdly long time to save a large Map to the database. This problem doesn't happen if you replace the Maps for Arrays.
This was initially reported at #5925 and requested to be made into a new issue when the reproduction script was created, so here it is.
Steps to Reproduce
Clone my reproduction repo here, then either edit your mongoKey into the
index.js
file or add amongoKey.js
file to/private
so it can be imported by theindex.js
file.Once that's done, you can either use
npm start
to start the script in colored mode ornpm run colorless
to use colorless mode.All major steps are color-coded, if you choose to retain the colors, so you can more easily parse the information.
The script will connect to a database and create 20K documents called Minis that only have a small string and a number, then use those to sort and create a Map and an Array version of those documents, sorted by timestamp value.
I've reused the script a few times and I consistently get ~6-minute timers for saving the Maps using mongoose, while saving the Arrays usually takes around ~5-seconds.
Try out the script and let me know if this is an actual bug with the library double checking If every value actually isn't a duplicate of the other value or if this was a PEBCAK.
Expected Behavior
Expected behavior was to not take over 6 minutes to save the Map to the database.
The text was updated successfully, but these errors were encountered: