Recommended way to migrate events between event stores #1066
Replies: 5 comments 2 replies
-
|
Let's get the easiest out of the way first. Migrating from v0 to v1 in regard to data, is a no-op https://geteventflow.net/migrations/v0-to-v1/ as the data format doesn't change. Regarding moving from one data store to another, there sadly isn't a tested method. Depending on your system, it could be many GBs or TBs worth of data. I can only give you my thoughts as this isn't something I have considered before, but it certainly is a valid case. Consider if you need to be live when the migration is being done. If you need to be live, you could potentially solution would be for you to create your own implementation of Is the wish to move to MongoDB due to the fact that EventStore isn't migrated to v1 yet? If so, then it might be easier and significantly more safe for you if I (or yourself) create a PR to get it moved to v1. |
Beta Was this translation helpful? Give feedback.
-
We do not need to be live, we have regularly scheduled maintenance windows where this could be done "offline".
No, the ultimate wish is to part ways with EventStore due to the high resource overhead of having a dedicated server compared to the relatively low event traffic it receives. We've been facing issues with resource spikes when EventStore restarts and re-indexes which requires way more resources compared to standard runtime operation. Ultimately, I was looking for guidance in replaying events from 1 store into another. I want to do this using EventFlow.MongoDB so that it sets up the databases/collections/events schema the way its expecting. I know there is support to replay the events, but that plays back into the read models. I'm sure I can dig into the source and figure out a way to do this by implementing interfaces somewhere. Probably will load all events using the EventStore store and save them all with a custom |
Beta Was this translation helpful? Give feedback.
-
|
FYI I intend on migrating EventStore to v1 early in the new year. I've made
a draft PR that hasn't had any movement on as I've been busy with the
domain model and only using an in-memory event store for unit tests. Will
pick it up after the holidays...
…On Mon, Dec 23, 2024 at 9:37 AM Rasmus Mikkelsen ***@***.***> wrote:
Let's get the easiest out of the way first. Migrating from v0 to v1 in
regard to data, is a no-op https://geteventflow.net/migrations/v0-to-v1/
as the data format doesn't change.
Regarding moving from one data store to another, there sadly isn't a
tested method. Depending on your system, it could be many GBs or TBs worth
of data. I can only give you my thoughts as this isn't something I have
considered before, but it certainly is a valid case.
Consider if you need to be live when the migration is being done. If you
need to be live, you could potentially solution would be for you to create
your own implementation of IEventPersistence that wraps both your current
EventStore store and your future MongoDB store. Then whenever an aggregate
is loaded it is automatically moved from one store to the other (if not
already there) and then only committed to MongoDB (or both if you want to
be able to gradually move).
Is the wish to move to MongoDB due to the fact that EventStore isn't
migrated to v1 yet? If so, then it might be easier and significantly more
safe for you if I (or yourself) create a PR to get it moved to v1.
—
Reply to this email directly, view it on GitHub
<#1066 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAYOIPD4VGF43KPK5LB7EXT2G64SZAVCNFSM6AAAAABTYP42FSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRUG43TENQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
Does this change anything? Yet another NuGet rename? 😅
|
Beta Was this translation helpful? Give feedback.
-
|
Oh wow, yes it does! I'll need to rebrand to align, and will reach out to
Rasmus in the new year about options. I'm not sure that supporting .net
standard is an option, I'd like to only implement a dot net v8+ version;
maybe create a completely new package aligned with the rebanded name that
reflects this is the best approach?
…On Tue, Dec 24, 2024 at 10:31 AM Rasmus Mikkelsen ***@***.***> wrote:
Does this change anything? Yet another NuGet rename? 😅
https://www.kurrent.io/blog/kurrent-re-brand-faq
image.png (view on web)
<https://github.com/user-attachments/assets/99501a6e-d855-453e-bf31-247c1b324018>
—
Reply to this email directly, view it on GitHub
<#1066 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAYOIPHXWUBM7AJ5PACQMX32HELURAVCNFSM6AAAAABTYP42FSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRVGY2TKOA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Yes, |
Beta Was this translation helpful? Give feedback.
-
|
They are moving fast |
Beta Was this translation helpful? Give feedback.
-
Our project is on v0 using EventStore for event sourcing and Mongo for read models. We would like to migrate to Mongo for event sourcing as well, and then ultimately migrate to v1. What would be the easiest way to go about the event sourcing migration?
Beta Was this translation helpful? Give feedback.
All reactions