OEP-65 Composable Micro-frontends

[arbrandes]

No description provided.

[adamstankiewicz]

Just did a quick read-through and added some initial feedback. I'll come back for a deeper dive later :) I'm excited about this OEP!

[brian-smith-tcril]

in the FWG meeting today the question of "what's going to be around in 5-10 years?" was raised in the context of which integration strategy to use. my gut feeling is web components would be a good choice based on that criteria

[adamstankiewicz]

[semi-related] FWIW, there is some desire/interest to try to have a web components implementation available for the components in the Paragon design system so that they could be potentially used in non-React environments (definitely an open question in what that would look like practically, but one of our long-term "vision" goals for Paragon is for it to become more framework agnostic in its usage).

[brian-smith-tcril]

react-router comes to mind, this may be part of a slightly larger conversation about how to handle routing with links between MFEs (since needing a full refresh when navigating from one to another is a pain point in the current SPA architecture)

[adamstankiewicz]

Definitely, routing is a big part of this proposed architecture :)

I think we'll need to consider both the container layer routing as well as any potential routing "local" to individual MFEs. Some micro-frontend frameworks (e.g., "single-spa") provide an API to create top-level routes as a way to define how the MFEs should be integrated (by route).

[adamstankiewicz]

I think it would be beneficial, yes. That said, perhaps only as an opt-in or as-needed basis so those connections between domains are intentional/explicit rather than defaulting to "everything can be shared anywhere"? 🤔

For example, we try to create a distinctly branded Enterprise experience for enterprise learners/administrators and part of that is keeping Enterprise users out of the B2C flow, partially to keep the cohesive Enterprise branding throughout their experience. It would be nice to extend our "walled garden" approach further to be able to pull in pieces of the B2C experience into the Enterprise/B2B domain where it makes sense to create a more cohesive B2B experience (e.g., profile MFE, account settings MFE, etc.).

Another use case: there could be a single learner dashboard MFE for displaying the course enrollments for a user across multiple domains (e.g., B2C and B2B), where the only substantive difference in how that MFE gets "consumed" by each domain is what enrollments are passed to the dashboard MFE (e.g., all enrollments for B2C learners vs. just the enterprise enrollments for B2B learners).

[adamstankiewicz]

A few additional questions for this section:

• What is the right granularity/size for an MFE? When should a larger MFE be broken out into smaller MFEs? This may be partially influenced by how does the container app orchestrates the MFEs (e.g., by URL route, by feature or use case within the domain, etc.).
• Should MFEs be able to define their own nested routes, or is routing solely determined by the container app?
• Can MFEs act as container apps themselves similar to the top-level container app? Bi-directional MFEs (in Webpack's module federation's vernacular, both a "remote" and a "host" at the same time, IIRC).

[adamstankiewicz]

Definitely, routing is a big part of this proposed architecture :)

I think we'll need to consider both the container layer routing as well as any potential routing "local" to individual MFEs. Some micro-frontend frameworks (e.g., "single-spa") provide an API to create top-level routes as a way to define how the MFEs should be integrated (by route).

[adamstankiewicz]

By moving to this architecture, we would inherently be moving more towards a world with lazy loading via "code splitting" 😄 Where only the pieces/code of the application necessary are loaded by the user's browser, as the user interacts with it. This would likely be a pretty noticeable performance improvement, and less bandwidth consumed by the user.

It does have the tradeoff of needing to consider the async nature of it with regard to any necessary loading states for slow networks or having a fallback in place when it fails to load for any reason. A strategy I've read about here is to default to "live sharing" of MFEs, but fallback to an installed/versioned NPM release of that same MFE for resiliency.

It would probably be good for us to also provide some guidance around code splitting for MFE applications, as well (e.g., dynamic imports, React.lazy + Suspense, etc.).

[adamstankiewicz]

Given the other recent concerns regarding our shared header/footer components (e.g., the "invisible coupling" between @edx/frontend-component-header and @edx/frontend-component-header-edx), I wonder if it may be worth doing a "requirements gathering" type exercise for understanding what use cases our shared header/footers need to support throughout the platform. Will the same headers/footers be shared across domains, or would it be assumed each domain essentially defines its own header/footer? What would the benefit be of treating any shared headers/footers as NPM packages vs. MFEs? For example, if they are MFEs, in theory any shared headers could be live loaded in each domain to ensure any shared headers across domains are updated consistently.

[adamstankiewicz]

"Global" data like authentication state might make sense to live in the domain's containing app, which currently comes from @edx/frontend-platform and it's AppProvider component.

In the Enterprise domain, we might consider treating authentication state, the metadata about the enterprise customer (e.g., its name), the subsidies available to the enterprise/authenticated user (e.g., subscription license) as "global" data applicable to all the potential MFEs in this domain.

[adamstankiewicz]

An initial thought here... I believe the majority of the UX elements in the container would be more layout based in terms of deciding when MFEs should be dynamically loaded and where they should go in the UI (e.g., there could be more than 1 MFE to dynamically load for a given route in the container app, and the container app decides the layout for displaying each, like a main content region and a sidebar region, for example).

[adamstankiewicz]

I think we'll want to deep dive a bit in terms of how version conflicts and breaking changes should be handled in this architecture (e.g., is it smart enough to fallback to older version for libraries in some MFEs, but not others?).

[ghassanmas]

My take/opinion on this, is that before we jump staright away to module federation, is that we try to load/run an MFE while providing it's depeceny at runtime. Here is what I exactly mean:

1. Tweak webpack config in candidate MFE to not bundle in the what we agree is the bare minumum of shared/common dependencies. i.e. react and its frineds, paragon, frontend-*-component
2. Generate a sepreate bundle/module for those that are excluded from step 1.
3. Mutate the html page to incldue assets from step 2 to the html of page 1. (i.e inject link and scritps tags)

My reasnoning behind this, is that of all the suggested path here, the common pattern is that when we are going bundle/build the MFE, we are not going to include the common depends and would provide those at runtime. This change might sound trivial, we just ensure the shared depends are loaded before the MFE logic starts, but there is more to that, for example how to ensure the exported moudle type from step 2. is what mfe/webpack at runtime expects. What about shared depencies that are not js modules, i.e fonts, icons, css/scss...

I think exploring that scenario would lead us to answer question or/and gather inforamtion about our current eco-system, upfront and shall help us do categorize/charactierse each depencey.

[adamstankiewicz]

@edx/paragon? I would imagine having a single Paragon version should lead to a more consistent UX in terms of design and functionality across the domain.

[ghassanmas]

+1 If we shared a single version of paragon across all MFEs, then do we by deafault load all of paragon components?, which the current target MFE might not use all of them. This is probably can be ignored if the cost of fetching all compoenets is low., I am just speculating...

[adamstankiewicz]

The Paragon library supports tree shaking, so it only includes the necessary code for the components an MFE is actually using, which should mitigate this concern. As far as I can tell, I don't believe that existing tree shaking behavior would be impacted even with a single copy of Paragon shared across multiple MFEs in a domain app.

[ghassanmas]

Aha but what about the case where the MFE is expected to use a shared version of paragon, which I expect suppose to be available at runtime? Below is a scinario of how I imagine but I might be wrong

• Paragon has compoenets: (A, B, C, D, E, F, G,H)
• MFE-1: depends on (A, B, C E, F)
• MFE-2 depends on (C, D, G).

The de-facto situation wihhout having the host app...etc is that webapck suppose to to do tree shakeing when building MFE-1 and MFE-2, such that they will denepd on a version of paragon that has a subset of paragon compoenents.

However in the host/container app siutation, then we are thinking we have a copy of paragon for all other MFEs, so we don't load paragon everytime a user change MFE? If yes then which componenet do we initially include in the host app, my top of head suggestion can be:

1. Just include all of them, if the added complexity doesn't worth the saved bandwidth...
2. Include an Union of all MFEs paragon compoenets i.e. $MFE1 \cup MFE2... \cup MFEn = PU$
   • The more MFEs we have the closet $PU$ gets to $P$ where $P$ is a set of all the compoenet
3. Incldue an Intersection of all MFEs paragon compoenets i.e $MFE1 \cap MFE2 ...\cap MFEn = PI$
   • The more MFEs we have then probably the less compoenents $PI$ would contain.
   • This path of course would add a bit complexity beacus loading a particualr MFE, then we need to load the difference in compoenets, i.e $MFE1 - PI$ _The component that are in $MFE1$ but not in $PI$
     • One other example is load each compoenet just when neede, but that of course cause that for each componenet in the MFE that doesn't exists in the third set, we to have a have roundtrip to the server

[adamstankiewicz]

@edx/frontend-platform?

[adamstankiewicz]

Also related to @edx/frontend-platform... It'll be interesting to consider how our current MFE bootstrapping / initialization strategy in @edx/frontend-platform will need to change, if at all, in order to support this architecture.

For example, each MFE currently utilizes AppProvider, which includes an AppContext.Provider, allowing children of AppProvider to read its context value, e.g. authenticatedUser, config, etc. which feels like data that would make sense at the domain app layer (i.e., more global, do it once), not the individual MFE layer, in an orchestrated domain app.

[adamstankiewicz]

Our aliased @edx/brand NPM packages? I wonder how the NPM aliasing we do today in order to support an Open edX vs. edX.org specific theme for Paragon would be impacted, if at all. For example, we typically install @edx/brand-openedx (aliased to @edx/brand) in the repository itself, and then in the GoCD build for stage/product, we re-install @edx/brand but aliased to @edx/brand-edx.org instead.

Would both @edx/brand-openedx and @edx/brand-edx.org have to be configured as a shared dependency in the Webpack config for module federation or would sharing the alias @edx/brand suffice, for example? How would these aliased NPM packages be configured in an open-source friendly way with shared dependencies, or would this (fragile) package aliasing at deploy time preclude them from being shared?

[adamstankiewicz]

[question] Should MFEs be able to communicate with each other in any way? Or asked slightly differently, what is the path of communication between the containing/host application and its consumed MFEs? Can an individual MFE only communicate with the parent container app or can it communicate to other MFEs throughout the application as well?

Is there an event system in place (e.g., pub/sub)?

[adamstankiewicz]

[question/curious] Are there any concerns that moving towards this container app architecture will result in more Git repositories to manage as an engineer? If a domain's application can be broken down in more granular pieces (e.g., possibly even multiple MFEs composed together for the same page/route), will we have to manage/maintain/own more Git repositories during local development? Should we be intentional about a poly-repo vs. mono-repo strategy (e.g., perhaps there's a monorepo per domain, how much investment would be needed into sustainable monorepo tooling e.g. Lerna)? Just thinking aloud and stirring the pot a bit ;)

[adamstankiewicz]

Related to another comment, is (or should be) the integration strategy compatible with server-side rendering? Could an MFE be created with server-side rendering (e.g., with Next.js) but still be consumed by a container/host application?

[adamstankiewicz]

Hinted at in another comment, but is there any eventing system like PubSub for communication between MFEs and/or an MFE and its container app as part of the interface?

[adamstankiewicz]

semi-related to integration costs: As a consumer of Open edX, if I want to customize/replace one MFE for another to change/swap functionality in the container app or run an experiment, would that be supported? E.g., using Webpack 5 module federation, could I change the path to the remoteEntry.js file to point to a different MFE instead?

[adamstankiewicz]

If an exposed component from an MFE changes its API but doesn't inform its consumers, how should/would errors be handled (assumes the consuming app is pulling a "live" version, not an explicit SemVer version)? How will this error handling impact the user experience? An example solution I've seen for this issue is to fallback to an installed version known to be compatible; that way, if the upstream components unintentionally creates a breaking change, it may be able to fallback to a known compatible experience? If that's not possible, ideally, the error would prevent the entire application from breaking and instead only break that one part of the application.

[arbrandes]

Status update: I didn't have time to reply to and incorporate comments (thanks for them, btw!) last sprint because of the Olive upgrade work, but tCRIL is moving forward with selecting a contractor to start answering the questions posed.

More on this as work starts on discovery.

[davidjoy]

I think we need to be careful to zero in on what problems we hope MFE modularity (is composability the right word?) will solve. There are a lot of problems listed here as motivation that are legitimate critiques of the architecture, but which I don't believe will be appreciably improved by this effort.

My take is that the main things we hope to get out of this are:

1. Reduced overall bundle size via shared files at runtime
2. Improved page load performance and cross-page UX via loading a 'shell' application once that contains both common UI elements, like headers and footers, and shared files.
3. Prioritizing "consistency" over "flexibility" - this is a choice we made early on in favor of flexibility. We allowed all MFEs to have shared libraries, yes, but to use whatever version of those they wanted. I think we probably went too far, and are learning we care more about consistency.

These correspond roughly to "Changes that affect multiple MFEs are costly" and "Inefficient use of network bandwidth" below.

Things this does not significantly impact:

1. Code reusability - Having a place to put shared code so we can reuse it - we have shared libraries that all MFEs use - Paragon, frontend-build, frontend-platform, the header and footer libraries - if you want to add a plugin interface, or consistent UI elements, we already have places to put those.
2. Documentation - loading our code differently doesn't impact the documentation written for our independent repos.
3. UX design - our design teams coordinate through Paragon to make consistent experiences. A technical solution for code loading won't affect that much. Prioritizing consistency of version numbers will help, yes, but it won't change how designers work on its own.
4. Deprecation of old UIs - I'm not sure how this changes as a result of this.
5. Changes across MFEs that aren't covered by the shared libraries - I'm not convinced this is common... if something is used by many MFEs, it should be in a shared library already (i.e, paragon, frontend-platform, etc.)

Long story short, I'm wondering if we can tighten up this list of motivations so we can prove we're addressing them with the proposed new technical capabilities.

[davidjoy]

I believe @adamstankiewicz mentions this elsewhere, but I think we should seriously consider Next.js or Remix to solve these problems, not just purely client-side solutions. An important critique of our MFE strategy is that we threw out the server completely.

[arbrandes]

Thanks everybody for the input over the lifetime of this draft, but I'm now closing this in favor of #575. It takes into account all the requirements and work done here, but goes in a different technical direction. I suggest folks take a peek. ;)