-
Notifications
You must be signed in to change notification settings - Fork 249
Improve collection loading for huge websites #609
Comments
Hi, thanks for sugessting this. The collection used to be in a seperated collection.json file and we used to load it async on initial load but it introduces some caveats. @MoOx I think dumping it to collection.js and loading it in a blocking way would be the best. (Or non-blocking way if you can fix previous problems) @DavidWells i tried phenomic on a REAL 10k pages site. The build time is about 8mins and the generated site is 800MB. |
Well, why did we inject collection to html |
We injected collection for better consitency between client & static build. And for small website, extra http request is not necessary. Obviously for large website it might seems huge, but at the end a single user is supposed to access a single html per session. |
Maybe there could be an option to inline or include collections via:
Larger sites could opt for (in my 1000 page test) Each inlined collection adds With the script tag it would be the single 292kb and the deploy/storage would be sooooo much smaller =) |
That's definitely something we should do. We could even ask the user during the deployment if we want collection to be outside html (if let say collection > 150kb?) and save the pref in package.json for future build? |
This will make larger sites way more viable and cost friendly |
@MoOx I'd like to take a stab at adding this option. Would here: phenomic/src/builder/server.js Lines 137 to 141 in 1a11c7f
|
@DavidWells I think it's best to leave it there. It's the dev server. For static build, you should change this line phenomic/src/static/to-html/url-as-html.js Lines 108 to 111 in 2b44e19
|
You will also have to emit a collection.json file from Phenomic loader in addition to previous additions you both mentionned. |
@MoOx it should be collection.js. Let's the browser load that synchronously for us. |
Btw, we should replace |
I prefer |
Then, More seriously, I will probably use |
this is from React code base right ? I like the idea but we referenced this at many places. I don't want to type this everytime |
I plan to add public methods like |
Great idea ! |
@DavidWells any update? |
@thangngoc89 how do you think we should implement this? Are config options available in this context? phenomic/src/static/to-html/url-as-html.js Lines 108 to 111 in 2b44e19
I can see a couple ways to turn this on:
What do you think? |
This should be a phenomic's config. FYI all configs in Phenomic can be altered via a cli flag or an entry in package.json/phenomic. For adding a new config, see this file https://github.com/MoOx/phenomic/blob/2b44e19bfa095d2a4bb512c122dbb24380a83ef1/src/configurator/definitions.js If you need some validation, add one in validators folder |
As for accessing the config, this func
It can be re-writtten like this: I hope these information are helpful |
Going to be fixed with #925 |
Just tested out a 1000 page site. Generated them with this handy script: https://gist.github.com/DavidWells/a75045c89d6f3d453ef4ab68499550f4
Good news, the build only took like 60 seconds!
The generated site was 296mb in total though. This was primarily due to every page inlining the entire
window.__COLLECTION__
into the html. (https://gist.github.com/DavidWells/675b1b838c44925d6487fff3cc129674)The
window.__COLLECTION__
makes html files weight 300k vs 8kPerhaps there is a way to lazy load in the routing data and stick it inside localstorage or a service worker?
Or build the
window.__COLLECTION__
into a singlecollection.js
file and include that once in the head tag? Then it could be cached and would make all the html files lighter.The text was updated successfully, but these errors were encountered: