-
Notifications
You must be signed in to change notification settings - Fork 455
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dynamic import #5703
Dynamic import #5703
Conversation
229257d
to
326977a
Compare
jscomp/test/Import.res
Outdated
@@ -0,0 +1,8 @@ | |||
let each = Js.import(Belt.List.forEach) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps wrap it in lazy, or add () =>
and check that both ways it work fine.
In terms of: the code generated works, and the load happens on use.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just double checking: the current example loads eagerly right?
Actually not sure whether it should. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I'll check after adding a test of lazy evaluation with () =>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've fixed the lazy evalutation e864128
I'm a little afraid to arg drilling |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't looked at the implementation yet, except trying examples, which seem to be working well and running fine from node
.
Added comments for the module-level mechanism.
jscomp/test/Import.res
Outdated
let _ = list{1, 2, 3}->eachIntAsync(n => Js.log2("async", n)) | ||
|
||
module type BeltList = module type of Belt.List | ||
let beltAsModule = Js.import(module(Belt.List: BeltList)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mattdamon108
Since this encoding is going to be painful to use, as discussed, what is an alternative encoding to represent:
module M = await import(Belt.List)
How about this:
module M = @res.await Belt.List
that's already the way normal await
for values is currently represented internally.
Actual syntax sugar to be added later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. I lost some of the context from the discussion. It's been a while to me 😃
module M = @res.await Belt.List
It makes sense to me. I'll fix it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding the module mechanism
module M = @res.await Belt.List // 1
let each = M.forEach // 2
What would the js output be?
let each = import("...").then(m => m.forEach)
If so, how about without expression 2?
let m = import("...") // ??
Without the expression 2, only module binding generating the value seems a little weird to me. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can dynamic import without the module mechanism:
let default = Js.import(Comp.default)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding the module mechanism
module M = @res.await Belt.List // 1 let each = M.forEach // 2What would the js output be?
let each = import("...").then(m => m.forEach)If so, how about without expression 2?
let m = import("...") // ??Without the expression 2, only module binding generating the value seems a little weird to me. What do you think?
Line 2 is another line. The mechanism should operate only on line 1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cristianoc ⬆️ Is it what you intended?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cristianoc ⬆️ Is it what you intended?
That looks great. With capitalised "M" I guess.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a syntax question. module type of X
is not available in inlined way?
module M = unpack(@res.await (Js.import(module(Belt.List: module type of Belt.List))))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
module type BeltList = module type of Belt.List
module M = unpack(@res.await (Js.import(module(Belt.List: BeltList))))
var M = await import("../../lib/js/belt_List.js");
var each = M.forEach;
This works fine, but the inlined module type of Belt.List
would be better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like it can't be inlined, but not clear why.
Or maybe neither? Not looked too much at the implementation so far, until the feature is complete for modules too. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Started to look at the implementation code, just a bit.
In general, I'm now wondering what range of cases are supported, and what should happen when unsupported cases are used. Presumably some kind of error.
For example:
let three = Js.import(3)
or
module M = { let x = 10 }
let i = Js.import(M.x)
jscomp/core/lam_compile_primitive.ml
Outdated
match args with | ||
| [ e ] -> ( | ||
match e.expression_desc with | ||
| _ -> ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unnecessary pattern match
jscomp/core/lam_compile_primitive.ml
Outdated
match module_value with | ||
| Some value -> wrap_then (import_of_path path) value | ||
| None -> import_of_path path) | ||
| None -> assert false)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is None
possible? If so what should happen?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not possible, I will clean up
jscomp/core/lam_compile_primitive.ml
Outdated
let module_id, module_value = | ||
match module_of_expression e.expression_desc with | ||
| [ module_ ] -> module_ | ||
| _ -> assert false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's an example where this happens?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One of example would be let three = Js.import(3)
. It asserts false.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this could be a dedicated error message about import instead.
jscomp/core/lam_compile_primitive.ml
Outdated
Js_packages_info.map packages_info (fun { module_system } -> module_system) | ||
in | ||
match module_systems with | ||
| [] -> assert false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at the code, there's test mode where the module system is empty. Not sure what this implies in practice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, when I print the dump lambda IR, the module system seems empty.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, when bsc
is called directly.
How about making it more robust, by e.g. choosing a default if not found. Say es6.
jscomp/core/lam_compile_primitive.ml
Outdated
(args : J.expression list) : J.expression = | ||
let rec module_of_expression = function | ||
| J.Var (J.Qualified (module_id, value)) -> [ (module_id, value) ] | ||
| J.Caml_block (exprs, _, _, _) -> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What set of cases is this match expected to cover?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need anymore. It was for the packaged module module(Belt.List)
. Now the packaged module will be unpacked.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this is the place to give an error if the pattern is not recognised.
There are other places in this file where a warning is issued. Hopefully an error can be issued too.
It seems impossible to prevent errors by syntactic restrictions, as @res.await Belt.List
is OK normally, but an error if there's a local module Belt
.
I've implemented the module mechanism. // original
module M = @res.await Belt.List
// transformed to
module type BeltList0 = module type of Belt.List // 1
module M = unpack(@res.await Js.import(module(Belt.List: BeltList0))) I've looked into // output
var M = await import("../../lib/js/belt_List.js"); |
Can you print a dump of the lambda IR of the code generated by "module M = @res.async Belt.List"? Still hoping that 90% of this can be avoided. |
|
Can that same code be generated directly instead? |
Sorry, I don't get your point. What do you mean generating the same code directly? |
I was hoping one could use a more lightweight way to compile modules rather than generate the pack/unpack mode, like the compilation for value is. But it looks like compilation for modules is not so easy and unclear how to change to obtain the desired lambda code directly. So just need to make sure that the generated code does not lead to surprising errors, such as repeated definition of the same module type (I think that's take care of automatically already) or other forms of errors. |
For example this currently gives a type error: module type BeltList1 = {}
module N = @res.await Belt.List |
Yes. I was exploring how to make a desirable lambda directly without pack/unpack, but no gain so far. Not clear with it. |
For quick fix for this case, I think this would be working. module M = {
module type BeltList0 = module type of Belt.List
include unpack(await Js.import(module(Belt.List: BeltList0)))
} |
Not a good idea, using |
This PR gets too far from the latest, so I've made code diffs based on the latest. |
Here's what I've explored to seek how to implement the module mechanism. module M = @res.await Belt.List
Currenlty, I think generating module type name safely as possible to avoid a surprising error such as |
Yes looks like getting something that works consistently is the way to go. The use of temporary module aliases is a bit indirect but the generated code seems fine. As long as unintended aliasing of the name is avoided, it should be good. |
Looking forward to this feature. Any update? |
Rebase on master is done. |
jscomp/core/lam_compile.mli
Outdated
@@ -25,6 +25,6 @@ | |||
(** Compile single lambda IR to JS IR *) | |||
|
|||
val compile_recursive_lets : | |||
Lam_compile_context.t -> (Ident.t * Lam.t) list -> Js_output.t | |||
output_prefix:string -> Js_packages_info.module_system -> Lam_compile_context.t -> (Ident.t * Lam.t) list -> Js_output.t |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
About invasive:
This and the function below are the only 2 entry points where the 2 new parameters are passed in.
No need to thread them though each recursive call if they never change and are passed from the outside.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once the number of changes is reduced, it should be possible to just isolate what change is responsible.
E.g. perhaps first remove Pimport, then remove gradually every other change in turn, locally, until the tests stops failing and one finds out the root cause.
jscomp/core/js_implementation.ml
Outdated
print_if_pipe ppf Clflags.dump_rawlambda Printlambda.lambda lambda | ||
|> Lam_compile_main.compile outputprefix exports | ||
|> Lam_compile_main.compile outputprefix module_system exports |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is the root cause breaking the tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems delaying the evaluation of js_program causing the test failure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See how it goes #6191
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bingo.
jscomp/core/js_implementation.ml
Outdated
@@ -166,7 +166,7 @@ let after_parsing_impl ppf outputprefix (ast : Parsetree.structure) = | |||
in | |||
let js_program = | |||
print_if_pipe ppf Clflags.dump_rawlambda Printlambda.lambda lambda | |||
|> Lam_compile_main.compile outputprefix exports | |||
|> Lam_compile_main.compile outputprefix NodeJS exports |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about passing NodeJS as always here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That does not sound right. Unless, this is only used by the code you have added?
Can you explain the context more here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can try using Js_packages_info.t
inside lam_compile_primitive
directly where the module_system needs to know.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That does not sound right. Unless, this is only used by the code you have added? Can you explain the context more here?
Pimport needs to know about the information what module_system is. That's why the argument drilling was all over the place in lam along with output_prefix. We can leave the output_prefix but module_system.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would help to first reduce the number of lines changes. As right now it's really hard to see where this is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Eg. instead of
let rec foo x y ... and bar x y ... and ...
do
let toplevel x y =
let rec foo ... and bar ...
So most of the change becomes only indentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeap, let me try experiments to get module_system from Js_packages_info inside Pimport.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeap, let me try experiments to get module_system from Js_packages_info inside Pimport.
Yes that would clean all of this, if it works.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Side comment: The previous design did not require a module_system when writing a js_program with lam because it only needed a module_system to represent the import(top of file) and export(bottom of file) of the js output. However, the dynamic import feature requires a module_system in the js_program as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Side side comment. I wonder for how long we'll need to support all these module systems. It would be nice to converge to es6 at some point.
- get module_system inside compiling Pimport primitive
Can't make reducing enough, because |
Syntax surface, parser for |
let get_module_system () = | ||
let package_info = Js_packages_state.get_packages_info () in | ||
let module_system = | ||
if Js_packages_info.is_empty package_info && !Js_config.js_stdout then | ||
[Js_packages_info.NodeJS] | ||
else Js_packages_info.map package_info (fun {module_system} -> module_system) | ||
in | ||
match module_system with | ||
| [module_system] -> module_system | ||
| _ -> NodeJS |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure in what cases the module_system in Js_packages_info can be multiple in one runtime. For now, I've handled the case where there is only one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we need some tests on fairly large projects.
Ah.. I got your point now 😄 2b69527 |
Anything left open in this PR? |
There's nothing left at the moment. |
@mununki looks like the changelog was missing |
Oops, It's been so long since I've worked on it that I forgot again. I'll PR it right away. |
Introduce the dynamic import (see forum RFC, discussion)
In this guide, we'll explain how to use the dynamic import for module and module value. This new feature allows you to express the dynamic import syntax in Javascript in ReScript without having to be concerned about the actual file names and paths. We'll go through several examples to help you to understand how to use this feature.
compiled to
compiled to
In cases where React components need to be lazy-loaded, you can use the following approach:
Parsetree
Lambda
.then(m => m.value)
for dynamic import a value into generated js output.