-
Notifications
You must be signed in to change notification settings - Fork 780
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Keyed VDom #141
Keyed VDom #141
Conversation
Codecov Report
@@ Coverage Diff @@
## master #141 +/- ##
==========================================
- Coverage 100% 94.32% -5.68%
==========================================
Files 4 4
Lines 160 194 +34
Branches 39 47 +8
==========================================
+ Hits 160 183 +23
- Misses 0 11 +11
Continue to review full report at Codecov.
|
Solves #117 (among many other important use-cases) Here is the code pen from that issue, demonstrating how it solves the problem. http://codepen.io/zaceno/pen/dvOmNr (wierdly though, something makes it break in safari, although it works in chrome. Need to look more at that before mergeing) |
Here is a nother, simpler problem this PR solves. http://codepen.io/zaceno/pen/ryWddK To demonstrate the problem, click the update button and notice how the images reload. |
keyed nodes solve many problems, broadly summarized here: |
It is definitely something that will be needed in some form or another for long term performance reasons. I haven't looked at the implementation, but the first thing that jumps out at me is why the leading |
keys rarely help performance, and if implemented inefficiently can reduce performance. the overhead of internally searching & managing keys usually offsets the gains of having more optimal dom ops. the point of keys is to prevent unintended dom node reuse and accidental busting of any intermittent dom state that's not explicitly managed by the vdom lib. the |
Great, thank you for the explanation @leeoniya |
@zaceno In your example what is EDIT: Okay, I see it's a Number. |
@leeoniya Could you review this PR? 🙏 |
i can sanity check it tomorrow. keep in mind though, an optimal keyed algo is thoroughly described here: https://github.com/localvoid/kivi/blob/master/lib/vnode.ts#L1284 |
@jbucaran: in my example model.current and model.next are strings with URLs |
Amen to some review on this! At this stage it's mostly a proof of concept that it's doable. But certainly needs some community-scrutiny |
src/app.js
Outdated
|
||
if (child && child.data && child.data._key && keyedChildren[child.data._key]) { | ||
var keyed = keyedChildren[child.data._key] | ||
element.appendChild(keyed.element) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably more effincient in most cases, if this line were: element.insertBefore(keyed.element, element.childNodes[i])
There is an issue with #141 and css transitions. It seems that the way we remove and reinsert keyed elements in the dom somehow breaks css-transitions in Chrome and Safari. In order to make my sliding images demo work, I have to add classes to the element after the patch function is run, i e in oncreate and onupdate lifecycle methods. http://codepen.io/zaceno/pen/dvOmNr ... not sure if there is something we should do about that in the patch-function, or just call it a browser-bug and accept the workaround. EDIT: to be clear: the patch function adds classes correctly when you just specify them in the data object normally. There is no regression in this regard. It's just that whatever in the browser that reacts to class-changes and makes the transition happen, looses tracking once we remove and reinsert the DOMNode. As far as I can tell. UPDATE 2: I investigated how to solve this, so that we kan do transitions with keyed elements without the workaround in the above pen. It is possible, but I didn't add it to the PR because I don't believe anyone will like it. Basically it involves deferring the All that, just to work around what seems like quite arbitrary browser behavior (I'd even go so far as to call it a bug)!
|
quick update, because i've been somewhat busy. i want to be clear about what i planned check for this PR, and if anyone gets to this first, go ahead and do it. for obvious reasons, i don't have much interest in dedicating a lot of time to hyperapp. if i spot anything obvious that i've learned to avoid, i will certainly point it out, but i will not be digging into the source here to validate that all things are coded optimally and without errors - that job is painstakingly huge, and i've already gone through it all in domvm via several vdom engine rewrites. that being said, here's an easy way to figure out whether this PR does what it's supposed to from external testing alone:
if the above tests all pass, then i'd give this impl a thumbs up. I don't have a timeline for when i can get around to this, life is quite hectic these days, sorry. [1] https://github.com/leeoniya/domvm/blob/2.x-dev/test/dominstr.js |
@leeoniya thanks for that excellent suite of tests. I couldn't help myself but started checking my implementation against the suite you recommended (even though I know self-verification is not real verification) -- and it did uncover bugs in the implementation that I will start looking at now. So there's no rush for anyone to verify this just yet (unless you're curious what the bugs were, and interested in fixing them, of course ;) ) |
that would imply that developers cannot be trusted to write unit tests for their own code 😆 |
Ok, so the problem was the To "prove" it, here's a code-pen that cycles through the tests. One each second so that you can follow them visually. http://codepen.io/zaceno/pen/VppvBg?editors=0011 If at any point the element.style.color doesn't match the element.textContent, the console should shout at you. Fortunately it doesn't :) ... Again, I prove nothing by verifying my own code of course. But consider it a green light to proceed with more verification & reviewing. |
@leeoniya ... hehe, you make a good point. But then I at least hope someone looks at my tests ;) |
no implementation is bug free. but code that passes a non-trivial set of smell tests and has automated regression tests can be merged. if all existing tests pass and no new tests fail, that's the best anyone can do. the only enemy is not enough existing tests - there are never enough. if domvm's tests have even 50% code coverage, i would be shocked - and it has 500+ assertions and thousands LOC worth of testing code. it's a major pain in the ass to write them all, but that's what separates toy MVP projects and production-grade libs you can trust not to break apps between releases and only improve with time. v2 was a major from-scratch rewrite of v1 but the tests ensured that essentially nothing broke [1]. [1] domvm/domvm#116 |
IMHO |
@ngryman -- fine with me. Makes sense considering onremove et c also have no special prefix. |
@jbucaran it made more sense in my head... To allow for keys to work and allowing real elements to be held by reference in the model (my PR), the So, if creating an element from a vnode with a key, also set the |
@AutoSponge Haha okay. I am currently working on my own keys implementation based in #141. Like you say I am not sure if it will be compatible with #151, but on the other hand, I still haven't quite understood what's the purpose of that PR. Don't take that as criticism please, on the contrary, I just need more time or perhaps if you have time you can try ELI5 to me again. |
@zaceno 👋 Is this done or you planning tweaking something more? |
@jbucaran, Hi sorry for the silence :) No, it's not complete. I am working on a rewrite where I do removeChild(...) only when absolutely necessary (to fix the css-transition issue, among other things). It's going to make the diff bigger, but it will be a more complete solution. I didn't get as much time as I had hoped this weekend, but I am working away at it little by little, and hope to have an update to this PR soon. By the end of the week at the latest. UPDATE: It's looking promising. All unit tests -- and all but a couple of @leeoniya:s smoke tests above -- are passing. Just need to tinker a little more to get it all working (+ some polish to make it more presentable) Then I will submit an update. But not tonight... it's late here now :) |
@jbucaran So now I believe this solution is complete. It ended up bigger than I hoped, and perhaps touching more of the code than you would like. But it works. (And because I don't needlessly remove & reinsert dom-nodes any longer, it does not break css transitions.) Here are updated versions of all the code pens I've mentioned before: @leeoniya s suite of smoke tests visualized here: http://codepen.io/zaceno/pen/wJqmjY My css transitions slideshow with images: http://codepen.io/zaceno/pen/WpEJQV My css transitions slideshow with iframes: http://codepen.io/zaceno/pen/oZeqRG the image gallery example: http://codepen.io/zaceno/pen/bqrvQO |
I solved a similar problem in my PR #151. The problem comes when replacing an element with a sibling element instead of just removing the preceding sibling. I simply perform the remove then, remove the reference from the If you want, I can try to add that to my PR and test your original demos against it. I just need your original demo code if you have it (unless I just need to remove the |
@AutoSponge, if I get what you're saying, I think a keyed vdom covers more cases than just the one you're describing, but the case you're describing is the problem I initially had with my iframe/image slidedshow demos. I know my case can be solved with something simpler than keyed vdoms (I had a PR out before this one, that solved it in a more minimal way) -- but I'm absolutely curious to see your solution (I think I'd understand better what you're saying if I saw it anyway), so go for it! :) That said, because keyed vdoms cover more cases (specifically: when you need to reuse but reorder dom elements), I'm hoping for that feature in hyperapp rather than a less complete solution. Hope I understood you correctly! If I didn't, it wasnt intentional :) |
@zaceno I started this work but I'm not quite done. As I worked, I thought I could continue to work on this implementation but don't let me hold up acceptance. My changes in #151 are likely to fold in easily since it's not that big. |
@AutoSponge Cool, thanks for sharing that idea! The thing about keys vs ids is, though, is that while id's must be unique within the entire document, keys only need to be unique among sibling-nodes. And there are cases where you might want to have the same key in multiple places in a document. Like if you have multiple of the same components on a page. By all means keep working, and don't hold your breath on this PR! I hope its not long before we can have keyed patching in hyperapp, but this may not be the implementation that makes it. @jbucaran expressed hesitance on the slack-channel, against both the performance-degradation and increased bundle size this brings with it. So to make this happen I'll need to both optimize and condense it. But frankly, right now I'm not sure I can do any better, so I'm not feeling that hopeful :( Thankfully @jbucaran is working on an implementation of his own, and I hope he fares better :) |
Given the state of this PR, I want to express my purely personal and subjective opinion that: if no better solution (more performant, smaller) can be found in the near term, this PR should be merged anyway. There are several use cases where this functionality (not necessarily my implementation) is absolutely critical. Hyperapp needs this if it wants to be a "general purpose" frontend framework. Despite the shortcomings of my implementation in terms of size and performance, it does work. Once it's in, we can and should continue looking for optimizations. ... that is of course only if no better implementation can be found in the short term. I'm still willing to wait for @jbucaran 's solution (or anyone else currently having a go at it) |
@zaceno Thanks for your work on this PR! I agree with you this is an important feature we need to ship ASAP. I hope I can dedicate some good time to this on the weekend. |
So I made some adjustments and brought back the batch-delete. (thanks @jbucaran for brilliantly reminding me that dom nodes don't have to be detached from the document to be inserted!!) I also tried to refactor it into a hopefully more compressible way. I wasn't able to get the benchmark thing running, so I havent been able to check performance. But hopefully this should be much better in performance, and a little smaller as well. |
src/app.js
Outdated
element = parent.appendChild(createElementFrom(node)) | ||
|
||
} else if (node === undefined) { | ||
function batchRemove (parent, element, node) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno We need to find another way to do this. onRemove should be called right before the node is deleted, so we need to save the onRemove handler in the batch array somehow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't se any problem batching the onRemove-handler, BUT even before this PR, the onRemove handler was deferred, meaning it gets was called after the node is removed. So... do you want me to fix that behavior as well, or leave it as it is?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I pushed it to master a while ago, it's like 10 bytes smaller so...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😆
src/app.js
Outdated
if (oldNode === undefined) { | ||
var n = createElementFrom(newNode) | ||
parent.insertBefore(n, element) | ||
element = n | ||
|
||
} else if ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno This logic changed in a recent commit on master.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you're referring to the "insert app directly into document.body" merge, then yes. This code has been updated specifically to adapt to that (to return the new element of a patch)
src/app.js
Outdated
} | ||
|
||
function patch (parent, element, oldNode, newNode) { | ||
if (oldNode === undefined) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If element is undefined/null, the new node will be appended to parent, which is what we want, but if element is not null, then the new element will be inserted on top of element
. Is this correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that is correct
src/app.js
Outdated
updateElementData(element, node.data, oldNode.data) | ||
var n = createElementFrom(newNode) | ||
parent.replaceChild(n, element) | ||
element = n |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅
src/app.js
Outdated
// We need to index all keyed nodes in oldChildren in order to | ||
// distinguish new keyed nodes from nodes to reuse in newChildren | ||
var keyedChildren = {} | ||
for (var i = 0; i < oldNode.children.length; i++) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would using a no-op branch instead of continue
mean less bytes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try it :)
src/app.js
Outdated
isKeyed(newChild) && | ||
cache[newChild.data.key] | ||
) { | ||
var c = cache[newChild.data.key] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use var i
unless impossible, it compresses better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean var i
compresses better than var c
? Is that because var i
is already used in a lot if places?
Either way, I'll fix that, no problem.
src/app.js
Outdated
|
||
for (var i = 0; i < len || i < oldLen; i++) { | ||
patch(element, element.childNodes[i], oldNode.children[i], node.children[i]) | ||
} else if (oldChild === undefined) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
== null
src/app.js
Outdated
return element | ||
for (var key in cache) { | ||
var unused = cache[key] | ||
batchRemove(element, unused.element, unused.node) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if a node is already scheduled for remove by patch
, we would end up batching the same node twice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure that can't happen. Unkeyed nodes get scheduled for removal as we iterate over them. Keyed nodes are put in the cache, and scheduled for removal after we see we haven't used them.
test/app.test.js
Outdated
}) | ||
}) | ||
|
||
it("fires onremove for ONLY the not-reused keyed nodes", done => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 It's gonna be more work for me later to clean these tests, so please remove them all from this PR. I'll write tests myself later. 🙏
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I'll remove them. But can we wait until it's definitely time to merge? Because they're pretty useful for knowing I don't break stuff when I fix merge-conflicts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of course, of course.
src/app.js
Outdated
patchChild() | ||
|
||
} else { | ||
cache[oldChild.data.key] = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe smaller would be to map keys to an array and store the node at 0 and element at 1?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm... not sure I follow. I'm happy to try it, but could you give me a code-snippet to explain more. What I'm envisioning is not smaller so I'm sure I'm misunderstanding :)
Latest adjustments brings the total compressed bundle size down to 1954 bytes. |
src/app.js
Outdated
@@ -254,93 +254,66 @@ export default function (app) { | |||
function patchChildren(element, oldNode, newNode) { | |||
// We need to index all keyed nodes in oldChildren in order to | |||
// distinguish new keyed nodes from nodes to reuse in newChildren | |||
var keyedChildren = {} | |||
var oldChildren = [] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno Nice!
src/app.js
Outdated
|
||
var len = node.children.length | ||
var oldLen = oldNode.children.length | ||
var isKeyed = function (node) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there should be no var x = function(){..}
. This pattern only adds more bytes.
src/app.js
Outdated
var cache = {} | ||
|
||
while ( | ||
newIndex < newNode.children.length || |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno We should try to access stuff only once. len
, oldLen
, oldNodeData
, etc.
src/app.js
Outdated
patch(element, oldChildren[oldIndex][0], null, newChild) | ||
newIndex++ | ||
|
||
} else if (newChild == null || (!isKeyed(oldChild) && isKeyed(newChild)) ) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno Perhaps you can store the oldKey and newKey in a variable, then just check oldKey != null
to see if a node is keyed or not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried that, but that change actually added 11 bytes. Either way I'll leave the change in the upcoming commit, so that you can see the result and perhaps help me find a shorter way to write it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting! Thanks for letting me know that :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I managed to compensate for those 11 bytes with some other changes (in the upcoming commit)
} | ||
|
||
return element | ||
for (var key in cache) { | ||
batchRemove(element, cache[key][0], cache[key][1]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Noooice!
src/app.js
Outdated
var oldChildren = [] | ||
var oldKeys = {} | ||
for (var i = 0; i < oldNode.children.length; i++) { | ||
oldNode.children[i].data && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno Let's if instead of && here. It adds no value since uglifyjs will do and undo with this code anyway.
src/app.js
Outdated
var len = node.children.length | ||
var oldLen = oldNode.children.length | ||
var isKeyed = function (node) { | ||
return node.data && node.data.key && oldKeys[node.data.key] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno This means keys can't be numbers, specifically the number 0. I don't have an issue with that, but is that what we want?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👎
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually I think we can have numbers. The number 0 as well. Not sure exactly how various browsers handle this, but in v8 at least (just popped open my node console and tested this)
> var a = {}
undefined
> a[0] = 'foo'
'foo'
> a
{'0': 'foo'}
> a[0]
'foo'
Sure, it will cast numbers to strings for the internal representation of the hash keys. But as long as you are not using a mix of numbers and stringed numbers to key your children (and that is a really unlikely scenario), it should work without conflict
To clarify: the problem we still have is that the keys 0
and '0'
(or 42
and '42'
) will be considered identical and result in undefined behavior if used for different sibling nodes. But I think that is an unlikely scenario. Either you are using strings, or you are using only numbers. Probably. If you are using a mix of strings and numbers, then at least your strings are probably not strings of numbers (e g '42'
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@leeoniya Which one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zaceno If node.data.key is the number 0, then the above condition will result in iskeyed returning false.
What I meant was, if we are going to support keys as Number
s, then the smallest check that works is using null
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right, the condition will fail for all falsey values, which is wrong; all strings and numbers are valid keys. it's not a hash issue, which wull stringify all object keys.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Current bundle size: 1935 bytes |
@jbucaran Of course! I've already left a couple of comments and am continuing to review it. As far as code style goes, I haven't got a lot to say since it's your project :). But I am going to pull it down and run it through some tests later tonight. Anyhow until 172 is merged, I'll try to keep 141 in parity. If only for the sake of personal pride (Since you mentioned it would be pulled into the history with 172 on top of it, I don't want my name attached to something half-assed ;) ) Removing the deferred is one such parity-keeping effort. It was a very small effort really -- took all of 5 minutes, so... |
Sometimes it may be useful to reuse dom nodes -- for performance reasons or to preserve browser state on them (for instance, an image loaded via the src attribute). This allows you to instruct the vdom-patcher which DOM-nodes to reuse, through a key attribute on vdom-nodes
Cleaned up the PR. Removed tests (as requested) and squashed to a single commit. |
@zaceno Cool! I benchmarked my implementation and it's not as bad as I thought, 1.72, but I am sure we can do better. Cache the old keys and old element lists might make a difference. Cache everything, we can even memoize toLowerCase. Also, going to try a different approach when removing stuff. It would be great if you could review my PR more and suggest other changes or how to use some of the features from your PR, since they are both pretty similar after all. It's not so much about making it smaller, but how to make it faster now, we can figure out how to make it smaller later. |
@jbucaran Actually I suspect that our attempts to make it smaller have led to poorer performance. I've been hearing a lot of chatter lately about how specific declarative checks like (I'll duplicate this post over on your PR so the discussion can continue there) |
@zaceno 🎉 |
Allow instructing the patch-function what dom nodes to reuse in update, via