makeDisplay efficiently with Hadoop

[jrounds]

Hi Ryan,

makeDisplay with a Hadoop back-end starts a MapReduce job. I have been wondering about if that is always needed and if there is anyway around that.

I concluded certainly for cogFn a MapReduce needs to be run-once, but most of the time I just want to update a panelFn. Is there anyway to just change out the panelFn render code in an existing display or do that in such a way that another MapReduce is not needed?

Thanks,
Jeremiah

[hafen]

That's a good question. You're correct that the only major purpose for the MapReduce job is for the cognostics to be computed.

There is a non-exported function to update other aspects of a display, such as the description, width, height, and panel function. It is available as trelliscope:::updateDisplay. If you look at the code you will see it's almost there - just need to ensure in the case of a panel function that all global variables are captured, etc.

If this is something you need right away let me know and I'll revisit it. Or if you have a little time and want to finish out the function, PRs are always accepted :)!

[jrounds]

Don't need right away (or at all). I was just making sure this wasn't one of those little things you already thought about and had a work around for. It only exist in my head for debugging/iterating. Also because I had a random conversation about if "makeDisplay" was slow and what made it slow. I came to the conclusion it was the cogFn that was the necessary work, and then I sat down at the computer updated a panelFn and thought, "hmm...", but talking it out this is another example where making a local ddo for making things pretty is the solution.

[jrounds]

I am thinking about paying more attention to this.

I am sharing a cluster and I realized this issue of unnecessarily re-calculating cognostics for each panel essentially makes Trelliscope less friendly in the multi-user Hadoop environment. It is more than just a convenience issue.

[hafen]

Can you take a look at updateDisplay and see if it is close to addressing this need now?