modelX/Y/Z() and screenX/Y/Z() for p5.js (WebGL)

[mkmori]

How would this new feature help increase access to p5.js?

This feature set would increase access by improving feature parity between Java P5 and p5.js. Js seems more accessible than Java these days, (esp. on the Web), and both are learning tools, so I feel, where possible, equivalent functionality would be desirable.

I also believe 3D is a core Processing.org feature, offering a uniquely low bar to entry.

The alternative to these "matrix access" functions seems to be direct matrix manipulation and/or access to WebGL. In my experience so far, these both add steep learning curves to what may be accomplished in Java P5 quite intuitively using push()/pop(), transformation, and these "matrix access" functions.

From the history of related requests, besides limiting common access to a well-implemented feature set, I feel the lack of easier "matrix access" functions also poses a risk to learners (like me!) getting frustrated trying to integrate unmaintained, external dependencies, or cooking up their own creative & idiosyncratic, but also potentially limiting theories & workarounds , (as valuable as such frustrations may be).

Most appropriate sub-area of p5.js?

• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Other (specify if possible)

New feature details:

This is a request to implement, specifically for WEBGL mode, modelX(), modelY(), & modelZ(), and screenX(), screenY(), & screenZ().

I've collected all the related requests I could find below, but most of them are P2D-specific and seem to be precluded by canvas-specific issues. (Maybe a future WebGL-implementation of 2D mode would remove that obstacle, and offer users a common 2D/3D code and visualization space...?)

As a compromise/alternative to implementation of modelX/Y/Z() (departing a little from Java P5 convenience), if "let gl = getGlMatrix();" were a thing in p5.js, could that be used à la applyMatrix( gl.a, gl.b, gl.c, gl.d., gl.e., gl.f., gl.g, gl.h, gl.i, gl.j, gl.k, gl.l, gl.m, gl.n, gl.o, gl.p ) to reproduce WebGL transformations outside of push()/pop()? applyMatrix() seems to require matrix elements as individual arguments...so maybe a separate array? mat4? version "applyGlMatrix( gl )" would be nicer? (Something like this seems to exist in the p5.Matrix functions.)

Is there already an easy way to grab the current 3D transformation matrix, (p5.Matrix.mat4 or something)? I feel this would be less accessible and intuitive than modelX/Y/Z(), but still sufficiently abstracted for a learner like me to get things done.... (Did it, I think! Please see next post....)

With the transformation matrix, (a bit more study, and a few more steps), I gather we should be able to get transformed coordinates just as we would with modelX/Y/Z(), right?

Lastly, there's the "projection matrix", of possible relevance to the screenX/Y/Z() part of this request, I think. I understand from playing with Java P5, that screenZ() returns z-buffer values, for 3d picking, "screen space" effects etc.--a little unpredictable, but still fun....

Out of my depth, but some of this even seems semi-doable for me! If I could figure out how to get the current GL context and transformation matrix...and get help writing code on par for contribution here....

Thanks for considering!

#243 (push()/pop() discussion)
#377 (modelX(), modelY() feature request)
#1553 (screenX() screenY(), screenZ() feature request)
#1873 (modelX(), modelY() feature request)
#4337 (matrix functions...completed!)

[mkmori]

I think I've figured out a partial workaround (for my purposes) for the missing modelX/Y/Z() functions, using the canvas "Model View" matrix uMVMatrix and p5.Matrix functions:

https://editor.p5js.org/mkmori.05/sketches/uoPdThLGU

myCanvas = createCanvas( 100, 100, WEBGL ); //to get canvas context....
//myMvMat = myCanvas.uMVMatrix; //this doesn't seem to work...idk why....
copiedMatrix = myCanvas.uMVMatrix.copy(); //to copy the current "Model View" matrix....
myCanvas.uMVMatrix.set( copiedMatrix ); //to set the current MV matrix to the previously copied MV matrix....

I was hoping a variable could reference myCanvas.uMVMatrix, but that doesn't seem to work...? That's a little cumbersome and this approach relies on p5.Matrix functions not in the "friendly" documentation, but copy() and set() do seem to get the job done. (Other downsides?)

I'm assuming people with matrix math skills could also get transformed coordinates using these matrix objects, so we could almost do away with the modelX/Y/Z() part of this request...? (I'll edit the initial request....)

[opheliagame]

hi @mkmori
I looked at your sketch and I think the reason that line 8 and 13 don't work is because the matrix transformations have not been applied at that point.

When replacing line 43 mvMat = cv.uMVMatrix.copy(); with mvMat = cv.uMVMatrix it seems to work for me as expected.
The difference between both of these is that copy returns a new object while cv.uMVMatrix would be an object reference.
here is a explanatory sketch.

@davepagurek
I was trying to verify this difference by console logging the matrices but doing so seems to make the sketch irresponsive for an unexpectedly long time. Haven't figured out why this is happening.
here is the modified sketch with console.log
I ran the sketch on my local machine and it runs fine so this might be an issue with the editor possibly.

[mkmori]

Thanks, @opheliagame! I tweaked my sketch to replace .copy() (and .set()) with object references....

I'm still a bit confused about how the object references seem to function like value assignments in practice: I found it also works to do cv.uMVMatrix = mvMat instead of cv.uMVMatrix.set(mvMat) in the above sketch....

As for screen space coordinates, here is another sketch, based on work done by @bohnacker & @tcoppex:
https://editor.p5js.org/mkmori.05/sketches/Tgvjq98vH

Thanks again--I am following with interest!