Just thinking out loud (let me know if you have any thoughts about anything):
The command queue could be on the UI thread. Some commands might be synchronous (f.e. handling of a DOM element) and some might be asynchronous (f.e. sending a request to a worker to calculate world transformation matrices or to a worker to calculate physics), and all can be queued using something like async.queue
.
Hmm, but if many of the functions are asynchronous, then they can run in parallel. Perhaps the fastest updating things will simply callback sooner, and those things render sooner (f.e. at 60fps while some other things are slower). There could be a result listener that simply applies whatever results were sent back within the frame. SO itās possible that some slow animations will be slower, and some will be faster depending on when results come back from workers. If such a design is done properly, rendering could always be at 60fps, and only drawing speed appear to be slow depending on the specific animation.
Can each node have itās own worker? Perhaps theyāre a bunch of workers all connect together in the same structure as the scene graph? Would that make sense for a componentās onUpdate to run in itās Nodeās worker? If it needs state from another node for itās calculation (f.e. the size or rotation of another node) it can request it asynchronously and eventually get the result back in order to completely itās calculation. Iām imagining something this might work really nicely when MessageChannels arenāt coupled to the UI-thread like they are now (as described in that Chromium bug you linked us to above @oldschooljarvis).
I think perhaps each Node worker in this design Iām imagining can obviously hold things like itās local state and world state (local transforms, local opacity, etc, and world/final transforms, final opacity, etc).
Iām imagining that in order to calculate the world transform of some Node in a tree of linked Workers, that the mechanism that will do these calculations (be it in some separate worker for this purpose, or perhaps in a sidekick worker of the Node whoās world-transform weāre calculating) will asynchronously query all Nodes in the path leading to the Node in question to get the local transforms of each node. The results might arrive in any order, but when they all finally arrive, the calculation can be completed, and the result world transform saved back onto the Node in question.
The WebGL renderer, f.e., would need all the world transforms of each Node in order to render them in the 3D world, so that would happen separately in a similar fashion: querying all nodes for world transforms, and applying only transforms that have changed. Thereād be some way to query for transforms of things we expect have been modified.
Maybe a Node can be marked ādirtyā on the UI side whenever a user has called a function to modify some of that Nodeās state, that way the renderer can query that worker once per frame until the Node says itās clean? Iām not entirely clear on this part yet, I have fog here.
But what I am starting to see clearly is that if we separate everything into workers in small pieces (f.e. one Worker per Node) then it seems clearer how things can perform well without blocking the UI thread.
At first I was imagining having a single āSceneWorkerā that contains all of itās worker-side nodes in (worker-side because there is also some UI-side āNodeā that the end user always has a reference to, and which is the userās interface to that āNodeā) and would run all updates in itās own self.
But now that Iām thinking about it, Iām liking the idea of splitting things into one worker per Node, which I think might actually make the design easier to implement because then the UI-side āNodeā that a user interacts with can have methods that simply call methods on the worker-side Node in an RPC-like fashion.
And now, when I think about something like a āTransitionā that performs calculations on any number of Nodes at once, I can imaging that the Transition thing can be itās own worker that queries Nodes for info needed then calculates results to give back to Nodes, which Nodes can then use to update their own local and world transforms.
So hereās an example of possible UI-side end-user API:
let node = new Node // `node` is interface for the end user in the UI-thread, but creates a worker behind the scenes?
let transition = new Transition(0) // Transition might make a new worker behind the scenes.
function someAsyncFunction(callback) { // some async function who's result we'll need in a calculation.
// ...
callback(result)
}
transition.to(2*Math.PI, {
duration: 5000,
curve: Curve.expoInOut,
deps: {
otherPosX: otherNode.position, // transition sees this is a NodeComponent, so gets that component for the following calculator function.
someNumber: someAsyncFunction // transition automatically knows to call someAsyncFunction on the UI-side each tick?
}, // list dependencies needed by the Transform, so that it knows what to query on the worker side?
calculator: function(currentValue, deps) { // calculator runs on the worker-side. ?
return { // return a result object, the results of some (possibly intense) calculations.
rotation: deps.otherPosX * deps.someNumber + currentValue + Math.random()*Math.PI/8
}
},
applier: function(results) { // used to apply results. Receives the result object of the calculator on the UI-side.
node.rotation.y = results.rotation // this tells the UI to send the update to the node's worker after caching it on the UI-side for DOM rendering? We know that at each frame we need to apply cached values, whatever may be.
}
}).loop()
What I was thinking there is that thereās some way to tell the API what it needs in order to perform some calculations worker-side (in this case, guided by a transforming number) and how to apply the result to other parts of the UI-side API.
But, maybe having that transition
in itās own worker is overkill? I initially thought that by making a new transform with new Transform(node)
that it might be a component of the Node that it received in itās constructor, and operate on that Node, but then that seems to make it complicated to do calculations involving state from multiple nodes, so I came up with that deps
idea just now. But the problem with the applier
function is that the result has come back to the UI-thread, and if it needs to be applied to another node, then the result is being sent back into a worker. Itād be nice for the result to go straight to where it needs to go without having to route through the UI (save for messaging limitations, but at least not via end-user API calls). Any ideas?
The overall main point that Iām seeing is that if everything is async on a worker basis, then nothing can block the UI except if the scene graph is just so big that it takes longer than a frame to apply results (i.e. pass transforms to renderers). But calculation wise, nothing will happen on the UI-side (if our API is used properly).
Iām imagining that subtrees in the scene graph could take longer than a frame to update if they involve large physics calculations for example, while other portions of the scene graph tree might update quickly, which could produce the effect of some things having sub-60fps lag, and other things moving smoothly. Thatād be interesting to actually see. I also fear some behavior like that would be funky in, for example, a first person shooter case.
So, thatās what Iām gonna do. Iām gonna try to make absolutely everything asynchronous, and the easy way to make mathematical things asynchronous is obviously doing them in workers.