Writing a 3D scene graph with HTML

Hello Everyone!

Over the weekend I hacked together an initial concept of an HTML interface for my prototype using the Custom Elements API. This lets us write a scene graph using traditional HTML. For example:

<div class="scene-container">
    <motor-scene id="scene">
        <motor-node id="node1" rotation="[0,30,0]">

            <div>Hello HTML!</div>

            <motor-node id="node2" position="[0,0,50]">
                <div>Hello more HTML!</div>
            </motor-node>

        </motor-node>
    </motor-scene>
</div>

These custom elements construct a scene graph behind the scenes using the imperative API.

The advantage of creating custom elements is that we can now manipulate our elements with any library of our liking. For example, we can manipulate the scene graph with jQuery:

$('#node1').attr('rotation', '[0,40,0]')

We can choose to use Angular, React, Mithril, Backbone+Handlebars, or any other DOM manipulation library, and it will just work. For example, here’s the previous HTML example written as a React component:

class Scene extends React.Component {
    render() {
        return (
            <motor-scene id="scene">
                <motor-node ref="node1" rotation="[0,30,0]">

                    <div>Hello HTML!</div>

                    <motor-node id="node2" position="[0,0,50]">
                        <div>Hello more HTML!</div>
                    </motor-node>

                </motor-node>
            </motor-scene>
        )
    }
}

ReactDOM.render(<Scene />, document.querySelector('.scene-container'))

In Meteor, we can simply use the elements in any Blaze template; they’re just normal (custom) DOM elements. I put together a demo. Take a look at the client/main.html file to see the basic markup example. To run the example, run

git clone https://github.com/trusktr/site.git
cd site
git checkout infamous-motor-html
npm install
meteor

the visit localhost:3000. The demo isn’t anything special, it just positions two DOM elements (one is a sub-node of the other in the scene graph). You should see this in your browser:

The downside to the HTML interface is that the custom elements introduce an extra functional layer on top of the imperative API. This extra layer must convert the string attributes of the elements into the actual numerical arrays that the imperative API requires. So, for each change to the rotation property of an element (for example), the HTML interface has an extra function call and string parsing. Depending on the use-case, you might not notice any performance loss (especially with newer devices).

It’s good to keep the aformentioned performance implication in mind. Having the HTML interface makes it easy to, for example, use simple jQuery to animate a node, like so:

let rotation = 0
requestAnimationFrame(function loop() {
    $('#node1').attr('rotation', `[0,${rotation++},0]`)
    requestAnimationFrame(loop)
})

But, as you can see, this involves creating a string, and in turn that string will get parsed into an array of numbers. This will happen on each tick. We can gain a little performance if we avoid jQuery and just use the native document methods:

let rotation = 0
requestAnimationFrame(function loop() {
    document.getElementById('node1').rotation = `[0,${rotation++},0]`
    requestAnimationFrame(loop)
})

To save even more performance, the best recommendation would be to use the HTML markup just for creating the structure of the application, then to use the imperative API directly to animate the elements. For example, the actual Node of the imperative API is exposed on the node property of the DOM element:

let rotation = 0
const actualNode = document.getElementById('node1').node
requestAnimationFrame(function loop() {
    actualNode.rotation = [0,rotation++,0] // we've bypassed the extra function calls and string-parsing
    requestAnimationFrame(loop)
})

If using other libraries like React or Meteor, you’ll have to use the methods in those libraries of getting a reference to the actual DOM element. Building on the previous React example, you can use the ref to get the element:

class Scene extends React.Component {
    componentDidMount() {
        let rotation = 0
        const actualNode = this.refs.node1.node
        requestAnimationFrame(function loop() {
            actualNode.rotation = [0,rotation++,0]
            requestAnimationFrame(loop)
        })
    }

    render() {
        return (
            <motor-scene id="scene">
                <motor-node ref="node1" rotation="[0,30,0]">

                    <div>Hello HTML!</div>

                    <motor-node id="node2" position="[0,0,50]">
                        <div>Hello more HTML!</div>
                    </motor-node>

                </motor-node>
            </motor-scene>
        )
    }
}

ReactDOM.render(<Scene />, document.querySelector('.scene-container'))

This is all very alpha, and I’ve already thought of a bunch of breaking changes that I’ll be making in order to improve performance, improve how easy the API is to use, and improve the experience when inspecting the scene graph in the element inspector.

For now and very temporarily, a requestAnimationFrame loop is started for each <motor-scene>, which renders the whole scene (all nodes), which is bad (as we’ve discussed in other threads, and one of the things Famous 0.5+ aimed to fix), but fine for smaller use cases. I’m working on a new concept to solve this: the properties of the motor-html elements map to identical setters/getters on instances of Node/Scene of the imperative API. Using setters/getters lets us have some control with what happens when a value is set. My plan is to create an update queue that requests an animation frame in order to perform the requested updates in batch. More details when I post an example…

Compared to A-Frame (not taking into account their awesome visual design), motor-html is already better in at least one regard: a <motor-scene> can be placed in any element, but in A-Frame the scene is always rendered into the <body>.

Lastly, I’m currently remaking the UI components in the src folder (PushMenuLayout being the primary example) using my prototype, and in the process of doing so I’m re-using some things from Famous 0.3 like GenericSync, EventHandler. @dmvaldman It’d be awesome if you could write a document on the differences between Famous 0.3 events/syncs and Samsara events/syncs and transitioning from Famous to yours! :]

Oops, I fixed the demo link.

This is a very interesting approach. Have you thought how to synchronize the animation between two nodes?

@dmvaldman If you mean something lime running all animations based on a universal time stamp (from a root rAF loop for example), no, not yet. Right now the rAF loop is just ad-hoc, with no consideration of the timestamp argument that is passed into the loop function. The Curves module uses Tween.js, which does ask for timestamps from Date or Performance, so there’s a little bit of sync there if we share a Tween across animations. I will eventually have to use timestamps once WebGL or workers are added (rAF is coming to workers). My rAF loops currently are very temporary, here and here. As you can see, motor-node is just making it’s own loop, and not using startLoop from motor. This is really temporary. In fact, if you look closely, you’ll notice that every motor-node in motor-html starts it’s own loop and renders it’s whole subtree, which doesn’t scale at all!!! This is temporary, and my plan is to remove the rAF loop from motor-html and manage it in motor, with the aforementioned update queue idea. Part of that plan involves hiding the use of the rAF loop entirely (from the end user). So, ultimately, all one will need to do is update some properties on any Node, and the rest (starting or stopping of a root loop) will happen behind the scenes. The end-user API will consist of Node, Scene, Camera (defaults to the root node if not used), a tween API, and possibly separate DOMElement and WebGLMesh (probably Three.js) classes (and eventually more things like a physics engine, possibly adopted from an existing library). There won’t be a need to Engine.init() or anything like that, it will be automatic, and when nothing moves there will be no loop running at all. There’s still lots to think about on this topic though!

Dang, I wrote that all on my phone. x}

Oh, another thing! We can also run all property updates of the motor-scene/node elements in their own rAF, so that string processing of properties happens between frames, but I haven’t added that yet.

Another benefit of the Custom Elements API is that the HTML markup can be rendered server-side. I’ll be back with more updates soon.

The future is looking good. In Chrome with experimental JavaScript enabled, the following works for defining custom HTML elements:

// CustomElement.js
class CustomElement extends HTMLElement {
  createdCallback() {console.log('created')}
  attachedCallback() {console.log('attached')}
  detachedCallback() {console.log('detached')}
}
CustomElement = document.registerElement('custom-element', CustomElement)
export default CustomElement
// app.js
import CustomElement from './CustomElement'

// use the element imperatively:
document.body.appendChild(new CustomElement())
<!-- or write HTML declaratively: -->
<body>
  <custom-element />
<body>

This is really cool because we can use ES6 Class syntax to define custom elements, then register the element in a higher-order-component style with the registerElement call! Here’s the higher-order component pattern, just in case. I’ve updated motor-html to this pattern. For reference, here’s what the <motor-scene> element definition looks like.