samsaraJS - first look

[dmvaldman]

Hi all,

Maybe some of you remember me. I used to be the Chief Architect at famo.us, but left the company in August 2014. I was famo.us’ second engineer. When I first came in, I wrote the physics engine, and later was in charge of most of the codebase. After I left, famo.us decided to go in the mixed-mode direction and rewrote the framework.

What you may not know is that long ago I forked famo.us at version 0.3, and have taken it in my own direction since December 2014. There are now over 800 commits as I’ve been quietly restructuring the framework the way I would have liked it to be from the beginning. It’s still far from done, but the core ideas are there.

I’ve only now started to speak publicly about it. The fork is called SamsaraJS. It’s located here

dmvaldman/samsara

The key difference from famo.us 0.3 is that it’s a lot more functional and stream-based. The render tree itself is actually one big stream, where nodes (formerly modifiers) act as stream transforms, and the stream exits in the DOM. It embraces more functional reactive programming principles, so you can write code that looks like

var transform = mouseInput
    .pluck('value')
    .map(function(value){
        return Transform.translate(value); 
    }

var layoutNode = new LayoutNode({transform : transform})

In the above code, transform is a stream, layoutNode is a stream. It’s all streams. They all emit start, update and end events, so hypothetically you can call

    layoutNode.on('start', function(spec){
        //spec = {transform : currentTransform}
    }

though this pattern is only used internally. There are many more differences, and I’m happy to discuss them further. But wanted to give you guys a peek and see what your thoughts are. There’s also a pretty minimal website: www.samsaraJS.org

Dave

[talves]

@dmvaldman

Maybe some of you remember me

Dave, how could we forget who you are. You were one of the first one’s to go infamous!

Welcome! Good to have you here. I am sure everyone will value any insights you are able to provide the community on what we are trying to undertake.

[oldschooljarvis]

Hey,

First, I just wanted to say your effort is really impressive. Glad to have you here.

I’ve been looking into FRP for about a week now as a means to handle concurrency and perhaps parallelism concerns with my rewrite candidate.

It looks like you custom-wrote all of the FRP constructs in samsara. Can you provide any insight on that choice? There’s third-party libraries such as RxJS and Kefir.js, I’m wondering if you’ve evaluated those at all. Personally they seem a bit heavy for my tastes, but I’m not entirely sure yet.

Also, as best I can tell samsara doesn’t appear to use web workers, which would stand to reason considering it’s based on a fork of 0.3. Did you ever consider parallelism via workers at any point?

Thanks.

[dmvaldman]

I have looked at RxJS and Bacon.js. There was a particularly important feature I needed that they couldn’t provide. For merging streams, they wait for all sources to emit an event for them to emit the merged event. In Samsara when you write

new SizeNode({
    size : ...
    margins : ...
});

this is a “merged stream” from two streams: size and margins. Both of them don’t need to be changing at the same time. It may be that only one is changing, and so I needed to come up with a way to still fire a change event on the merged stream (SizeNode). This involves a batching strategy tied to the request animation frame. Unfortunately this is a break to the modularity of the code, because it couples the FRP stuff to the raf cycle. I haven’t yet figured a way around this though. Maybe someone here has some good ideas.

I haven’t yet experimented with web workers. I think it should be relatively straightforward, as there is a clear separation between the purely JS parts and the parts that touch the DOM. Though honestly the JS execution time is quite small. Unlike famous, where the render tree was constructed from scratch each frame, because of the stream nature of samsara, only the subtrees where a change has occurred needs to be recomputed. There is only one way I’ve thought of to make it faster (and indeed optimal), which involves some aggressive caching. But with this method you can actually say that if N objects on the screen are moving, at most N matrix multiplications are happening, which is pretty cool.

[oldschooljarvis]

@dmvaldman

This involves a batching strategy tied to the request animation frame. Unfortunately this is a break to the modularity of the code, because it couples the FRP stuff to the raf cycle. I haven’t yet figured a way around this though. Maybe someone here has some good ideas.

I was watching a presentation on ES6/7 async features and someone had a question that was along similar lines:

https://www.youtube.com/watch?v=DqMFX91ToLw#t=60m (60:00 to 61:01)

Not sure if that’s any use for you, but figured I’d mention it.

@dmvaldman

Unlike famous, where the render tree was constructed from scratch each frame, because of the stream nature of samsara, only the subtrees where a change has occurred needs to be recomputed.

Nice. While it’s still possible to achieve aggressive caching with a non-FRP approach, I’m sure it’d result in a bunch of messy state everywhere (e.g. dirty flags).

[trusktr]

@dmvaldman

But with this method you can actually say that if N objects on the screen are moving, at most N matrix multiplications are happening, which is pretty cool.

Are renderable nodes can have child nodes (i.e. they aren’t limited to leaf nodes) then we’ll have the sub trees to recalculate for WebGL but we can let nested DOM cache subtree transforms.

[trusktr]

@oldschooljarvis

I’m sure it’d result in a bunch of messy state everywhere (e.g. dirty flags).

Why not just have a Node communicate to the Engine and tell it when it’s “dirty” and only in that case traverse to that Node (and it’s subtree if renderables have children)?

[trusktr]

@dmvaldman Nice to have your support! Would you like to join us in the merge when we put our best ideas from our prototypes into a single project? Feel free to move your project to github.com/infamous if you’d like to (you’ve got access to the org (check email for the invitation)).

[gadicc]

Hey @dmvaldman!

Of course we remember you Very happy to have you here with us in the forums, and thanks for this awesome update… had no idea something like this was in the works. The stream based approach is super cool!

Having said that, you’re in a very different place from the rest of us… 9 months of work on an existing code base, vs from-scratch rewrites of a few weeks. Are you interested in collaborating with us? What kind of collaboration did you have in mind?

As @trusktr said, what’s happening now is that we’re each working on our own engine candidate. I know it kind of sounds crazy at first, but instead of theoretical chats about what will work best, we’re trying to each get a real / practical feel about what’s involved and generate a few different ways of solving the same problem to see what works best. And then ultimately, combine all our efforts into a single code base, a long these lines:

What is the merge game plan?

Just want to make sure we’re all on the same page. How is it going with everyone? Should we decide on a deadline and how we plan to handle the merge? e.g. (modify this as needed) Current plan is as follows (pending additional team member input): Finish Engine prototypes (until roughly end of November) Finalize Common Design spec (can start before part 1 is completely finished) Merge, as follows: Evaluate each of the following aspects of each engine: Components Events according to thes…

I’m guessing you’d probably want to see some finished candidates before committing to anything, after so much of your own work. (Do you want to share any of your existing plans for samsaraJS?).

Two differences I’d immediately note between your current code base and what we have planned, is:

1. Web workers - as @oldschooljarvis mentioned - for simultaneous calculations and/or just to keep the UI thread more free in general and specifically for super slow DOM updates which of course can only happen there.

2. WebGL - we discussed in some other threads here. The general feeling was that we quite liked the idea of mixed mode… and that most devs aren’t interested in it only because it’s a chore, but with low barriers to entry it’s just cheap awesome. Also a few of us felt that it was a 2nd priority, but an important one… i.e. after “a DOM model that feels native everywhere”, but definitely something we wanted to be able to accomodate, probably by leveraging ThreeJS as part of an optional plugin.

Lastly, any input you have about anything is very welcome

[trusktr]

@dmvaldman In Famous oh-three there was an issue where even if the user removed a node from the scene graph and lost all references to it, the Engine would still keep a reference to it, so it’d essentially be a memory leak for the end user.

Have you addressed this issue? If not, I guess that a simple solution would be to use WeakMap wherever it is that Engine is keeping the references, so that, when the end user loses all references, the Node is deleted from the WeakMap automatically.

[dmvaldman]

Yes this issue is addressed, though other issues of removing nodes aren’t It’s what I’m currently working on.

In famous 0.3 there was a list of “entities”. This is taken from the entity/component architecture of game engines. Nothing was removed from this list. This was not ideal, but it was intentional, because you could always add a Surface back in, even if all references to it were lost.

[dmvaldman]

Looking forward to seeing your merge strategy Looking through the projects, they’re all quite different. Though I think the “explore first” attitude is a good one.

For Samsara, I don’t plan on including any WebGL support. Looking forward to seeing what you are planning there.

[aknip]

Hi @dmvaldman ,
I started to explore samsaraJS - this looks very promising!
Do you have a code snippet / example how to connect a ‘resize-Stream’ (or something like that) to a LayoutNode/Transform.translate ?
The demo for connecting GestureInput-streams to the LayoutNode work very well, but I want to resize/translate a surface when the browser window size changes.
Thanks in advance!
Ansgar

[dmvaldman]

Yep!

var size = new ResizeStream();

var layout = new LayoutNode({
    transform : size.map(function(size){
        //size = [100,100]
        return Transform.translate(size);
    })
})

size.emit('resize', [100,100]);

All SceneGraph (to be renamed RenderTree) nodes and Views have a .size property (which is an instance of a RenderStream) built in, so it’s rare to need to create your own. That way if your layout is dependent on knowing your containing size the components are already there for you.

Here we are taking a single source of input (size) and mapping it to a transform. If you are interested in taking several sources of input before returning a transform, you’ll need to use Stream.lift. Which has the API

Stream.lift(function(data1,data2,...){
    ...
}, [stream1, stream2, ...]);

Hope this helps. I’ll make a google group soon, I think that will be a better place for questions.

[aknip]

Thanks for the quick reply - that helps a lot!
But I am afraid my question was a little bit misleading: I was looking for a browser-windowsize-stream as an input for transform.
Usecase: Window size changes => Position of surface changes (dependent on window size).
Thanks!
(I’ll wait with further discussion until your google group will open)

[dmvaldman]

Yep, you can do that too. Unfortunately I should expose a small change to make this easier (expose Engine.size or Context.size), but for now if you extend a View it will have a .size property. If you don’t specify a size it will default to the parent size, which is this case will be the entire screen. So…

//kinda like Backbone.View.extend
var MyView = View.extend({
    initialize : function(){
        //this.size is implicitly created in a View
        var layout = new LayoutNode({
            transform : this.size.map(function(size){
                return Transform.translate(size);
            })
        })
        this.add(layout).add(more stuff)
    }
});

var myView = new MyView();
var context = Engine.createContext();
context.add(myView);

Also, Google Group created! https://groups.google.com/forum/#!forum/samsarajs

[dmvaldman]

Pushed up a change (on master) for Engine to have a size property, so now it’s just

var layout = new LayoutNode({
    transform : Engine.size.map(function(size){
        return Transform.translate(size);
    })
})

[aknip]

Works like a charm - thanks a lot!

[dmvaldman]

Been a little bit of time since I’ve talked more about samsaraJS. Check out the examples! http://samsarajs.org/examples

[bardu]

Very impressive!
Just wish you get more support… Do you have any timeline for a BETA version suitable for developing more serious projects?