Web Worker Tests

I’ve slightly altered the test to lower the verbosity and measure the roundtrip instead. I’m getting the following numbers (Firefox outperforming Chrome!):

Chrome (Mac 45.0.2454.85): 0.21ms
Firefox (Mac 39.0.3): 0.17ms

Note: this is a roundtrip from UI --> Worker --> UI

index.js:

var t = performance.now(),
    samples = 60,
    i = 0,
    n = 0;

console.log('UI load timestamp: ' + t);

var worker1 = new Worker('worker.js');

worker1.onmessage = function (e) {
    i++;
    // Receive and log.
    t = performance.now();
    if (i % samples === 0) {
        console.log('UI --> Worker --> UI: ', (t - e.data.ui), '~', n / samples);
        n = 0;
    } else {
        n += (t - e.data.ui);
    }
};

var rafLoop = function () {
    //Send a message, and receive a reply (once per-frame).
    setTimeout(function () {
        var t = performance.now();
        worker1.postMessage({ui: t});
    }, 0);

    window.requestAnimationFrame(rafLoop);
};

rafLoop();

worker.js:

onmessage = function(e) {
    //Receive and log.
    var t = performance.now();
    e.data.worker = t;
    postMessage(e.data);
    t = null;
}

update:

Using a hacked-together polyfill worker class in the main UI thread gives me the following roundtrip data:
Note: this is without actual workers, just a fake polyfill one.

chrome: 0.013ms
firefox: 0.002ms

making the overhead roughly 0.2ms and 0.17ms

Firefox however will drop to 0.0005ms after a period of 15 seconds… which feels really weird to me. The other way around would make more sense probably.

Next test: doing some actual work inside the workers to see how that influences the behaviours.

update 2:

Adding heavy computation inside the workers just adds to the overall overhead, as expected.

One thing that I took out of this test is that it’s fairly easy to create a polyfill Worker, allowing the user to maybe determine whether to use real workers, or route everything through the polyfill worker for ‘normal’ behaviour.

Using the polyfill in a browser without real workers would add more overhead to the UI thread than without the polyfill. We should use only real workers when supported, and the library should also work without works (can be configurable, and by default tries to use workers if the browser supports it).

Do we even want to support a browser that doesn’t have workers (<IE10)?

In my opinion, we do not want to be supporting any browser that does not support it. Was it you @joe that said we are building for the future? I cannot remember who said it. :smile:

Someone can always fork or shim to support the old browsers if they need to.

1 Like

Yes, yes it was! :smile:

1 Like

Hey @dieserikveelo, sorry for the late reply - haven’t had much time the last couple days.

Nice test. While I had my suspicions that Firefox’s console.log() method was causing the terrible performance in my last test (Test05), your test basically confirmed this was the case.

In the name of purity (and science!), I rewrote a new test to do effectively the same thing as Test05–that being, to send and receive a message each frame, recording the results:

Source:

Live (hosted):
http://project42.xyz/webworkerperftests/test06/

The difference is that this test no longer uses console.log() at all while it’s running the test, and it logs the results to a pair of pre-initialized arrays. After the test is done, it computes the results and dumps to console. While the code is an utter mess, I don’t think it’s possible to get the test itself any leaner as far as performance is concerned.

The setTimeout(fn, 0) hack is no longer needed, though I suspect it’ll still be useful in Firefox for times when the UI thread is busy.

Also, keep in mind this is still just two messages per frame. Next I’ll probably adapt Test04 (maximum message volume) to this format, and see how Firefox copes. Right now, Firefox edges out Chrome for me. I have a feeling Chrome will prevail in real-world scenarios, because in those the UI thread is almost always busy. We’ll see.

@trusktr

Using the polyfill in a browser without real workers would add more overhead to the UI thread than without the polyfill.

I’m not sure this would be the case. The engine is almost assuredly going to run on events anyways. Either way, I think we’ll probably arrive at a nice solution that handles both cases elegantly and in a manner that achieves the highest performance reasonably possible.

@trusktr re: IE10

As far as I’m concerned, IE10 can go see itself into the dumpster.

2 Likes

Since I’ve been slacking on these lately, I figured it was time for a new test.

This one tests inter-worker communication. Here’s what it does in order:

  1. Spin up a couple of workers.
  2. Create a MessageChannel.
  3. Send port1 of the MessageChannel to Worker1, and port2 to Worker2.
  4. Block the UI thread (~20 seconds on my hardware).
  5. Have Worker1 and Worker2 send each other messages one second apart, recording send and receive times.
  6. Unblock the UI thread.

Note that since we invoke console.log() from workers while the UI thread is blocked, we only expect console output once it unblocks. This is normal; console.log() is a UI thread function, workers just proxy it.

Source:

Live (hosted):
http://project42.xyz/webworkerperftests/test08/ - WARNING: This may freeze your browser, as it intentionally blocks the UI thread!

Here’s my Firefox (Windows) output:

UI thread start.
Worker2 start @ 88.71000000000001
Worker1 start @ 88.71000000000001
Begin blocking UI thread @ 5052.6900000000005
End blocking UI thread @ 22589.475000000002
Message received by Worker2 @ 6087.85, contents are: Hello from Worker1! @ 6087.695
Message received by Worker1 @ 7087.88, contents are: Hello from Worker2! @ 7087.735000000001

and Chrome 64-bit (Windows):

UI thread start.
Worker1 start @ 161.08
Worker2 start @ 164.67000000000002
Begin blocking UI thread @ 5084.385
End blocking UI thread @ 27932.81
Message received by Worker1 @ 27933.305000000004, contents are: Hello from Worker2! @ 7171.8550000000005
Message received by Worker2 @ 27933.305000000004, contents are: Hello from Worker1! @ 6168.145

What this shows is that in Chrome, workers can’t talk to each other if the UI thread is blocked, at least not via the MessageChannel API. :frowning:

Both Firefox and Chrome dispatch messages just fine while UI is blocked, but only Firefox workers receive messages during this time. Chrome is unable to receive messages in its workers until the UI thread is no longer blocked.

Update
Just ran a modified test for IE11 - Date.now() instead of performance.now() - and IE11 behaves the same way Firefox does, with worker message receipt being unaffected by a blocked UI thread. Why Chrome, why? Why?!

Update 2
Related: https://code.google.com/p/chromium/issues/detail?id=443374

Oh wow, that’s something super important to know! I didn’t even think to test that :frowning:

Thanks for the linked issue, probably worth mentioning this issue there too, relating back to what they refer to in the last comment as “'main thread working on behalf of worker thread”.

Inconveniently, we can work around this in our own code by dividing up our UI loops and breaking/resuming via setImmediate… we’ll lose some time though and of course are beholden to any other user code on the UI thread :confused:

Thanks for all these tests!

Hey, no need to thank me. We’re all in the same boat/team effort/etc. :sunglasses:

Anyways, yeah–this unfortunately means if we have the engine loop running on a worker, we’ll see no improvement with Chrome.

The good news is that the implementation is effectively the same and requires no changes.

That’s really limiting!

This isn’t a test so much as a finding:

When using SharedWorker in Chrome, if you have only one tab open that owns a SharedWorker instance, and you refresh the page, the SharedWorker will fail to function in the subsequent refresh. If you then refresh yet again, it’ll work fine.

If you close the tab manually, then open a new tab, it will work fine. So, clearly some sort of issue with SharedWorker destruction in Chrome.

In Firefox under an identical scenario, what happens is the SharedWorker remains alive and thus maintains its state. This isn’t exactly sound either, but at least it’s better behavior than Chrome.

IE11 and Safari simply don’t support SharedWorker. :’(

1 Like

New test, same as Test08, but using SharedWorker instances instead of Worker instances.

Results are the same unfortunately. I was mainly hoping to find a way around the aforementioned Chrome issue. Still more things to try, though.

Source:

Live (hosted):
http://project42.xyz/webworkerperftests/test08-SharedWorker/ - WARNING: This may freeze your browser, as it intentionally blocks the UI thread!

Update

Possibly related:
https://code.google.com/p/chromium/issues/detail?id=327256

Blink/Chromium Worker Implementation Notes

Update 2
https://code.google.com/p/chromium/issues/detail?id=344814

:expressionless:

1 Like

Just a brief note:

https://code.google.com/p/chromium/issues/detail?id=405956

This means that in situations where you want multiple SharedWorkers created from the same file, you have to actually copy said file multiple times and load each accordingly. Very lame.

https://code.google.com/p/chromium/issues/detail?id=334408

:expressionless:

Update

So after a bit of testing, I’ve confirmed this only applies to explicit MessagePort objects, as opposed to the implicit type you get via the onmessage handler with regular workers. What this means in practice is that, if you’re on Chrome and want to send an ArrayBuffer from Worker A to worker B, you’re going to have to route it through the UI thread. Despite the fact you’re sending two messages, it’s still dramatically faster than a single/direct message using a structured cloning operation.

Update 2

This also means that in Chrome, if you have two UI threads, they won’t be able to communicate with each other (or each other’s workers) using transferables.

1 Like

That is quite a setback! The web is evolving, and these will be necessary for creating the performant apps that we’re imagining.

https://code.google.com/p/chromium/issues/detail?id=376637

Adding that to the list of Worker-related things broken in Chrome. Not a deal breaker, just very annoying.

Yeah, it’s interesting, all these issues. It shows that the web is transforming into a platform to compete with native, but that it’s not entirely there yet, although it’s getting a lot closer compared to a few years ago. One thing is certain: I really like the direction it’s going in. I guess all of us do since that’s the reason we’re discussing things in this forum anyway. :smile:

Well, my take is the majority of *Worker functionality was implemented 3-4 years ago, but only partially, and things have just kind of rotted since. I keep getting the impression the people actually working on the browser implementations resent those APIs and/or view them as bad specs, and I tend to agree—it’s just that spotty implementations end up making the situation far worse.

Unfortunately it’s all we have right now though, and it’s going to be quite some time before SharedArrayBuffer lands in at least two or more major browsers. Ironic really that constructs intended to insulate developers from the pitfalls of traditional parallel programming actually ends up making the situation far worse.

This might be relevant to the discussion at hand ?

Multithreaded Toolkits, a failed dream? (2004) [HN]

It’s about how every major interface toolkit that tried to go multi-threaded,
ie: allowing the threads (or workers) to update the UI, ended up walking
back from that decision to settle on using an event loop.

1 Like

I read that yesterday actually.

For what it’s worth, this work isn’t specific to UIs. That article was also written over a decade ago, and there’s been success stories since.

Of course, a couple months ago I was probably the most vocal opponent of using Workers due to the complexity cost. :laughing:

1 Like