[API] Build System Goals

Design
#1

Obviously we want this project to be module system agnostic. What about the build system?

One of the main complaints with Famous was that you basically had to use its build system, and thus integrating it into an existing project was quite difficult if you had your own build pipeline.

How do we achieve the best of both worlds - have the project build itself out of the box, but also easily integrate into other pipelines?

Do we want to just use npm, or something heavier like Gulp?

#2

Things like Gulp and Grunt are unnecessary extra tools that can be left to devs to use with their apps.

have the project build itself out of the box, but also easily integrate into other pipelines?

We should support the most common set of build systems without any extra layers (no gulp/grunt): Webpack+NPM, Browserify+NPM, JSPM (by Guy Bedford, who leads efforts on things like this), Bower+RequireJS (yes, we can! Bower still has 470,000 downloads per week), and a global build (that’s a build system for many people still :laughing:).

I’ve been thinking about how to be compatible with all those cases since when I started the infamous/infamous repo using Famous 0.3. Check out the package.json over in https://github.com/infamous/motor/tree/joe.

The idea is fairly simple:

  • package.json includes scripts to build to AMD, CJS, and UMD (assuming we’re writing in ES6).
  • Source lives in the src folder.
  • On prepublish to npmjs.com, files are built to CJS format into to root of the package, then published. Prepublish only uses the build-cjs script, and the other scripts are there in case someone wants to make use of them, for convenience (for example, someone might have a custom RequireJS workflow, and can use the build-amd script to get AMD files). The clean script removes the build files after prepublish, or manually if needed (it takes a little extra effort to maintain these scripts as the code base changes, but I feel it’s worth it).
  • The packages is published with both the build files in the root and the original files in the src folder to maximize coverage of use cases. Some people will require the root files directly, using Webpack or Browserify, or perhaps even natively in some Node.js-based GUI, or even on the serverside for testing or something (or if we one day support server-side rendering). The src folder is intact so that people who use JSPM can set directories.lib = src and JSPM will set that folder as the root of the package once installed. Other people might like to set an alias in Webpack to the src folder in order to use the original ES6 modules for some reason. Basically, let’s let people do what they want.
  • The devDependencies include babelify in case someone configures browserify to use the src folder’s ES6 files instead of the CommonJS ones. No harm done including it.
  • TODO: We need to precompile GLSL during prepublish. We can use either Webpack or Browserify for that with glslify or glslify-loader. @talves has thought about this. How can we inject the GLSL without modifying the rest of the code in the file where it is injected? Or might it not matter if Webpack is compiling, perhaps, a Broweserified/glslified file?
  • There are some extra fields for jspm and bower. Bower is todo.
  • The babel-runtime package is a runtime dependency, which simply includes some function helpers like those used to create prototype-based classes in ES5, etc. The --optional runtime option in the build-* scripts tells babel not to include these helper functions in each and every file, instead using require to get the helpers. It’s a good thing to have the runtime.
  • The format property helps some build systems. F.e., JSPM doesn’t have to make a guess.
  • The browser, main, and global fields help in some cases.

Basically, the idea was to maximize coverage of the most notable use cases. Not necessary, but I’ve also included a Meteor package to make it easy for Meteor users with no module experience to consume it. I wrote that fast, I feel like it was sloppy. x}

1 Like
#3

@joe Sorry I didn’t reply to this earlier.

All I can say is that’s hardcore!

1 Like
#4

I agree with this statement. Although, I am not sure if we would need this dependency in the library except in a single build. Recommending it to the users with examples. If we want to include these helpers, we may want to consider just using core-js as the dependency instead. Research Needed

#5

Workers are another concern for the build system.

The simplest idea I can think of would be to output a single monolithic bundle file that’s capable of running as either UI or worker, depending. Lots of dead code that wouldn’t get hit depending on the context, but that approach would still be size-efficient. It would also be simple; the library could load whatever it wanted wherever it wanted from a single file, at runtime, without having to translate those requirements to the build system.

Beyond that, things get ugly fast.

#6

I’ve come to the sentiment that we shouldn’t include it, and have decided to write only ES5 that compiles to straight ES5 with no need for helpers. So, for example, using let is fine, because that compiles to pure var statements. I noticed that babel-runtime was wrapping Object.create with a helper function (to polyfill older browsers), etc, and we honestly just don’t need things like that.

I’m starting my engine prototype now, and I’m using my lowclass lib to write all my classes and making absolutely nothing needs helpers along the way. Thanks to @gadicc’s initial tests and research, I’m motivated to get as much performance as possible (except for rare cases like when extending classes it’s necessary to use Object.getOwnPropertyDescriptor and Object.defineProperty calls in order to copy over getters and setters, which is less performance than simply settings keys with object[key] notation (in that case the getter is read and it’s return value assigned rather than the getter itself being copied over).

#7

I don’t think that’ll result in the best API (Famous tried to be like that, and the result is you end up running API in workers where you want to get a handle to a DOM element for example, and that simply doesn’t work). I think the best solution will be to always have a UI-side API with automatic delegation to threads. That’s one of the key concepts I’m working on right now for my prototype.

We also want fast load times through the network, so it’d be better to avoid dead code. It also takes more time to execute all the unused modules that define unused classes.

I don’t think it’ll get ugly, because the thread-creation logic will lie in the library, not the end user, and the goal will be that it still works even if you’re only using half of the classes and the rest don’t even exist in the app. I’ll be working on designing this the rest of the evening.

#8

@joe Dead code as in code that isn’t reached during execution. If said parts were reachable in a worker, it would cease to be dead code, and if the code tried accessing DOM from a worker context, it would also be faulty broken code on top of it.

My main point was, unless the workers load up the exact same script the UI thread is running on, you’re going to have to compile the worker script(s) separate, and that’ll have serious build implications.

Also, a single script would presumably be cached and thus wouldn’t incur the network overhead of loading multiple scripts.

#9

What might you mean? As in stringify the function during runtime and send it into the thread at runtime?

Check out this awesomeness (BridgedWorkers): http://stackoverflow.com/a/16799132/454780

#10

@joe:
What might you mean? As in stringify the function during runtime and send it into the thread at runtime?

No, as in building worker scripts at compile time that contain only what functionality is needed (or may be needed).

To be clear, I think one monolithic file is superior to that approach.

Also, the BridgedWorker thing is really neat. Not sure if it’s suitable for Infamous though.

1 Like