FalcorJS with AngularJS v1.x

Ever since it was presented at ng-conf, I've been excited to get my hands on Netflix's new "One Model Everywhere" data-fetching framework: FalcorJS.  Now it's been released on the world as a Developer Preview, I wanted to show how it can be used with AngularJS v1.x.

What is FalcorJS?

"A JavaScript library for efficient data fetching"

That sums up FalcorJS pretty well.
It's designed to retrieve only as much data as you need at a time by letting the view specify the values it needs, then the model can batch those requests up when it calls the server.

Falcor represents your data as a big JSON 'graph', which means you can request the same data from different paths and it will use the cache from the first request.

Sample from FalcorJS' JSON Graph Guide

One odd thing you will have to get used to with FalcorJS is that you only request 1 value at a time. And by "value" I mean: string, number, boolean, or null. This feels totally alien coming from... everything else, but it works because FalcorJS takes care of the efficiencies for you. Instead of having 1 request for an array of objects, which you then have to split up into 20 fields on your view - you instead make 20 requests from your view, which are automatically batched for you into 1 request.

Other mechanics that Falcor provides is functions on your JSON Graph than you can 'call()', which is good for things like creating whole new objects and transactions where you want to be sure everything is being handled in one go. Responses can also include multiple values, for situations where you're sure it's going to be required, and can specify what routes should have their cache invalidated.

The last thing to mention is that FalcorJS is built on Observables from RxJS for all it's asynchronous operations. For those unfamiliar with them, Observables and the whole ReactiveX API are the ultimate asynchronous toolkit. Observables fill in the gaps with Promises, things like how to handle multiple values over time, and whether new 'observers' see just new values or the preceding values too. I won't go into Observables too much, but there are lots and lots of excellent resources online.

Using with AngularJS v1.x

Disclaimer: Once again, FalcorJS is in Developer Preview. So what's written here today may not work tomorrow. Check the FalcorJS API Reference and Github if you have difficulties.

In this example we're going to build a UI with AngularJS v1.x that uses a FalcorJS Model with a HttpDataSource pointed at our falcor-express-demo and falcor-router-demo server.

With our Model setup we can pull values from the server using this.model.getValue(path). It will then build the necessary XHR call and return an Observable for it's result.

The $apply problem

Wait a sec... if it makes it's own XHR call, how will Angular know when to start a $digest loop?

Well the usual solution in these circumstances is to use $scope.$apply(), such as when binding to an event that's not triggered within Angular's digest loop. However we can't actually be 100% sure that we're not in the digest loop either. For example, if we're hitting cached values, all the callback functions could be called immediately. A better solution is to use $scope.$evalAsync() which will start a new digest loop if, and only if, one is not already in progress. So any time we subscribe to an Observable, we simply need to call $scope.$evalAsync() and we're good to go.

Calling from the View

The next thing we want is to start making calls from the View rather than making it the controller's responsibility. This helps to decouple the view and the controller further (all the controller supplies is a reference to the model), and means we're only requesting as much data as we're actually using. eg. If we're showing a cut-down mobile friendly view, we don't have to pull back as much data.
But how are we going to get our results onto the View? All our requests are going to return fresh Observables.

This is where the Model.getCache(path) method comes in. It synchronously returns the value from the Model's cache. If that returns undefined, we can call Model.getValue(path).subscribe(_ => $scope.$evalAsync()) - It'll request the value and call $scope.$evalAsync() once it returns, which means a new digest loop, which will trigger a new call, which will get a successful hit from the cache.

Dealing with collections

Since FalcorJS doesn't support returning objects and arrays (well, sort of) we need a way to deal with collections in our data. The solution that I prefer is to generate an array of index numbers which represent a 'page range', and request the length of your collection ahead of time to set your max index.

Another thing you may be able to use is the Model.deref(path) method, passing the result to another directive that can handle resolving the Observable.

What about writing data?

Falcor comes with two primary methods for writing values: setValue(path, value) and call(path, args).

setValue() sets the value for a particular path, returning an Observable for the result. Integrating it with ng-model is simple using ng-model-options="{getterSetter: true}". We create a function which takes the model and the path, then returns a function which either calls getValue() or setValue(), depending on the number of arguments.

setValue() is great if you want to set one value at a time. But what if you want to save the entire form at once, after you've confirmed your data is valid?
This is where call() comes in.

The Falcor Router supports 3 different types of operations on each route: get, set, and call. call turns the route into an RPC endpoint where you pass in arguments, which can include objects and arrays (anything that can be JSON encoded), and it returns a set of PathValues and cache invalidations for the Falcor Model to interpret.

Summary

Once again I have to repeat: Falcor is in "Developer Preview" mode. Which means that not only is it not ready for production, but it's also only recommended for those of us who like to cut ourselves on the bleeding edge.

With that in mind: Falcor is pretty solid, and the learning curve is quite smooth. It does require rethinking your data API to fit "the falcor way", but that's a common cost with opinionated frameworks which pays back with simpler and more consistent code.

Where it really excels most is with "mostly-read/rarely-write" type applications, but you can still do write operations with `setValue()` and `calls()`. I probably wouldn't try to write any games with it (though that might be a fun exercise all the same).

I've started a github repository with a working example of everything I've shown here, along with a "falcor" module that encapsulates the falcor specific code.

Cheers,
Jason Stone

Tips for designing a JSON Schema

I've been writing a lot of JSON schemas lately.
Some were just small API requests and responses.
Some were big all encompassing documents that involves over 20 unique object types.

When I started the big ones, I went looking around the internet for some resources.  What I found was plenty on how to write a JSON Schema (This being the best one), but not on how to design one.
To contrast, there's plenty of material available on how to design relational database schemas - from blog articles to university subjects.
I still managed to come up with something resembling a process, so I thought I'd share with the class in the hopes it'll help someone else.

Just a quick disclaimer: These are just my own personal findings and opinions.
If you have your own findings that you think are better, by all means stick to them.  Share them with the class too, if you can.

Read the 'JSON API' standard

I cannot emphasise this point enough.
Go to http://jsonapi.org/ and read through the documentation and examples.
I'm not expecting you to implement it - especially if you're not creating JSON Schemas for a web API - but it's great example of a JSON based standard, and well worth the time to look at.  I've based a lot of my own schemas off elements of it, without implementing it completely.
Things like using the "id", "type", and "links" keywords.

If you do decide to implement it, take into account that it's a work in progress.
In the last month they've published a release candidate which had some significant differences from the previously published versions. But the fact that they're calling it a "release candidate" says it should be pretty stable by now.

Keep a flat structure

Don't embed objects within objects within objects within objects.
You're better off keeping a flat structure of one or many arrays at the top level containing objects.
Rather than nesting one object inside another, create a separate object in one of those arrays and link the two together - even if you're sure it's a 1-to-1 relationship for both objects.
The JSON API uses a single "included" array that contains all additional objects.  I personally prefer having multiple arrays that group the objects by "type" - but that's me.

You can also use the same prefix for property names, rather than creating a new embedded object.
eg. {"name": {"first": "John", "last": "Smith"}} becomes {"nameFirst": "John", "nameLast": "Smith"}

Why?

• It means less code like this:
  var value = data.article && data.article.author && data.article.author.phone && data.article.author.phone.number;
• It simplifies things if you're marshalling and unmarshalling your JSON into and out of other data structure. eg. Classes and SQL tables.
  Even if you're using a NoSQL DB with NodeJS today - tomorrow you might create a new microservice in something new. Doesn't hurt to be flexible.
• You have a 1-to-1 link today, but a new feature tomorrow might change that.
  Keeping things flat give you some extra flexibility for the future.

Work out how relationships are defined

Even if you don't go with a flat structure, you're going to need some way to say "X relates to Y because Z" - unless you honestly plan to duplicate and embed each and every object.
Take a look at how JSON API does it - the way the "links" property contains relationships, and each of those has a "linkage" object(s) with the "id" and "type" of the other object(s).
You don't need to do the exact same thing. But you should be consistent throughout your entire schema.
Preferably throughout all the schemas in your organisation, if you can manage it.

Why?

Because if you have a consistent format for defining relationship, you can write functions that will traverse those relationships for you.

New properties are cheap

Sometimes there's the temptation to reuse the same property for a slightly different purpose.
Maybe you want to define an "accountType" which then gives a completely different context to the 5 other properties, and you think "I just saved myself from writing 5 new properties into the schema, for the price of 1 - woooo!"

Don't do that.

Of all the possible schema changes you could make - new properties are the cheapest.
Don't make things more confusing by trying to shove square pegs into round holes.
Unless you are writing super memory efficient software, where you need to squeeze every last bit from the hardware (in which case, why are you using JSON at all?) - just make a new property.

Prepare for change

No matter how hard you try, how long you work, how much you analyse and survey and workshop - there's going to be something you need to change in the structure of the schema.
Now you can either live in constant fear of this day, or you can work out your strategy ahead of time.

Embed the version of the schema in your JSON documents and have a process in place for running migration scripts.
Whether your strategy is a 'big-bang' during downtime or an ongoing background process.
Just have your migration process ready.

Use "allOf" to create schema mixins

I said this wasn't about how to write a JSON Schema, just how to design it - this is the exception.

I gave the example earlier of prefixing property names rather than nesting objects - 'name' to 'firstName' and 'lastName'.
But what if you have 5 different objects that need to have a 'name'?
Am I supposed to duplicate that in the schema 5 times too?

No.

The JSON Schema standard defines a number of keywords that can be used to combine schemas: "oneOf", "allOf", and "anyOf".
"oneOf" is good for applying "switch" logic branching.
"anyOf" is for when you're a lot more forgiving.
"allOf" is great for constructing a schema from sub-schemas.
For example you can create a sub-schema for an object with "nameFirst" and "nameLast" properties, and include it as part of the schema for a completely different object.

Cheers,
Jason Stone

Using ES6 with your AngularJS project

Earlier this month Glen Maddern (aka. The <x-gif> guy) posted an article, Javascript in 2015, with a YouTube video giving "A brief tour of JSPM, SystemJS & ES6 features".
If you haven't seen it already, go take a look.

In a short time he has a front-end application written in ES6 running, with modules being loaded and transpiled (via Traceur) by SystemJS.  And at the end he creates a self-executing, minified bundle of that application code, including source maps.

These tools make it easy to start using ES6 now with your existing AngularJS applications - which will almost certainly be step 1 on the migration plan to Angular v2.0. Even without that, the goodies from ES6 are too good to pass up.

As a demonstration, we're going to take angular-seed, update it to ES6 using SystemJS, bundle the app for production with systemjs-builder, and configure Karma to run unit tests using the karma-systemjs plugin.

SystemJS Setup

OK - lets get started.
We're going to clone an angular-seed repo, switch to a new es6 branch, then install dependencies from npm and bower. The dependencies for angular-seed are a bit old, so we've updated them too:

git clone https://github.com/angular/angular-seed.git
cd angular-seed
git checkout -b es6
npm install --save-dev systemjs-builder karma-systemjs karma#~0.12 karma-chrome-launcher karma-firefox-launcher karma-jasmine
bower install -f -S angular#1.3.x angular-route#1.3.x angular-mocks#~1.3.x system.js

Next thing we need is a config file for SystemJS: `app/system.config.js`. This is where we tell SystemJS how to find certain modules.  In this case, we're going to map the module names 'angular' and 'angular-route' to their long paths:

Next we'll change `app/index-async.html` to use SystemJS rather than angular-loader:

Last thing to do is add `import` statements to `app/app.js` to include the rest of the application, as it's what will be loaded by `System.import('app')`:

Now if you load up `app/index-async.html` using a local webserver (The one included with angular-seed start with `npm start`) you should see the angular-seed application running as normal, only it's been loaded as a set of ES6 files.

This is just the bare minimum required to get existing code to ES6 using SystemJS.
You can add more ES6 syntax and rearrange the modules as you like.

Here's my suggestions:

• Use classes for Controllers (Controller as), Services (.service()), and Providers
• Use ES6 modules over angular modules
• Use arrow functions - cause they're sugary sweet!

Bundling for Production

With our SystemJS config file already in place, we only need to pass that into systemjs-builder with a slight tweak to bundle our code for production. This is a simple node script you can run with `node bundle.js` that will bundle all the modules imported by `app/app.js` and output the result to `app/bundle.js`:

And finally we change `app/index.html` to include `traceur-runtime.js` and the `bundle.js` file we just created:

Unit Testing

Karma works like most a typical browser application: Load all <script/> tags, then start the application `onload`. In this case, we want Karma to let SystemJS handle loading everything, then start the test runner once everything is ready.  This is essentially what karma-systemjs does for us.

First we need to adjust `karma.conf.js`:

Next we update our test suites to act like ES6 modules - importing the code to be tested, along with any dependency libraries.

Note how `module()` has been changed to `angular.mock.module()`.

And that's it.
An existing AngularJS project converted over to using ES6 with SystemJS, with working unit tests and a means of generating bundles for production.
You can find the complete code here: https://github.com/rolaveric/angular-seed/tree/es6

Cheers,
Jason Stone