Thursday, May 19, 2016

Baibulo is born!

Howdy, folks!

As promised (although it took forever and then some) my first Node module is released! Baibulo (version in Chewa) is a very simple versioned content server backed by Redis. Check it out - the README should be enough to get you started but if that is not good enough there is a working example that you can basically take and play with :)

I know there's probably a ton of things that should be done differently. Please share if you think it makes sense or not. Feedback is very much welcomed!

Happy coding!

Wednesday, May 11, 2016

The world around us is changing

A few years back when working on a redesign of a small internal application for market managers at Travelocity I made a decision to go with the new and shiny single page application approach. Back in 2010 it was quite unheard of and spawned questions like "is the browser going to be fast enough?" or "do we have good enough developers to do it?". To tell you the truth both questions should back then be answered negative but as it turns out the competition that I was going against was of such low quality that anything that actually works would have been better. To sum it up the app was created in Grails of which I have mostly used the GSP engine to create the index file and a few controllers to get the API in place. In all seriousness this was quite a simple app but the effect it made on users exceeded my wildest dreams.

Now forward a few years and not only everyone knows what a single page application is but the world gets divided into the client side and the server side. It seems that everybody that forgets this is going to be left behind soon-ish :)

One of the benefits of thinking about the system as 2 entities (the client app backed by an API) is that intuitively one will arrive at a place where those two get to be developed independently. This means 2 version control repositories, 2 separate teams, 2 different build systems and 2 different sets of skills required to do the job right.

As some of you already know I am working on an intelligent home system for myself. Recently I have decided that I'd like it to be a testing field for ideas that I have stumbled upon elsewhere that would allow me to exercise in a more realistic scenario than a hello-world-type app the things others came up with. That being said I did come across a presentation from 2014 on the railsconf about splitting the deployment of static assets (a.k.a the frontend application) and using Redis to store the content of an index file so that it can be, sort of, versioned. This is one of those ideas that are really worth exploring and since I already have made the decision to split up the backend and frontend into two separate repos it was the perfect scenario to play with.

Due to my current professional occupation I've gotten a lot more interested in JavaScript development and that in both the backend as well as frontend. And so I have decided to implement the first version of the versioned content server in Node using Redis as advertised by Luke Melia but with a slight twist. I'm not really interested in all the S3 portion of it since I will be doing a thing that is mostly served by a server I know about which means I can do all the file serving myself. That being said I have decided to just version everything and have Redis as a versioned file system for my static assets.

Since I am a fanatic enthusiast of the Sinatra framework there is no surprise that I have selected ExpressJS as my weapon of choice. Both Sinatra and Express have the notion of middleware that can sit between the bare metal and your app doing god knows what. I my case I wanted to completely take over a portion of server and just serve things from Redis instead of file system like the express.static middleware would. The schema for naming keys I came up with is quite simple:

prefix:/context/path:[version]
prefix:/context/path:content-type:[version]
prefix:/context/path:etag:[version]
and to store the current version that will be served when no version is specified I'd use
prefix:/context:[version]
One of the benefits of taking the "store it all in Redis" approach is that one can query Redis to give back a full list of available versions thus making a sort of a versions selector page an easy task. Specifying the version happens via a version=[specific-version] query parameter as it allows for easy creation of links to particular versions. It just so happens that all the assets being retrieved have the header Referer that contains that particular information from the original URL so other requests (including XMLHttpRequests) can take advantage of the specified version. And since this is mostly used in scenarios I have full control over (for testing, preview and the like) there is no problem with any proxies stripping that info out of the request. And since the version can be any string I am able to deploy a feature branch, a test branch, a new proper version - whatever I want and it just transparently works. The middleware is to be mounted per context, like /hello and will use that mount point as part of the Redis key to differentiate it between different frontend apps in case there will be more than one.

The one problem I needed to solve was how to upload stuff to Redis in an efficient manner. For now I have not solved the "efficient" part of it and I just store each file with some metadata in each version in Redis. This has the added benefit of me being able to completely remove one version from Redis whenever I want in an easy fashion.

The next challenge I am working on is to be able to serve a single version of frontend against 2 or more versions of the backend. This can be quite useful when updating the backend and validating it against the current or upcoming release of the frontend app.

I will soon publish that as part of the Aplaster project and maybe deploy the server and content uploader as a gem for everyone to take a chance to use it? Will see :)

Happy coding!