Jerry Jacob's Blog

Writing Testable JavaScript (written by Ben Cherry)

Posted in Uncategorized by jerlinworld on March 5, 2015

The engineering culture at Twitter requires tests. Lots of tests. I haven’t had formal experience with JavaScript testing before Twitter, so I’ve been learning a lot as I go. In particular, a number of patterns I used to use, write about, and encourage have turned out to be bad for writing testable code. So I thought it would be worthwhile to share a few of the most important principles I’ve developed for writing testable JavaScript. The examples I provide are based on QUnit, but should be just as applicable to any JavaScript testing framework.

Avoid Singletons

One of my most popular posts was about using JavaScript Module Pattern to create powerful singletons in your application. This approach can be simple and useful, but it creates problems for testing, for one simple reason: singletons suffer state pollution between tests. Rather than creating your singletons as modules, you should compose them as constructable objects, and assign a single, default instance at the global level in your application init.

For example, consider the following singleton module (contrived example, of course):

var dataStore = (function() {
	var data = [];
	return {
		push: function (item) {
			data.push(item);
		},
		pop: function() {
			return data.pop();
		},
		length: function() {
			return data.length;
		}
	};
}());

With this module, we may wish to test the foo.bar method. Here’s a simple QUnit test suite:

module("dataStore");
test("pop", function() {
	dataStore.push("foo");
	dataStore.push("bar")
	equal(dataStore.pop(), "bar", "popping returns the most-recently pushed item");
});

test("length", function() {
	dataStore.push("foo");
	equal(dataStore.length(), 1, "adding 1 item makes the length 1");
});

When running this test suite, the assertion in the length test will fail, but it’s not clear from looking at it why it should. The problem is that state has been left in dataStore from the previous test. Merely re-ordering these tests will cause the length test to pass, which is a clear red flag that something is wrong. We could fix this with setup or teardown that reverts the state of dataStore, but that means that we need to constantly maintain our test boilerplate as we make implementation changes in the dataStore module. A better approach is the following:

function newDataStore() {
	var data = [];
	return {
		push: function (item) {
			data.push(item);
		},
		pop: function() {
			return data.pop();
		},
		length: function() {
			return data.length;
		}
	};
}

var dataStore = newDataStore();

Now, your test suite will look like this:

module("dataStore");
test("pop", function() {
	var dataStore = newDataStore();
	dataStore.push("foo");
	dataStore.push("bar")
	equal(dataStore.pop(), "bar", "popping returns the most-recently pushed item");
});

test("length", function() {
	var dataStore = newDataStore();
	dataStore.push("foo");
	equal(dataStore.length(), 1, "adding 1 item makes the length 1");
});

This allows our global dataStore to behave exactly as it did before, while allowing our tests to avoid polluting each other. Each test owns its own instance of a DataStore object, which will be garbage collected when the test completes.

Avoid Closure-based Privacy

Another pattern I used to promote is real private members in JavaScript. The advantage is that you can keep globally-accessible namespaces free of unnecessary references to private implementation details. However, overuse of this pattern can lead to untestable code. This is because your test suite cannot access, and thus cannot test, private functions hidden in closures. Consider the following:

function Templater() {
	function supplant(str, params) {
		for (var prop in params) {
			str.split("{" + prop +"}").join(params[prop]);
		}
		return str;
	}

	var templates = {};

	this.defineTemplate = function(name, template) {
		templates[name] = template;
	};

	this.render = function(name, params) {
		if (typeof templates[name] !== "string") {
			throw "Template " + name + " not found!";
		}

		return supplant(templates[name], params);
	};
}

The crucial method for our Templater object is supplant, but we cannot access it from outside the closure of the constructor. Thus, a testing suite like QUnit cannot hope to verify that it works as intended. In addition, we cannot verify that our defineTemplate method does anything without trying a .render() call on the template and watching for an exception. We could simply add a getTemplate() method, but then we’d be adding methods to the public interface solely to allow testing, which is not a good approach. While the issues here are probably just fine in this simple example, building complex objects with important private methods will lead to relying on untestable code, which is a red flag. Here’s a testable version of the above:

function Templater() {
	this._templates = {};
}

Templater.prototype = {
	_supplant: function(str, params) {
		for (var prop in params) {
			str.split("{" + prop +"}").join(params[prop]);
		}
		return str;
	},
	render: function(name, params) {
		if (typeof this._templates[name] !== "string") {
			throw "Template " + name + " not found!";
		}

		return this._supplant(this._templates[name], params);
	},
	defineTemplate: function(name, template) {
		this._templates[name] = template;
	}
};

And here’s a QUnit test suite for it:

module("Templater");
test("_supplant", function() {
	var templater = new Templater();
	equal(templater._supplant("{foo}", {foo: "bar"}), "bar"))
	equal(templater._supplant("foo {bar}", {bar: "baz"}), "foo baz"));
});

test("defineTemplate", function() {
	var templater = new Templater();
	templater.defineTemplate("foo", "{foo}");
	equal(template._templates.foo, "{foo}");
});

test("render", function() {
	var templater = new Templater();
	templater.defineTemplate("hello", "hello {world}!");
	equal(templater.render("hello", {world: "internet"}), "hello internet!");
});

Notice that our test for render is really just a test that defineTemplate and supplant integrate correctly with each other. We’ve already tested those methods in isolation, which will allow us to easily discover which components are really breaking when tests of the render method fail.

Write Tight Functions

Tight functions are important in any language, but JavaScript presents its own reasons to do so. Much of what you do with JavaScript is done against global singletons provided by the environment, and which your test suite relies on. For instance, testing a URL re-writer will be difficult if all of your methods try to assign window.location. Instead, you should break your system into its logical components that decide what to do, then write short functions that actually do it. You can test the logical functions with various inputs and outputs, and leave the final function that modifies window.location untested. Provided you’ve composed your system correctly, this should be safe.

Here’s an example URL rewriter that is not testable:

function redirectTo(url) {
	if (url.charAt(0) === "#") {
		window.location.hash = url;
	} else if (url.charAt(0) === "/") {
		window.location.pathname = url;
	} else {
		window.location.href = url;
	}
}

The logic in this example is relatively simple, but we can imagine a more complex redirecter. As complexity grows, we will not be able to test this method without causing the window to redirect, thus leaving our test suite entirely.

Here’s a testable version:

function _getRedirectPart(url) {
	if (url.charAt(0) === "#") {
		return "hash";
	} else if (url.charAt(0) === "/") {
		return "pathname";
	} else {
		return "href";
	}
}

function redirectTo(url) {
	window.location[_getRedirectPart(url)] = url;
}

And now we can write a simple test suite for _getRedirectPart:

test("_getRedirectPart", function() {
	equal(_getRedirectPart("#foo"), "hash");
	equal(_getRedirectPart("/foo"), "pathname");
	equal(_getRedirectPart("http://foo.com"), "href");
});

Now the meat of redirectTo has been tested, and we don’t have to worry about accidentally redirecting out of our test suite.

__Note__: There is an alternative solution, which is to create a `performRedirect` function that does the location change, but stub that out in your test suite. This is a common practice for many, but I’ve been trying to avoid method stubbing. I find basic QUnit to work well in all situations I’ve found so far, and would prefer to not have to remember to stub out a method like that for my tests, but your case may differ.

Write Lots of Tests

This is a no-brainer, but it’s important to remember. Many programmers write too few tests because writing tests is hard, or lots of work. I suffer from this problem all the time, so I threw together a little helper for QUnit that makes writing lots of tests a lot easier. It’s a function called testCases which you call within a test block, passing a function, calling context, and array of inputs/outputs to try and compare. You can quickly build up a robust suite for your input/output functions for rigorous testing.

function testCases(fn, context, tests) {
	for (var i = 0; i < tests.length; i++) {
		same(fn.apply(context, tests[i][0]), tests[i][1],
			tests[i][2] || JSON.stringify(tests[i]));
	}
}

And here’s a simple example use:

test("foo", function() {
	testCases(foo, null, [
		[["bar", "baz"], "barbaz"],
		[["bar", "bar"], "barbar", "a passing test"]
	]);
});

Conclusions

There is plenty more to write about testable JavaScript, and I’m sure there are many good books, but I hope this was a good overview of practical cases I encounter on a daily basis. I’m by no means a testing expert, so please let me know if I’ve made mistakes or given bad advice.

JavaScript Module Pattern: In-Depth (written by Ben Cherry)

Posted in Uncategorized by jerlinworld on March 5, 2015

The module pattern is a common JavaScript coding pattern. It’s generally well understood, but there are a number of advanced uses that have not gotten a lot of attention. In this article, I’ll review the basics and cover some truly remarkable advanced topics, including one which I think is original.

The Basics

We’ll start out with a simple overview of the module pattern, which has been well-known since Eric Miraglia (of YUI) first blogged about it three years ago. If you’re already familiar with the module pattern, feel free to skip ahead to “Advanced Patterns”.

Anonymous Closures

This is the fundamental construct that makes it all possible, and really is the single best feature of JavaScript. We’ll simply create an anonymous function, and execute it immediately. All of the code that runs inside the function lives in a closure, which provides privacy and state throughout the lifetime of our application.

(function () {
	// ... all vars and functions are in this scope only
	// still maintains access to all globals
}());

Notice the () around the anonymous function. This is required by the language, since statements that begin with the token function are always considered to be function declarations. Including () creates a function expression instead.

Global Import

JavaScript has a feature known as implied globals. Whenever a name is used, the interpreter walks the scope chain backwards looking for a var statement for that name. If none is found, that variable is assumed to be global. If it’s used in an assignment, the global is created if it doesn’t already exist. This means that using or creating global variables in an anonymous closure is easy. Unfortunately, this leads to hard-to-manage code, as it’s not obvious (to humans) which variables are global in a given file.

Luckily, our anonymous function provides an easy alternative. By passing globals as parameters to our anonymous function, we import them into our code, which is both clearer and faster than implied globals. Here’s an example:

(function ($, YAHOO) {
	// now have access to globals jQuery (as $) and YAHOO in this code
}(jQuery, YAHOO));

Module Export

Sometimes you don’t just want to use globals, but you want to declare them. We can easily do this by exporting them, using the anonymous function’s return value. Doing so will complete the basic module pattern, so here’s a complete example:

var MODULE = (function () {
	var my = {},
		privateVariable = 1;

	function privateMethod() {
		// ...
	}

	my.moduleProperty = 1;
	my.moduleMethod = function () {
		// ...
	};

	return my;
}());

Notice that we’ve declared a global module named MODULE, with two public properties: a method named MODULE.moduleMethod and a variable named MODULE.moduleProperty. In addition, it maintains private internal state using the closure of the anonymous function. Also, we can easily import needed globals, using the pattern we learned above.

Advanced Patterns

While the above is enough for many uses, we can take this pattern farther and create some very powerful, extensible constructs. Lets work through them one-by-one, continuing with our module named MODULE.

Augmentation

One limitation of the module pattern so far is that the entire module must be in one file. Anyone who has worked in a large code-base understands the value of splitting among multiple files. Luckily, we have a nice solution to augment modules. First, we import the module, then we add properties, then we export it. Here’s an example, augmenting our MODULE from above:

var MODULE = (function (my) {
	my.anotherMethod = function () {
		// added method...
	};

	return my;
}(MODULE));

We use the var keyword again for consistency, even though it’s not necessary. After this code has run, our module will have gained a new public method named MODULE.anotherMethod. This augmentation file will also maintain its own private internal state and imports.

Loose Augmentation

While our example above requires our initial module creation to be first, and the augmentation to happen second, that isn’t always necessary. One of the best things a JavaScript application can do for performance is to load scripts asynchronously. We can create flexible multi-part modules that can load themselves in any order with loose augmentation. Each file should have the following structure:

var MODULE = (function (my) {
	// add capabilities...

	return my;
}(MODULE || {}));

In this pattern, the var statement is always necessary. Note that the import will create the module if it does not already exist. This means you can use a tool like LABjs and load all of your module files in parallel, without needing to block.

Tight Augmentation

While loose augmentation is great, it does place some limitations on your module. Most importantly, you cannot override module properties safely. You also cannot use module properties from other files during initialization (but you can at run-time after intialization). Tight augmentation implies a set loading order, but allows overrides. Here is a simple example (augmenting our original MODULE):

var MODULE = (function (my) {
	var old_moduleMethod = my.moduleMethod;

	my.moduleMethod = function () {
		// method override, has access to old through old_moduleMethod...
	};

	return my;
}(MODULE));

Here we’ve overridden MODULE.moduleMethod, but maintain a reference to the original method, if needed.

Cloning and Inheritance

var MODULE_TWO = (function (old) {
	var my = {},
		key;

	for (key in old) {
		if (old.hasOwnProperty(key)) {
			my[key] = old[key];
		}
	}

	var super_moduleMethod = old.moduleMethod;
	my.moduleMethod = function () {
		// override method on the clone, access to super through super_moduleMethod
	};

	return my;
}(MODULE));

This pattern is perhaps the least flexible option. It does allow some neat compositions, but that comes at the expense of flexibility. As I’ve written it, properties which are objects or functions will not be duplicated, they will exist as one object with two references. Changing one will change the other. This could be fixed for objects with a recursive cloning process, but probably cannot be fixed for functions, except perhaps with eval. Nevertheless, I’ve included it for completeness.

Cross-File Private State

One severe limitation of splitting a module across multiple files is that each file maintains its own private state, and does not get access to the private state of the other files. This can be fixed. Here is an example of a loosely augmented module that will maintain private state across all augmentations:

var MODULE = (function (my) {
	var _private = my._private = my._private || {},
		_seal = my._seal = my._seal || function () {
			delete my._private;
			delete my._seal;
			delete my._unseal;
		},
		_unseal = my._unseal = my._unseal || function () {
			my._private = _private;
			my._seal = _seal;
			my._unseal = _unseal;
		};

	// permanent access to _private, _seal, and _unseal

	return my;
}(MODULE || {}));

Any file can set properties on their local variable _private, and it will be immediately available to the others. Once this module has loaded completely, the application should call MODULE._seal(), which will prevent external access to the internal _private. If this module were to be augmented again, further in the application’s lifetime, one of the internal methods, in any file, can call _unseal() before loading the new file, and call _seal() again after it has been executed. This pattern occurred to me today while I was at work, I have not seen this elsewhere. I think this is a very useful pattern, and would have been worth writing about all on its own.

Sub-modules

Our final advanced pattern is actually the simplest. There are many good cases for creating sub-modules. It is just like creating regular modules:

MODULE.sub = (function () {
	var my = {};
	// ...

	return my;
}());

While this may have been obvious, I thought it worth including. Sub-modules have all the advanced capabilities of normal modules, including augmentation and private state.

Conclusions

Most of the advanced patterns can be combined with each other to create more useful patterns. If I had to advocate a route to take in designing a complex application, I’d combine loose augmentation, private state, and sub-modules.

I haven’t touched on performance here at all, but I’d like to put in one quick note: The module pattern is good for performance. It minifies really well, which makes downloading the code faster. Using loose augmentation allows easy non-blocking parallel downloads, which also speeds up download speeds. Initialization time is probably a bit slower than other methods, but worth the trade-off. Run-time performance should suffer no penalties so long as globals are imported correctly, and will probably gain speed in sub-modules by shortening the reference chain with local variables.

To close, here’s an example of a sub-module that loads itself dynamically to its parent (creating it if it does not exist). I’ve left out private state for brevity, but including it would be simple. This code pattern allows an entire complex heirarchical code-base to be loaded completely in parallel with itself, sub-modules and all.

var UTIL = (function (parent, $) {
	var my = parent.ajax = parent.ajax || {};

	my.get = function (url, params, callback) {
		// ok, so I'm cheating a bit 🙂
		return $.getJSON(url, params, callback);
	};

	// etc...

	return parent;
}(UTIL || {}, jQuery));

I hope this has been useful, and please leave a comment to share your thoughts. Now, go forth and write better, more modular JavaScript!