Asynchronous tests

JS.Test provides a simple mechanism for asynchronous testing based on continuation-passing style. Test suites are organised using contexts with each test delimited by an it block. For example:

JS.ENV.AjaxSpec = JS.Test.describe('Ajax', function() { with(this) {
    it('adds numbers', function() { with(this) {
        assertEqual(2, 1 + 1);
    }});
}});

The test runner assumes that your tests are synchronous by default; that is when we reach the end of the it block that test has finished and we can move onto the next. But it might not have finished – we may have started an Ajax request or set a timeout that we’re waiting on before finishing the test.

To tell JS.Test that a test is asynchronous, we can add a parameter to the it block. The test runner will then fill this parameter with a callback function that you can use to resume the test. When you can the resume function you pass in a function containing the assertions you want to make.

For example, here’s a quick test using jQuery to make an Ajax call to a server:

JS.ENV.AjaxSpec = JS.Test.describe('Ajax', function() { with(this) {
    it('fetches a page', function(resume) { with(this) {
        jQuery.get('/foo.html', function(response) {
            resume(function() {
                assertEqual('Hello, World', response);
            });
        });
    }});
}});

If you add the resume parameter to the it block, the test runner will not consider the test complete until the resume function is invoked. If you do not invoke the resume function the test runner will never move onto the next test.

These resume blocks can be nested if you have multiple asynchronous steps in a test, but this typicaly doesn’t scale well in terms of readability. If you’re running an integration test with a lot of asynchronous parts, it’s more useful to abstract the steps in the test into functions that hide the async plumbing.

Asynchronous stories

Let’s take a somewhat contrived example: we want to build an API to manipulate blog posts over HTTP, and we’re going to call it using jQuery. Our test will involve the following steps:

We’re going to test all of this from the outside by making requests. A test for these steps might look like this:

JS.ENV.BlogPostSpec = JS.Test.describe('blog post API', function() { with(this) {
    it('creates a blog post on the server', function() { with(this) {
        create_blog_post({title: 'JavaScript testing'});
        assert_json_response();
        assert_response_has_field('id');
        get_blog_post_by_id();
        assert_response_has_field('title', 'JavaScript testing');
    }});
}});

This test has a few properties worth noting:

JS.Test gives a way to write asynchronous tests that look like the example above. The first step is to implement all the steps the test needs, but give each function an additional callback argument (which we’ll call resume) that it should invoke when its work is done. We create a set of testing steps using the asyncSteps() function:

JS.ENV.BlogPostSteps = JS.Test.asyncSteps({
    create_blog_post: function(attributes, resume) { with(this) {
        var testCase = this;
        jQuery.post('/blog_posts', attributes, function(response) {
            testCase.response = response;
            resume();
        });
    }},
    assert_json_response: function(resume) { with(this) {
        assertNothingThrown(function() { JSON.parse(response) });
        resume();
    }},
    assert_response_has_field: function(field, value, resume) { with(this) {
        var data = JSON.parse(response);
        assert(data.hasOwnProperty(field));
        if (value) assertEqual(value, data[field]);
        resume();
    }},
    get_blog_post_by_id: function(resume) { with(this) {
        var testCase = this;
        var id = JSON.parse(response).id;
        jQuery.get('/blog_posts/' + id, function(response) {
            testCase.response = response;
            resume();
        });
    }}
});

Note how all functions used as test steps must take a resume callback after all the arguments used explicitly in the test, and invoke it to indicate the work of that step is finished. JS.Test uses these callbacks to glue your steps together when running a test.

If you need to retain state during a test, for example holding on to an HTTP response as shown above, you can store data on the current test case object (referred to by this) and it will be available in subsequent steps.

Once you’ve made all your test steps, you just need to add them to your test using the include() function:

JS.ENV.BlogPostSpec = JS.Test.describe('blog post API', function() { with(this) {
    // Make the step functions available in this test
    include(BlogPostSteps);

    it('creates a blog post on the server', function() { with(this) {
        create_blog_post({title: 'JavaScript testing'});
        assert_json_response();
        assert_response_has_field('id');
        get_blog_post_by_id();
        assert_response_has_field('title', 'JavaScript testing');
    }});
}});

Note that when you write a test like this, JS.Test does not make the steps blocking; the purpose of the code in the it block is to queue up a set of actions that define the test, then JS.Test deals with sequencing the actions for you. For this reason, you can’t inspect the state of the test from within the it block, you must put any debugging into the step functions themselves.

One other side effect of this is that if you write an after block, your test code will still be running when the after block starts. To wait for the test to complete, use the sync() function; for example if we were stubbing out jQuery in our test above:

JS.ENV.BlogPostSpec = JS.Test.describe('blog post API', function() { with(this) {
    include(BlogPostSteps);

    before(function() { with(this) {
        JS.ENV.jQuery = {};
        stub(jQuery, 'post').yields(['{"id":1}']);
        stub(jQuery, 'get').yields(['{"title":"JavaScript testing"}']);
    }});

    after(function(resume) { with(this) {
        // This waits for the async commands to complete
        sync(function() {
            JS.ENV.jQuery = undefined;
            resume();
        });
    }});
}});

It is also good practise to confine the steps themselves to one asynchronous action at most per step. The purpose of this pattern is to break a large async test up into composable parts that can be easily re-ordered. Note how the async examples above don’t have complex logic in their callbacks, they typically just store the result of an action for processing by other steps.

Placing complex logic or assertions in the async callbacks means that JS.Test cannot catch any exceptions they throw, which is another reason to keep them as simple as possible.