I was recently on The Cognicast with Craig Andera where we discussed using Generative Testing on a large non-Clojure(script) codebase, in particular Ruby on Rails and Backbonejs. If you haven’t listened to the show yet I highly recommend it first.
- Compile JS into one file (just like prod).
- Compile tests into a single file.
- Combine them in a PhantomJS process.
- Let the tests do their thing.
Here are the challenges that lie between us and Generative Testing bliss.
- Picking the right library
- Setting up Leiningen & Cljsbuild
- Dodge PhantomJS issues
- Avoid mangling your app, and defeating dueling dependencies
Picking the Right Library
First of all, there are two libraries that exist, Test.Check and DoubleCheck. Because Test.Check is an official Clojure library it is Clojure (JVM) only, so I recommend DoubleCheck (maintained by Chas Emerick) which is capable of cross compiling to Clojure and Clojurescript.
The only catch with DoubleCheck is that it’s not currently possible to segregate tests via metadata for running in groups. But with the way we will be running these tests that won’t be an issue.
Setting up Leiningen
I highly recommend you send the output of the compilation process to either a temporary or gitignored location. The output will be fairly large, and it will bog down your repository with its size.
I don’t want to duplicate the Cljsbuild how-to, so if you don’t know how to make it work, you should check their docs. Our project.clj is reproduced at the bottom of this post if you have issues.
At this point, we can write the simplest test possible:
1 2 3 4 5 6 7 8 9 10 11 12 13
In order to run the tests, you would typically have a :tests section in :cljsbuild that looks like this:
1 2 3
This will load our JS into the app, along with the tests, and then run them. But you might notice errors that look like this:
That means that your app code is trying to access local storage, and PhantomJS does not like it when you do that without loading a webpage. The solution for this is to start a server so we have a page to visit, and visit it via PhantomJS. So on Tracker we use the following two bits of code.
generative_runner.js, to visit the actual page:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
To make all this work together, update project.clj to reference the
generative_runner.js file instead of :runner, and use
test:generative to kick off the run.
Don’t Mangle Your Code
If your application is anything like Tracker, you might use some Google Closure dependencies without using the entire Closure compiler. And even if you don’t need use Closure, you certainly have functions and classes in the global namespace that you don’t want mangled.
To get around this, I recommend the following settings:
:libs [ "compiled-application.js" ""] to the cljsbuild section
in project.clj. This prevents DoubleCheck compiler errors due to
classpath issues, and it allows the Closure compiler to see everything
that your application provides. So if your tests and applications have
overlapping Closure dependencies you won’t get double provide errors.
Secondly I recommend that you only use the simple compilation mode. This will prevent Closure from mangling global names, which will make debugging easier and prevent your tests from being able to find the production code. The space saving and code elimination that advanced mode provides is more of a problem than a benefit for testing, so it’s not worth fighting to get advanced to work.
Have Fun and Make More Tests
Once you have that going, it should be possible to open up and create increasingly complicated tests. As a teaser and a good example, the following code caught a tricky JS ordering bug.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
And the following is our project.clj, with unnecessary details elided for readability.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
1 2 3 4 5 6 7 8 9 10 11
Good luck, and don’t hesitate to reach out to me on Twitter if you have any questions!