Property-Based Testing in Typescript with Fast-Check

Property-based testing is probably the thing I missed the most from my time working with Haskel. It is such an elegant way of testing functionalities that it is hard to not use it. As you can imagine, I look for a Property-based testing framework in any language I have under my hands. Usually, unsuccessfully.

However, in recent years I am working a lot with Typescript and, luckily, Typescript has a good property-based testing library: fast-check.

Wait. You don’t know what is property-based testing? Let’s start from the beginning.

What is Property-Based Testing

Property-Based Testing (PBT) is a testing methodology mainly introduced by Haskell’s Quick Test library. As the name says, it is based on properties. A property, in short, is a condition that is verified independently of the specific input.

For instance, “reversing a list twice return the original list” is a property. “Sorting a sorted list does not change the list” is another one. “Adding N elements to a queue and removing N elements gives me the empty queue” is another one. As you can see, these are facts that are always true for every possible (legal) input. As a consequence, we can leave to the PBT framework the hard work of generating inputs and edge cases that will make our test fail.

With this simple intuition, we get two great benefits:

We do not need to generate the input data: we just need to define what is a legal input for this function (e.g., it is a string, it is a number, it is a list of strings, it is a tuple, it is a pair <string, number>, and so on).
Writing a test is reduced to just specifying the properties we need to check (e.g., “sorting a sorted list does not change the list”).

The PBT framework will take care of everything else. Not only it will be able to test your properties using a wide range of randomly generated inputs, but it will also return the smallest problematic input in case of failure. A really useful information for debugging.

Introducing fast-check

Fast-Check is a library for Javascript and Typescript for property-based testing. It is easy to use and to integrate with existing unit test frameworks (such as mocha).

Let’s see the example code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
const fc = require('fast-check');

// Code under test
const contains = (text, pattern) => text.indexOf(pattern) >= 0;

// Properties
describe('properties', () => {
	// string text always contains itself
	it('should always contain itself', () => {
		fc.assert(fc.property(fc.string(), text => contains(text, text)));
	});
	// string a + b + c always contains b, whatever the values of a, b and c
	it('should always contain its substrings', () => {
		fc.assert(fc.property(fc.string(), fc.string(), fc.string(), (a,b,c) => {
			// Alternatively: no return statement and direct usage of expect or assert
			return contains(a+b+c, b);
		}));
	});
});

As you can see, we are testing two properties of our contains function:

A string should always contain itself.
A string should always contain its substrings.

In this snippet, we can identify three main elements.

Runners

A “runner” is the element that executes and tests a property. In the example code fc.assert is the runner. fc.assert is also the most basic runner: it executes the tests on the given property and throws if it fails.

There are more runners in fast-check (fs.check, fs.sample and others). However, fc.assert is a good solution 99% of the times. If you want to know more about runners, you can check the documentation.

Properties

The second element is fc.property. This is a function that creates the core element of PBT: a property.

In short, a property is defined by a list of input generators (called arbitraries) and a function taking such inputs.

For instance, let’s look at

1
fc.property(fc.string(), text => contains(text, text))

With this line, we are creating a property (defined with a function returning a boolean, e.g., text => contains(text, text)) with one string parameter generated by fc.string().

There is also fc.asyncProperty for properties that need asynchronous properties.

Arbitraries

Arbitraries are object used to generate the random inputs for the properties. Fast-check includes any kind of arbitraries in addition to the possibility to create custom ones.

You can have fc.string(), fc.float(), fc.json() and many, many more! You can find a complete list here.

Running the tests

Running the tests is straightforward. If you already have a mocha runner configured, you just need to do npm test as usual. It is important to know that PBT is slower that ad-hoc tests because each property is tested hundreds of times.

In case of failure you may get a message like this.

1) should always contain its substrings
    Error: Property failed after 1 tests (seed: 1527422598337, path: 0:0): ["","",""]
    Shrunk 1 time(s)
    Got error: Property failed by returning false

Not only you get an error notification but you also get a counterexample of your property (that is, an input for which your function is not correct). In this case ["","",""].

Note that, this is not just “a counterexample”: it is the smallest. The process for which a PBT goes from a failure to the smallest counterexample is called shrinking and it is THE beste feature of PBT. Maybe it could be the topic of an article by itself.

Conclusions

So, what do you think? I am personally pretty happy with fast-check so far. I hope you will be too.