today I want to continue a topic about testing graphQL queries thanks to the FsCheck library. How we could achieve that I describe briefly here.
So I want to focus on a situation when our tests spot some problems. By default, we should get in a unit test output window information about the query which caused a failed test thanks to that we could copy-paste this query and check manually what is wrong.
But what would be when our query or graphTypes are very complex and not every field is incorrect? In a situation like this we would like to get the minimum query which produced a failure, so we could leverage time needed to spend on a manual check. We could achieve that by writing a custom shrinker. If some test fails shrinker tries to change somehow an input for next iterations of a test so we could get the minimum failure query in our case.
So to see this in some example, previously we write tests for a Query and GraphType which are defined as follows:
And finally a query:
We could assume right now that
model field is resolved incorrectly. This bug was spotted by our FsCheck tests, but because we don’t implement a shrinker, query for which our test fails looks like this:
What we want is to get a minimum query which caused tests to be red, which should look like this:
To implement shrinker in our case we have to modify our test input type. Previously it looks like this:
Now we want this type to contains information about name, arguments, and fields and how to build the whole query. So it would look like this:
We modified an input test object, right now we have to go to the BuildArb function, which was responsible for building an Arb object. We have to modify it so at the end of a function based on a Gen object it would also accept a shrinker function which would be responsible for minimizing an input data.
As we could see instead of previously used
gen.ToArbitrary, right now we use an
Arb.From function which also accepts a shrinker parameter. How does shrinker function look like?
Implementation of a shrinker is pretty straightforward. The first thing that comes to mind is a return type which is an IEnumerable of QueryTest instead of simply QueryTest. This is because we create k combinations of an input against which our test would be run in n shrink phases. If we go deeply into implementation we could see that we want to generate some “shrunk examples” till the query would contain a single field (graphQL expect at least one field in a query). Then we go through all fields and we remove one of them. I think this example would explain it.
As we remember the field
model in a
car was resolved incorrectly. FsCheck produces following test case in the first phase of a test (shrink = 0). The test was run against the following query.
In the next phase (shrink = 1) tests were run against:
next phase (shrink = 2):
As we could see after two phases (shrink = 2) we could gather a minimum query which produces failure:
Of course, a solution like that has some drawbacks. For example what with very complex fields? With such minimal implementation of a shrinker, we could get a query with a single field, but this field could have a lot of fields for example:
Although in our case such simple implementation is enough and it helps gather an information what exactly is wrong and also leverage the time needed to spend on manual checking of a query.
Thanks for reading! :)