Yesterday Andrea Leopardi released StreamData,
a data generation and property testing library for Elixir.
In 2015 at Erlang Factory Light in Berlin I attended a talk by Thomas Arts about property testing and it’s benefits,
so I was already interested and gave StreamData a little test run today.
I took the PryIn client library and replaced
some tests for which I thought a property testing based approach would make sense.
After writing my first test and trying to run it with mix test, I got an error message:
This was easily fixed by adding the following line to my test_helper.exs:
The reason this was necessary is that the PryIn client still defines an applications list in it’s
mix.exs to support Elixir versions before 1.4.
And because :stream_data is only a test dependency I did not want to add it to that list.
The next little problem I ran into was that some of my tests rely on ExUnit’s setup callback to reset
some global state. With StreamData, setup still gets called, but only once before the whole property
definition, not once for each check. To fix that, I just needed to call the reset code as part of my checks.
Not a big problem, but something to be aware of.
Writing property tests was quite easy then.
I mainly changed tests that were using constant values before to now use generated data.
Where before for example I had something like this:
Now I have this:
You can see how I replaced the constant @env module attribute with
dynamically generated values.
I also created a little helper module to hold some custom generators:
I was actually trying to come up with a generator for all possible valid strings
but that turned out to be quite difficult. For now I’m sticking to only testing
with alphanumeric strings, which StreamData provides a generator for.
So far no new bugs were revealed,
but I quite like this way of writing tests.
Besides from possibly surfacing problems,
it also just frees me of having to come up with artificial values.
If you want to check out my new tests,
you can find them in the stream_data_property_testing branch
PryIn - Application Performance Monitoring made for Elixir