Some months ago, I was frustrated by the monotony of the task of writing, running and collecting data from experiments. I was bored of facing always the same challenges, writing always the same code and facing always the same problems. In addition, every experiment ran on different platforms, and they quickly become difficult to replicate (and this should be the very point of every experiment).

Thus, I decided to write my personal framework for running experiments in Python: YoshiX. The idea behind YoshiX, inspired by classical Unit Testing libraries, is quite simple. You write your test in a separate file, then you run yoshix and it automatically finds every experiment in a specific folder, runs them, and collects the output in the format of choice.

It is a simple tool; I still have not spent a lot of time in it. But I think it has some interesting developments. Let’s look into some details.