This allows to test that the supervisor build actually runs and opens up the possibility of running more exhaustive API tests against a working supervisor. Change-type: patch