We do some automatic testing of the Xen hypervisor and tools branches. These tests form the “push gate” between the “staging” branch of xen-unstable (and the stable releases) and the non-staging branches.
Changes are committed to non-staging, and if the tests pass, the test system pushes them through to staging. So if a commit breaks one of the tests, the contributors don’t get to see their changes pushed and that tends of focus minds on fixing or reverting the problem commits. There’s also a cross-repository bisector which tries to automatically pin down which specific changeset is causing trouble.
I’ve been doing some work recently to disentangle this test system, which we now call “osstest”, from its supporting infrastructure. It’s now possible to use it standalone – for example, to reproduce failures which the test system has spotted, or to write new test cases.
It’s still rather rough-and-ready, but you can find the code here: http://xenbits.xen.org/gitweb/?p=osstest.git;a=tree;h=refs/heads/standalone;hb=refs/heads/standalone. (Make sure you look at the “standalone” branch; that has the disentangling I just spoke of; I intend to push it to the live production system soon.) You should start with the README of course.
If you’re interested in testing Xen I’d love it if you’d take a look at osstest and play with it. I’ve had some early feedback from Ian Campbell.
I’m particularly looking for help extending the set of test cases. Currently they are rather limited, and don’t test a very wide set of configurations and features. Adding new test steps which test particular features of Xen would be particularly useful and is a way that you can help stop the features you care about from breaking :-).
Adding a new test step, let us say one which wombats a guest, you’ll need to write a ts-guest-wombat script, probably in Perl. You can use something like ts-guest-migrate as a template, and you’ll find the functions in Osstest/TestSupport.pm useful.
If you are interested, I’d love to help you contribute. If you have questions, please ask on xen-devel at lists.xen.org and CC me, Ian.Jackson at citrix.com.