One of the stated goals for 2013 and 2014 of the Xen Project Advisory Board is to Increase upstream Xen Hypervisor quality including the quality of its latest CPU and Platform features and to address problems with the code in a timely and proactive manner, including defects, security vulnerabilities and performance problems.
OSSTest and Coverity Scan
Earlier this year, first steps have been taken by making available the code for the OSSTest system (a smoke test system, which has been running since 2010). As a second step the Advisory Board worked with the Xen Project Technical Coordination Team, to add the Xen Hypervisor to the Coverity Scan static analysis service, that is open to open source projects with nonprofit status. There is still some detail to sort out on how we grant access to this resource: you can find the ongoing discussion here.
XenRT as basis for a comprehensive Test-as-a-Service Platform for the Xen Project
In the kick-off meeting for the Xen Project Advisory Board in April this year, Citrix offered to open source XenRT, the Citrix XenServer automated test system. This created the opportunity, to establish an independently hosted and expansive Test-as-a-Service Platform for the Xen Project.
XenRT began its life 8 years ago testing the Xen hypervisor and XenSource’s XenEnterprise product in 2005 and stands for Xen Regression Test and has evolved from an automated test system comprising a few tens of automated test cases running on a few servers, to a highly sophisticated Test-as-a-Service Platform that is used by the XenServer engineering group to validate XenServer. Tests are run in a huge distributed lab comprising a wide variety of hardware types. A team of twenty-five developers in the UK and India spend the majority of their time either maintaining, operating or developing new automated test cases for XenRT. A good introduction to XenRT can be found in this blog post.
As of yesterday, the code for XenRT has been made available to the community for review (see announcement). The following resources are of interest if you want to know more:
- README document
- Main XenRT tarball
- Third party test resource tarball
- Source for third party resources (not required for normal operation)
- Mailing List
This means that phase 1 of establishing a comprehensive Test-as-a-Service Platform for the Xen Project is complete. I wanted to thank Citrix for making this happen. From an architecture viewpoint, XenRT can be used to validate the Xen Hypervisor, Xen on ARM, XAPI and Mirage OS. Note that some of the plumbing to do this has not yet been implemented. Creating a Test-as-a-Service System for the Xen Project is no trivial task and will also require funds to operate.
Xen Project Test Framework Working Group
To ensure that we can get such a system off the ground, the Xen Project Advisory Board has agreed in the August meeting to create a Test Framework Committee and make funds available for initial implementation work (or phase 2). The committee will initially be staffed by employees from AMD, Amazon Web Services, CA, Calxeda, Citrix, Intel and Oracle (others may join later). Of course, the Test Framework Committee will work closely with the Technical Coordination Team, project leads and wider community to ensure that community members can extend the test system (i.e. add new test cases, control what is in daily and release builds, etc.) and that sub-projects can make use of the system. Do expect some sessions related to XenRT and the proposed Test-as-a-Service system at the Xen Project Developer Summit.
Do remember, that it will take some time to implement a Test-as-a-Service. As such, it is important to continue adding test cases to OSSTest (and it is quite possible that OSSTest and the new system will be run in parallel).