Security vulnerabilities - the coordinated disclosure sausage mill
Laws, like sausages, cease to inspire respect in proportion as we know how they are made. – John Godfrey Saxe, 1869.
Most open source projects, Xen.org included, do what is called “coordinated disclosure” of security problems. The idea is that we keep security bugs secret until people have had a chance to patch.
Mostly this process looks serene on the outside, but from the inside it can be very messy indeed. Particularly if, as happened recently with XSA-7 / CVE-2012-0217, large and powerful corporations apply pressure to keep the bug and the fix under wraps for months while their sclerotic update processes grind on.
Many of you will already know about this vulnerability, a bug in Intel’s implementation of the sysret instruction in AMD’s amd64 (aka x86_64) processor architecture. George Dunlap has already explained the technical details. This serious problem was discovered in the context of Xen and FreeBSD on the 9th of April. The fix was originally scheduled to go out on the 1st of May but in the end was not made available to all of you, the users, until the 12th of June.
There were some other problems too: we in the Xen.org security team made some key mistakes. We didn’t involve other organisations early enough, and the patches weren’t written carefully or reviewed closely enough.
So to try to make sure that things go better next time, the team have posted a formal request for discussion about how to improve the policy. This also contains, as an exercise in Free Software / Open Source transparency, a summary of what went on behind closed doors during the embargo period.
If you’ve ever wanted to see how the “coordinated disclosure” sausage is made, here’s a glimpse into that world. Warning: it may put you off. Hopefully it will put you off using the loaded term “responsible disclosure” for something which involves keeping the majority of deployed installations exposed for months to a bug which was first discovered in 2006.
Have your say!
So, following the request for discussion there is now a thread on the xen-devel mailing list to discuss and agree on improvements.
Which way this discussion goes, and what the revised policy looks like, is largely up to you. After all, as members of the Xen community, it is you that the vulnerability process is intended to serve.
So, this is your chance to stick your oar in. We welcome the views of everyone who needs to deal with security vulnerabilities affecting Xen. That includes users, vendors, cloud providers, operators of private systems, and anyone else with an interest. We want to hear from everyone, from individuals to organisations small and large.
Please join in on xen-devel.
Mincing one’s words – an aside
So why has this bug been lurking in Xen, FreeBSD, Windows, and perhaps elsewhere – despite some people having known about it six years ago?
Intel of course have been telling the tech press, and anyone else who will listen, that this is not a CPU bug but a software bug – because they documented the bug in their version of the amd64 architecture spec (they don’t call it amd64 of course). George has explained why this is wrong. And indeed when the issue was originally discovered in the context of Linux, the people who wrote the advisories about that blamed Linux, probably because the person who wrote the original text didn’t want to annoy Intel. Just as now the issue summary on the CVE website blames Microsoft.
That’s a really nice way to make sure that other software affected by a CPU bug doesn’t get fixed. After all, who in the Xen or FreeBSD community has the resources to look up every one of the zillions of advisories about Linux and try to figure out (from reading the source code patches) whether in fact it’s a hardware bug that we also need to work around.
So perhaps if you find this posting – which contains strictly my own views, not those of the Xen.org team – rather forthright, consider this: the first step to fixing a problem is to describe it.