Why it matters that the Oireachtas just killed KildareStreet.com

UPDATE: See the bottom of this article for an update on an effort to bring KildareStreet.com back to life. – 18th October 2012

Today was the first day back at school for the Houses of the Oireachtas. Senators got to stand up and make speeches on their hobby horses. The Government came back with the long-delayed wording for a referendum on Children’s Rights.

And, buried in all this bustle, the dead hand of the State managed to kill off KildareStreet.com, one of the brightest and best examples of what the Government is always saying it wants to see. Citizens (mostly one citizen) taking public data sets, provided in an open standard, and making something much much better than the state had ever managed.

If you have ever tried to search for anything on the Houses of the Oireachtas website, you can understand why I can confidently say it is the worst thing in the Universe. It is probably worse than that but language must fail beyond a certain level of awfulness.

The search doesn’t work and never did. You can’t link to any particular part of a debate. You can’t look for contributions by a particular Oireachtas member. Basically, you can’t do anything you could possibly imagine you might actually want to use a record of the Oireachtas debates for.

KildareStreet.com works. It does everything the Oireachtas website should always have done. It even does extra things- like let you sign up for email alerts if a particular phrase is mentioned.

And on the 17th September, the Houses of the Oireachtas just pulled the plug on the whole glorious thing. They did it without warning (though they were fully aware of KildareStreet.com’s existence and utility) and they did it would caring about the consequences for KildareStreet.com’s over half-a-million users. In doing so, they demonstrated that our State is either guided by petty minded malice or is driven by block-headed ignorance.

The magic sauce that made KildareStreet.com possible was the provision of the debates record in structured XML format. This is basically an open format common to debate records in the UK, the UK and around the world. It is owned by nobody and is available to all to write code around. It was this common base that allowed KildareStreet.com to reuse lots of the code which runs the TheyWorkForYou.com website in the UK.

As of the 17th, the Houses of the Oireachtas has just stopped producing XML. They’ve even stopped producing an RSS feed, which even we here at Tuppenceworth have had for years. From their web addresses, it looks to me that they have moved from the international open standard of XML to… Lotus Notes.

Yes. I know. Lotus Notes. Not just a proprietory format. But a really stupid one. Here’s a hint: The future should never involve the phrase “More Lotus Notes”.

We’re told that we should never ascribe to malice something which can be explained by stupidity. But I do think it is important to recognise the context in which this decision- to kill XML without debate or warning- was taken. Here’s the KildareStreet.com blog. Sample

“The lazy, incompetent fools who get paid substantial amounts of your money for not doing the job they’re paid for, in respect of publishing the Official Transcript of Dáil proceedings, have now actually surprised us.

They’re now not bothering to correct their errors at all, which is a new and unpleasant departure from their previous form, where they’d shove any old rubbish up and then quietly airbrush their failings out of existence in the ensuing day or two.”

There is a cartoon on the front page showing counting the days since the last time the Dáil Official Record was published without errors. It currently shows 237 days.

It is hardly a step too far to imagine that a bureaucracy would react to criticism from a person who is passionate about the outcome of their work by happily silencing him.

For whatever reason it was taken, we, the public have been harmed by this decision. KildareStreet.com should have been embraced by the Houses of the Oireachtas, prickles and all. It provided a plethora of services in an area where the state had simply failed. Instead, it has been stifled.

In November of 2011, the Irish Open Data week set out the opportunities for both commercial and public-spirited reuse of public data sets. (Report: €27bn public data opportunity highlighted ) After the Houses of the Oireachtas has killed off, without warning, the biggest and best such project in the country, who would bother to try again?

Any sufficiently advanced incompetence is indistinguishable from malice.

UPDATE: The indomitable sorts who run KildareStreet.com have decided not to wait around for the Oireachtas to bring back their XML feed. Instead they’re fundraising to bring their service back from the dead by building their own parser. Go to KildareStreet.com/Zombies to see more. You should throw them a few quid to support their efforts.