Terrific post by Giulio Quaggiotto at the World Bank PSD blog on the trend towards more publication of data, rather than or as well as information and analysis (and as well as spin). The key point is that organisations (such as government donors and international institutions) should focus on getting the data out there, rather than trying to intermediate it for their users. Giulio says:
If resources are limited, focus your efforts on making your data open rather than in producing generic “lessons learned” documents (or other knowledge management products) that have little contextual value for practitioners on the ground. In a world where SMS makes it possible to connect with affected communities even in rural areas, those products will sound increasingly hollow.
In our work on aid transparency, we’ve heard a lot of staff of aid agencies insist that aid agencies have to package the data, otherwise it will be no use to anyone. The charitable interpretation is that they want to make sure that information is useful; less positively, this impulse may come from the desire to avoid difficult questions that may arise from the raw data.
There is an excellent slide show by Chris Taggart at countculture on this latter point: the risk that open data will lead to the exposure of problems and to difficult questions being asked.
I do not have a problem with public authorities using data to present information and analysis that they think is useful and which will help build their reputation. But they should publish the raw, underlying data as well. Any services which they provide to information consumers – such as websites – should use the same data, and the same public access interface, as is available to everyone else. So if someone else wants to set up a different website, telling the story in a different way or mixing it up with data from another source, they can do so. There is no reason why the authorities should have privileged access to the data: it should be a common, universally accessible layer on which anyone can build their service or tell their story.
There is a particular challenge in publishing foreign assistance: the consumers of information want information from many different donor agencies and international organisations. In most cases, citizens in developing countries don’t want to know what a particular organisation is up to everywhere; they want to know what all organisations are up to in a particular place or on a particular topic. So information intermediaries serving these users need some way to pull together data from many different sources, and turn it into a single stream of comparable, consistent and coherent data. To a large extent information intermediaries could do this automatically, if the organisations publish enough detail about their activities to enable the data to be compared; but to some extent it requires that data is deliberately classified and structured to enable this kind of mash up. A good example is the ability to trace aid from one organisation to another: a lot of aid passes through many organisations before it arrives at its intended beneficiary, and even if every organisation is transparent about all its spending, there is no direct way to track the aid through this chain. That would need an agreed way of tagging the data so that we can all see how money flows through the system.
So for me, the key messages are:
a. publish the raw data, either instead of or alongside the information and analysis (and sometimes spin)
b. to the extent necessary, agree a minimal set of standards for the way the data are structured and the detail it contains to enable users easily to mix and mash the data so that they can use it. The International Aid Transparency Initiative has the potential to do this.
c. Aid agencies should not feel that they themselves have to meet the needs of information consumers; they should provide financial support to information intermediaries who will access this data, mix it with other data, and provide locally useful and relevant information which meet a wide range of needs. The more the donors make detailed, raw data easily available in a consistent format, the less financial support they will need to provide to information intermediaries enable them to use it.