I’m back from holiday, so here is the promised second of a pair of posts reflecting on three years of working on aid transparency.  In the first post I talked about eight lessons mainly about why different kinds of aid transparency are important.  In this post, I’m going to look at the next steps,  particularly focusing on how we can provide meaningful transparency for citizens in developing countries.

There is a lot of detail below, so for busy readers here is a summary of the proposed ten steps for aid transparency.

1. Donors cannot achieve meaningful user-centred transparency just by putting project data on their websites.  Users need information which comes from many different organisations simultaneously.  Yet it is not realistic to try to maintain lots of different manually-updated databases which collate information for users. The answer is for organisations to publish online all the information they have about aid projects and programmes, in a common, reusable format, which can then be used as the basis for user-centric databases and applications. The International Aid Transparency Initiative (IATI) is the best chance for a generation of creating such a public infrastructure for information about aid. All donors, foundations, international organisations, NGOs and aid contractors should implement the IATI standard as the key first step to meaningful, user-centred aid transparency.

2. Any organisations which do not implement IATI voluntarily should be pushed to do so by the organisations and people who fund them. For example, official aid agencies should require every organisation to whom they give a grant or contract to implement IATI as a condition of handling public money.  Citizens should refuse to put money into a collecting tin if the charity is not implementing IATI.  Governments should consider making IATI compliance a precondition for charitable status and tax relief.  Developing country governments should make IATI compliance a precondition of local registration by international NGOs.

Google has a policy that it should eat its own dogfood

3. Donors, foundations and NGOs should ‘eat their own dogfood’ – that is, any information on their website and any analysis and data that they publish about aid should use be based on the publicly available data infrastructure.  This will give the organisations an incentive to ensure that the information they make available through IATI is up-to-date, comprehensive and accurate and that the system is fit for purpose.

4. Once donors and foundations are (a) publishing their data through IATI and (b) using IATI for their own websites and analysis, they should consider (c) helping other users, especially in developing countries, to make the best use of this information. But donors’ priority should be getting their own house in order by publishing their information in a reusable format, since this is something only they can do, and using that public data infrastructure themselves, before they help others to do so.

5. One of the highest priorities for new information about aid is that all aid spending should be classified in future according to the recipient country budget classifications as well as agreed international classifications.  The Technical Advisory Group for IATI should agree the mechanism for this as soon as possible.

6.  It seems so obvious that it shouldn’t need saying, but aid would clearly be more effective if we had more information about the future plans of donors, foundations and NGOs. Homi Kharas, in Measuring the Cost of Aid Volatility, estimates that the cost to aid recipients of historic unpredictability of committed aid flows is at least 15 percent. It could be much higher. Finance ministries, line ministries, the IMF, other donors, NGOs and the private sector would all do a better job with their money if they knew what was planned by others.  Organisations should publish whatever they know about their future aid plans, generally (with some possible exceptions such as for procurement) at the level of detail they know it.   This is likely to be the hardest part of IATI for many organisations, as few have mechanisms to keep systematic track of their forward spending plans.

7. A global system of traceability in aid, enabling money to be tracked from taxpayer to services delivered, via multiple layers of multi-donor funds, international and local NGOs and private sector contractors, is less difficult and expensive to implement than you might think.  Traceability of aid would bring about a huge step forward in efforts to make aid more effective and less prone to corruption and waste, and for building public support for aid.  Done right, it could also substantially alleviate the reporting burdens of aid recipients, NGOs and implementing agencies, and reduce donors’ costs of monitoring compliance.  Priority should be given to implementing this part of the IATI standard.

8. Donors, foundations, NGOs and implementing organisations should start recording and publishing detailed geographical information about aid projects and programmes using the newly-agreed IATI standard format for geocoding of aid, and they should require their implementing partners to do the same.

9. Some donors and agencies have defined, or are in the process of defining, their own internal standardised output indicators. Organisations should now make a big effort to reach an international agreement on a common set of standardised ouput indicators to facilitate international comparability across organisations.  This information can be reported through IATI.

10.  When we connect feedback from citizens in developing countries to a rich public data infrastructure about aid, we will have a much more realistic inderstanding of the impact and effectiveness of aid. That day  is coming sooner than most of us realise.

You will doubtless think me guilty of hyperbole when I say that the emergence of an open, international infrastructure for development information has the potential to transform the development business, much as the internet has transformed so much of our society, and for similar reason.  I’m sorry that this is an absurdly long blog post, but I hope it will convince you of the amazing opportunities which are there if we seize them.

Recap: two key themes

Two themes from my previous post are directly relevant to the next steps for aid transparency.

First, transparency needs to be centred on users, not organisations.  Only a few people are interested in the details of specific institutions. Most users want to know about all the resources and activities in their country, their sector or their community. They are mainly not very interested in the distinction between aid and other sources of finance. They want comprehensive information about resources from all organisations, whether or not it is classified as aid, so that they can monitor and influence how that money is allocated and used.  This means that it is not sufficient for each aid organisation to be individually transparent: the information has to be accessible in a form which enables users easily to see in one place comparable, consistent information from dozens of different organisations which they can add up and use.

Second, transparency should focus more on execution and not just allocation. Many parliamentarians, NGOs and academics in donor countries are primarily interested in how aid has been allocated across countries, sectors and activities. They often want to ensure that donors are living up to their promises. But in our analysis of stakeholder needs, people in developing countries repeatedly told us that they want to be able to see how money is actually being used. They want to know how much money is really arriving in their country or town; how much is taken in overheads and by whom; which organisations are contracted to provide particular services and on what terms; what outputs are produced; and what difference all this makes to people’s lives.

Organisation-centred transparency

Meaningful aid transparency cannot be achieved in the way that many aid agencies previously assumed.  While it is a welcome step forward that some aid organisations are now publishing online databases of all their aid projects and programmes, this does not meet either of the two key needs described above.  Agencies’ online project databases are an example of organisation-centred rather than user-centered transparency. As anyone knows who has tried to analyse aid spending in a particular place or sector, it is not feasible for a user to trawl through the websites of dozens of bilateral aid agencies, hundreds of multilateral agencies and thousands of NGOs, to identify the relevant activities.  Even if you could assemble all the details published by different donors, you could not create any kind of meaningful overview. Every project is described in different terms by different donors, in different currencies, languages and time-frames.  Often you can’t even tell whether donors are reporting contributions to the same project or describing different projects.  There is no way to remove double counting when money flows from one organisation to another. Nor do these project databases give us enough information about execution: they do not tell us how much money has arrived in the country, how much has been given to each subcontractor or implementing agency, or what outputs and outcomes have been achieved; and they don’t tell us anything about the agencies’ plans for future projects and programmes.

The magic database?

So user-centred transparency cannot be achieved by individual organisations putting their own project databases online, because that information cannot be aggregated across donors.  An obvious alternative is to build a database, or perhaps a few databases, to bring together comparable information from a variety of sources.

I’m writing this blog post in a café in Addis Ababa which is much frequented by ferenjis.  At the next table is Gary, a Canadian consultant who has been paid by CIDA and the World Bank on behalf of the donors to build a bespoke database of donor projects in the rural livelihoods sector in Ethiopia.  Gary has done magnificent work over the last year, visiting donor offices to collect information from each of them about what they are doing, and entering it manually into his database.  It has been an expensive exercise for donors but they already think it is has been well worth the investment to be able to have an overview of all aid-financed activities in the sector.

There are at least three other aid databases which already collect information about livelihood projects in Ethiopia. Yet Gary’s database does not draw information from any of them: he has had to construct it from scratch.  Why can’t he use the existing databases?  Because one of them is not publicly available, one publishes information with a 2 year lag, and the third was built three years ago and has not subsequently been maintained.   None of them meets exactly the needs for which Gary’s database has been designed.

Gary’s story is not unusual. There are probably other consultants like him in Ethiopia working in other development sectors, and there are hundreds more like him all across the developing world.  Donors are spending a lot of taxpayers’ money on consultants to do this kind of work again and again; and donor staff are also having to supply the same information repeatedly, in slightly different form each time, to each of these databases which one of them has commissioned.

It would be nice to think we could replace all this effort by building a small number of comprehensive and authoritative databases to meet all these different needs.  That would save everyone a lot of time and money.  But sadly it is not practical to build a one-size-fits-all database that does anything.  Any database primarily serves the perceived needs of the institution that built it, at the time it was built.

The OECD-DAC maintains the two aid databases which provide the most authoritative source of aid information.  (The DAC CRS database used to be the most comprehensive too, but in this respect it has been overtaken by AidData which contains everything in the CRS database and more).   The DAC databases were built for a particular purpose: for donors to share information with each other so that they can be held to account for meeting their promises. The DAC has never asked governments or citizens in developing countries what information they need about aid, because it is not part of their mandate to meet these needs. As a result, the DAC databases are not designed to provide even very basic information needed by developing country governments and stakeholders, such as the amount of aid which is actually spent in the recipient country, or donors’ plans for the coming year.

The introduction of country level aid management system in around fifty countries has been a welcome advance in recent years; but these too have only limited use.  They are generally designed to facilitate relationships between governments and donors, and they oftern serve this purpose perfectly well.  But there are many other important information needs which they do not serve. For example, they are not usually designed to be consistent with local budget classifications, so they cannot be used by finance ministries to support domestic budget planning.   Still less do they contain the level of detail needed by line ministries, for example to enable them to plan their activites to complement the investments made by donors.  Furthermore, the majority of country level aid management systems are not accessible by the public, so they do not meet any of the needs of parliaments, civil society, the media or individual citizens to enable them to hold governments and donors to account.

The education ministry in Cambodia told us that when they asked donors for detailed information about aid projects in the education sector many donors refused to supply the data on the grounds that they had already given a lot of aid information to the Cambodia Development Council database and they did not have sufficient resources to respond to multiple information requests. But the CDC database, which is among the best in the world for what it does, does not contain the level of detail needed by the education ministry, such as where donors planned to build schools or the type of text-books they proposed to supply.  These details were not needed by the Cambodia Development Council, and so are not included in the database, but are essential for the education ministry to manage their programmes well.

This is no criticism of the DAC database, or the aid management systems in developing countries. It just isn’t possible for a single database, either globally or for each recipient country, to meet the needs of every different stakeholder. That is why we end up with many ad-hoc databases built by consultants like Gary.   But if all those databases have to be built and maintained manually, the cost is prohibitive.

In the face of a growing proliferation of requests for data for different purposes many donors have, not unreasonably, decided they have to focus on priorities. They supply data to the DAC databases, because this is the donor club of which they are members.  They also generally supply data to the aid management systems, because these databases are clearly a priority for their partners in developing country governments.  For many donors, anything else they provide is on a ‘best endeavours’ basis – some donors do what they can to provide information in response to reasonable requests, but this manual exchange of information by fax and email, or by sneaker-net (i.e. a consultant going by taxi from one office to another) is slow, patchy and expensive.

This partial access to informatioin tends to reinforce existing power imbalances.  In 2008 civil society organisations from both donor countries and developing countries met in Accra and compared notes.  Representatives from northern NGOs reported that they generally could, with some effort and sufficient time, get information they needed from donor agencies in response to specific requests. By contrast the southern NGOs reported that when they asked donors for the same kind of information they often did not even enjoy the courtesy of a reply.

Meaningful aid transparency will occur when information is available in many different forms, in the detail and form required for particuar users, often combined with data from other sources. We can’t achieve this with a single international aid database, or a single database for each recipient country.  But if we build many separate bespoke, manually-populated databases to meet the needs of different users, we are left with a nightmare of duplicate reporting and inconsistent, incomplete and out of date databases which are too expensive and difficult to maintain.

A public data infrastructure

Fortunately there is a solution which – in common with many of the best solutions in life – lies somewhere in between.

If donors, foundations, international organisations, NGOs and implementing agencies publish all the up-to-date information they have about their aid projects, online in a common, machine-readable format, then it is possible for everyone to access that information easily, and to collate information from many sources, and to add up and compare across donors.  The reusable data format makes it possible to turn donor-centred information into user-centred information.

Once we have this information in a reusable format, any number of databases  and websites can be built quickly, and can be easily maintained.  Instead of spending a year building a database of livelihoods projects in Ethiopia, Gary could have done it in an afternoon; and the information would stay current automatically.

That is why the International Aid Transparency Initiatve is so important.  It solves the problem of turning donor-centred transparency into user-centred information. It is neither a new database, nor merely an encouragement to publish project details online.  It is an international public infrastructure for reusable data.  Donors accounting for for two thirds of global aid have now said that they will, during 2011, publish their data online in this common, reusable format.  (My former colleagues at aidinfo have done a very good short video explaining how IATI works.)

This will bring about a revolution in aid transparency.  If a country chooses not to open its aid management system to the public, that won’t be a problem any longer for citizens who want to know what is happening in their country, because it will be easy for anyone to build a copy of the database and populate it with exactly the same information from the same source.

More importantly, the creation of an international public data infrastructure for development unleashes possibilities which we could not even contemplate today. It will become easy to connect aid information to other kinds of data, such as government budgets, the distribution of poverty, or feedback from citizens.  It will unlock new analysis and insights, and allow different, less controlled, more user-centred ways of increasing accountability.

That’s why the most important step organisations can take towards meaningful aid transparency is to sign up to and implement the International Aid Transparency Initiative.  The UK has already started to publish its data in the IATI format, other donors are expected to do so in the near future.   Donors, foundations, international organisations, NGOs and private contractors should all follow suit voluntarily, or be required to do so by the people who fund them.  Government donors should make it condition of eceiving a grant or a contract that the organisation must itself implement IATI.  Bilateral agencies should not put money into an international organisation or multi-donor trust fund that does not comply with IATI.  Citizens should refuse to put money into a collecting tin unless that charity implements IATI.  And charities should not expect to continue to benefit from tax relief if they are not prepared to adhere to this international transparency standard.

Dogfooding

The raw information is not, by itself, very useful.  As well as pumping out data in a reusable format, it is right and understandable that many donors will want summarise and synthesize, to highlight key trends, draw out key lesssons, and tell their story.

As Ranil said on AidThoughts:

By all means, publish the data to allow people to interrogate it themselves. But also provide high-quality, penetrative analysis in a simple, easily understood form to support their understanding. They can always go beyond that if they wish.

I largely agree with this (notwithstanding Ranil’s suggestion to the contrary).   But I would add an important qualification: the principle of ‘dog-fooding’.

Donors should be mindful that the public increasingly expects the authorities to show rather than tell. Though most members of the public will never look at the information which underpins the summaries and narratives, they will trust the summaries more if they know that the underlying information is available for anyone who wants to check it and perhaps to construct an alternative interpretation.  So while donors should be encouraged to provide easy-to-understand analysis, they should also publish the raw data which supports it.

Furthermore, in the must-read paper Government Data and the Invisible Hand, Robinson et al from Yale proposed the following principle:

The new administration should specify that the federal government’s primary objective as an online publisher is to provide data that is easy for others to reuse, rather than to help citizens use the data in one particular way or another.  The policy route to realizing this principle is to require that federal government Web sites retrieve their published data using the same infrastructure that they have made available to the public. Such a rule incentivizes government bodies to keep this infrastructure in good working order, and ensures that private parties will have no less an opportunity to use public data than the government itself does. The rule prevents the situation, sadly typical of government Web sites today, in which governmental interest in presenting data in a particular fashion distracts from, and thereby impedes, the provision of data to users for their own purposes. [My emphasis]

This is a principle which can and should be applied to aid information.  If donor agencies were to agree that the information and analysis that they publish on their websites and elsewhere would be retrieved entirely through the publicly available IATI infrastructure, this would incentivise them to maintain their IATI data up to date and in good order; it would ensure that their analysis can be reproduced (and challenged) by others; it would increase public trust in the analysis; and it would reduce the risk of inconsistency between the summaries produced by donors and the analysis done by others.

Donors who resist the principle that they should ‘eat their own dogfood’ by using the publicly available data infrastructure for their own website and analysis have to explain why they think that the information they use is sufficiently important to be included in their analysis but should nonetheless not be publicly available for others to use.

The dogfood principle is famously practised at Google, which uses its own products internally, both before and after public release, to eliminate bugs and to make sure the organisation is always aware of the limitations of its products so that they remain focused on priorities for new features and improvements.

Helping citizens to use data

The Robinson et al paper quoted above argues that priority for government should be to publish reusable data, rather than to help citizens to use data in a particular way.

Perhaps it is reasonable to expect that the wealthy, educated citizens of America, supported by the technology of Silicon Valley, will be able to interpret and government data.  But is it sensible to expect that citizens, civil society organisations and parliamentarians in developing countries will be able to do the same?

Here’s Ranil again:

In recipient countries, both civil society and the Government needs to be helped to use the data available to work out how far the aid received in total and from each country deviates from their needs, and this again needs to be backed by a real form of accountability – and this is the hardest part of all.

I agree with this, of course.  The publication of raw data is no use unless citizens and their representatives are able to use it and to exercise real accountability over their governments, donors and service providers.  This will require investment in tools and technology and in capacity and skills, and we should expect a period of only partial success while we learn what works.

Perhaps I am more optimistic than other people because I believe that, once there is an infrastructure of publicly available reusable data, people will work out to use it.  I have a great deal of confidence in the energy and capacity of people in developing countries to sieze the opportunities of freedom when they can.

Perhaps I am too cynical in observing that many of the people – and Ranil is an exception to this generalisation – who argue most passionately for donor funding of capacity building and pilot projects and the like are also people who might expect to secure lucrative contracts from such efforts.

My anxiety about putting donors under too much pressure to focus on this is that it may reduce the priority that donors give to what only they can do, namely making available up-to-date, disaggregated, comparable, information about their aid projects.   If a donor doesn’t fund work supporting civil society groups to use aid information, then someone else can fill that gap.  But if a donor doesn’t make their information publicly available in a reusable format, nobody else can do it for them.

Furthermore, the most valuable contribution that donors can make to making it possible for citizens to use aid information is to reduce the costs of accessing and using it, by making the reusable data easily and cheaply accessible.   A modest investment which sharply brings down the costs for everyone of accessing data will have a much higher return than spending the same money enabling one particular group to assemble and use information for a particular purpose.

This means that I am in favour of encouraging donors to do what they can, with funding and expertise, to enable people to use aid information to increase accountability and so improve services; but I think it is a lower priority for donors than getting a comprehensive public data infrastructure working properly.

This means that donors should:

a. First, implement the International Aid Transparency Initiative so that there is as much information as possible freely available and meaningfully accessible to everyone;

b. Then use exclusively that public data infrastructure for their own websites, analysis and publications (the dogfood principle); this will create incentives for donors to ensure that the public information is up-to-date, comprehensive and accurate;

c. Then – and only then – invest in helping citizens to use the information in different ways.

This is a lexicographic ordering of obligations, meaning that no item on the list should be considered until the obligations above it have been discharged in full.

I do not mean to claim that helping citizens to use the data is objectively less important than getting the data out there; but I am saying that it is less important for donors to address this, since other entites can help citizens but only donors get the data published.

Budget classifications

By putting in place a public data infrastructure for development (namely IATI) we have opened up almost limitless opportunities to make more information more accessible at little cost to donor organisations and data users.  So now let’s ask how we should use these opportunities.

A key priority must be to make sure that aid information is categorised according to local budget classifications.  This has been agreed in principle in the IATI data standard, and the IATI Technical Advisory Group has identified several possible options for how it might be implemented.   The TAG should be asked to come to an agreement quickly on this so that donors can make it happen.  (The reasons why it is essential to be able to read aid information alongside national budget information were were set out in my earlier post so I won’t repeat them here.)

Information about planned future aid

The DAC databases are designed to keep track of what donors have spent, rather than future plans, reflecting their primary role of allowing donors to hold each other to account for keeping their promises.  Country aid management systems usually have more forward-looking information, reflecting their function as supporting the dialogue between government and donors.  But the forward looking information they contain is frequently patchy and incomplete.

Homi Kharas, in Measuring the Cost of Aid Volatility, estimates that the cost to aid recipients of historic unpredictability of committed aid flows is 15 percent.  Finance ministries, line ministries, the IMF, other donors, NGOs and the private sector would all do a better job with their money if they knew what was planned by others.   It seems so obvious that it is scarcely worth saying, but it is preposterous that a government cannot make an informed decision about where to supply new water points because they don’t know where donors and NGOs are already planning to provide services.  Lack of information about current and future aid spending leads to duplication and overlap in some places, and woefully under-served communities elsewhere.  We miss the synergy of complementary investments (investment in agriculture together with rural feeder roads, for example), and we create uncertainty for the private sector.

Furthermore, for people in developing countries, know what donors are doing and planning to do is a critical first step in injecting some local accountability.  The people we spoke to were not mainly interested in the past: they wanted to be able to find out what is planned and how they can become involved.

This is a challenge for many donors, for two main reasons.  First, aid projects often don’t get into their systems until they are into the implementation stage and beginning to disburse.  Before that, plans are often held as less structured information – such as in planning documents or email exchanges, and these plans are rarely collected into a central repository.  Finding a way to get this information systematically into IATI is therefore a non trivial task.

Second, some donors are worried about saying too much about their plans until they have considered their options in some detail, secured high-level or political approval within the agency, obtained approval from legislators who must appropriate the funds, and reached an agreement with the host country.  Donors do not want to announce the budget for a project before they put it out to tender,  as they don’t want the bids to congregate around the budget ceiling.

Neither of these problems is insurmountable, and given the importance of forward looking information we should aim to make it a priority to address them.  Concerns about pre-empting the decisions of the legislature seems to be a case of inventing obstacles (it is easy to include disclaimers, and governments talk about future spending plans all the time).  The genuinely hard problem is logistical: most donors don’t have much of this information is a reusable form.

The IATI mechanism is designed to enable users to collate information from many different sources.  This may be the solution for some organisations who do not keep forward looking information in their management information system.  These organisations may find it most practical to publish information about actual spending from their central finance system, while decentraliszing publication of planned spending to country offices or embassies.

The people and organisations who want information about future aid plans – such as developing country partners, NGOs and civil society organisations in developing countries – are not the most powerful stakeholders, and so it is no surprise that our existing systems are not designed to meet these needs.   Our systems are mainly designed to record and report past spending, because that is what donor countries have decided to monitor, and that is what they need to report to parliaments and auditors.

Organisations should adopt the principle (proposed to us by a statistician from a donor aid agency) that ‘if anyone knows it, everyone should know it’.  Though this is the part of the agenda that may require the most administrative change,  the benefits of sharing forward looking information are potentially huge.

Traceability

This is a simplified diagram by Homi Kharas of how aid flows. There are many more layers than depicted here.

When courier companies introduced barcode systems to track envelopes and parcels, they faced non-trivial implementation costs, but the savings have been enormous.   Barcode scanning has replaced a lot of manual recording, so streamlining administration, and has enabled companies to trace missing items easily.  It has provided management information to identify bottlenecks and so drive performance improvements. It has hugely reduced the companies’ customer support costs, since customers can see for themselves where packages have reached through self-service websites.  Perhaps more importantly, the fact that customers can track packages themselves has increased customer trust in courier companies, and some larger customers have integrated the tracking information from their courier company into their own management information systems.

A system of aid traceability is technically feasible, and while the implementation costs would not be negligible, the savings would be huge.  There is a placeholder for such a mechanism in the IATI standard waiting to be fleshed out.  My view is that this should be a very high priority.

How would it work? The underlying design principle is that each organisation disbursing money would report the details of its spending and that this would include the transaction identifier (or identifiers) corresponding to the source of funds for that spending.  For example, a multi-donor trust fund would publish each item of spending and include an attribution of that particular expenditure to the fund’s various income from donors.  Some spending would be linked to particular grant (e.g. where a donor had made a grant to the trust fund for a particular purpose), and other spending would be attributed pro-rata to various general sources of funding.

Once implemented, a system of traceability would not be a significant burden on aid recipients and implementing agencies. Each organisation and agency would be asked to publish information it already has: the source of the money it spends. If this is done consistently and the data published through IATI, it would enable any stakeholder to cumulate the information across the aid system and so obtain an overall picture of where aid is actually going.  It would be possible for the first time to trace money from taxpayer, through donor agencies, trusts funds, NGOs, governments and private contractors to implementation on the ground.  It will become possible to compare overhead costs and margins, to see whether supply chains are unnecessarily long, and to establish how much money actually reaches the intended destination.

The same traceability standard would also solve completely the problem of checking whether donors are living up to their multiple spending pledges on aid – for example to create new sources of finance for climate change.  (I’m largely hostile to these spending pledges, but that is a different matter.)  All a donor would need to do is designate each spending pledge as a different “source” of their money, and it would then be possible for anyone easily to trace whether each donor had in fact spent the money they promised and to see what had eventually happened to it.  This is a much simpler and more effective solution to ensuring that spending pledges are kept than the leading alternative, which is to set up brand new global funds for the sole purpose of enabling the money to be accounted for separately.  Traceability is a much cheaper, more efficient way to track spending pledges and prevent double counting.

Indeed, a standard for traceability could greatly simplify aid management and reduce the bureaucratic burdens of the aid system.  Intermediary organisations already have to provide information in considerable detail to their donors, to enable the funders to see how the money has been used and whether it has been spent in accordance with various constraints.  An NGO might have to comply with a rule from one funder its money is not used to finance capital equipment, and a different rule from a different funder that the grant is not used to finance travel to and from the United States.  (This is a real life example.)  So all organisations in receipt of grants or contracts are already having to apportion their spending across various sources of income so that they can show they have complied with the different obligations imposed upon them in grant agreements and conracts.  Traceability would greatly simplify reporting by NGOs and implementing organisations. Instead of manually completing forms and spreadsheets for each funder, they would simply publish the details of their spending electronicallly, with all their spending attributed to particular sources of income.  Donors would be able to access the information electronically through the IATI data infrastructure to confirm that their particular grants or payments were being used by grantees and contractors in the agreed way.  This would both simplify and streamline reporting by NGOs, contractors and other implementing agents, and streamline compliance monitoring by donors.

A system of tracability would also eliminate double counting by implementing agents.  I know a former MP from Mozambique who was asked to officiate at three separate opening ceremonies for the same school in his constituency, each with a different donor as the guest of honour to view the school for which – according to the invitation – they  had paid. Each donor was able to report to its headquarters that the money had been properly used for the purpose intended, and the result was this new school.

This scam is widespread in the aid industry and without traceability there is nothing in the system which prevents it.   Traceability would make transparent where administration overheads are too high.  It would show which organisations are not disbursing money, whether through incompetence or graft, and it would narrow the scope for corruption and waste.

The system of traceability proposed here would not require a central database or a complicated new set of reporting requirements.  All that is needed is that  implementing agents should have to identify the source of each transaction in a consistent way.  Donors could simply require this in their contracts and grant agreements.  Far from adding to the workload of NGOs and contractors, such a system could greatly reduce reporting and bureaucracy. And the IATI information infrastructure is ideallly suited to enabling these fragments of information reported by many different decentralised organisations, each individually meaningless, to be added up into a overall picture which is not only useful but potentially game-changing.

Geographical coding

The Ethiopian Government – one of the poorest countries in the world – has a GIS database of all the public health facilities in the country.  But there is no equivalent information about the location of health facilities provided by donors and NGOs.

World Bank health projects in Kenya overlaid on a map of poverty levels

The technology for geographical coding has changed out of all recognition in the last few years.  Everyone with a smartphone has the technology in their pocket to record the location of a piece of a school, a well or a clinic and to add it automatically to a database.

When aid projects in Nepal were geocoded, and then compared with a map of where poverty is most acute, the donors and government found there was no correlation. The aid projects were all concentrated around the offices of the NGOs and along the tarmac roads, far away from the people living remotely in mountains whose need is greatest.

AidData has a very useful summary of the benefits of geocoding and mapping.  More and more donors are seeing its value. I’m told that when the US Embassy in Yemen geocoded its projects it rapidly found that this was the framework they used most often for planning and mointoring their aid.  The Gates Foundation has similarly found it useful to geocode all its agricultural projects.  Yet neither organisation has chosen to publish this geographical information for others to use. The World Bank, working with AidData, has geocoded all of its active projects and made this information publicly available (see http://geo.worldbank.org/).   Their Mapping for Results programme is at the forefront in the development industry.  (I thought I saw a recent announcment by the US Government that it was moving to geocoding in ten pilot countries, but I can’t find any trace of that now.)

This is an example of increasingn returns to information.  The addition of geographical information to aid data enormously increases the value of the information that is already being collected.  Geographical information offers one of the most useful and intuitive way of organising information, and opens the way to new platforms for information sharing and gathering.

It is important that as more and more donors move to geocoding information, they do so in a consistent way.  That will increase the value of the information, and reduce the burdens on implementing organisations who will otherwise find themselves having to report the same information in multiple formats.  The IATI standard does not yet include a requirement to geocode data, but it does set out a common format for voluntarily doing so.

An important step towards meaningful aid transparency would be an agreement among all donors, NGOs, and implementing agencies to geocode all their activities from now on, and to publish that information through the IATI infrastructure.

Outputs and outcomes

The discussion so far has been mainly about spending, and the need to keep track of how money is spent.  But none of us thinks that is what is ultimately important. What we all really care about is what outputs are produced as a result of all this and what difference they make to people’s lives.

Some donor agencies have realised that it is very helpful to have standardised measures of outputs within their organisation. This enables them to compare performance across projects and programmes, and so learn what works best.  It enables them to identify wasteful or expensive programmes and put more aid into the most effective programmes.  It enables agencies to estimate totals of the outputs which their work is supporting across the world, which is useful as part of their accountability to taxpayers.

When the World Bank looked at the different ways it was measuring its textbook programmes, it found a vast range of different output measures (including, in one case, a text project whose outputs was specified in metric tonnes).   When one bilateral agency put together comparable measures of textbooks purchased by different programmes in different countries, it found that the difference in unit costs between the cheapest and the most expensive programmes was substantial – a discrepency of two orders of magnitude which could not be explained by differences of circumstance between the countries.  (Sadly this analysis was never published.)   We can only make these comparisons when we standardise measures of outputs.  In practice the process of arriving at standardised indicators has been fairly boring, but they have not been particularly difficult to implement.

Common output measures would be even more useful if they were standardised internationally across aid agencies. Then we could compare the cost effectiveness of different international organisations, including comparing bilateral donors, development banks and NGOs.   We could learn not only within aid agencies, but between them.

Internationally comparable output measures is, in my view, the most important step on the road to a sensible division of labour in the aid industry.   Specialisation will only increase the productivity of the system if organisations specialise in what they are good at, and we can’t know that until we have comparable measures of their cost-effectiveness.  When it is apparent to everyone how much it costs for different organisations to provide the same outputs, there will be public pressure on the worst-performing organisations either to raise their game or to focus instead on the things which they can do better.

There are inevitably squeals of protest from the aid industry about all this.  In part this is the modern equivalent of political pressure which led to the Calico Acts: vested interests resisting any kind of competition which might undermine their privileged position. Competition reduces the producer surplus, so you should expect incumbent producers to resist it.

But there are more creditable worries about introducing standardised output measures: that they may push donors to funding what can be measured rather than what is important, or that they will undermine the principle of country ownership by defining a rigid, global idea of what is ‘good’ in development.

I don’t find either objection persuasive.

I have no difficulty accepting that there many activities in development which have rather diffuse, difficult to measure and unpredictable benefits which are nonetheless worth doing because the potential benefits are large – I have, after all, spent the last three years of my life promoting aid transparency, which is an example of just such an activity.  But I don’t think the advocates of spending part of the aid budget in these ways should expect to be funded without having to make a robust case.  If we have good measures of the benefits of alternative uses of aid – such as vaccinating children – then someone who thinks that aid should be spent on capacity building or public sector reform should produce the evidence and analysis which justifies their view.   We should not be subject to levelling down, in which we refuse to do the best we can to quantify outputs whenever possible on the grounds that it might make other kinds of investment look relatively less attractive.

Nor do I believe that standardised output measures will undermine country ownership.  Aid donors already require recipients to provide a raft of information which they say they need for their domestic accountability.  Recipient governments, NGOs and implementing agencies would be overjoyed if donors could get their act together and ask for reports on the same, rather than slightly different, measures of output.

Some donors have shown that they understand this by moving to internal standardised output indicators. Before they become too attached to these, they should make a big effort to get international agreement to a common set of indicators which they are all willing to use. This would be a big step forward towards meaningful aid transparency, especially for those people whose primary interest is in understanding what aid achieves, and not simply in tracking how it is spent.

Citizen feedback

The aid industry has relied for too long on monitoring and evaluation by so-called experts, brought in from donor countries to conduct stakeholder interviews and review logframes.  The real experts on whether an aid programme is working are the people who are supposed to be benefiting from it.

Can we link information about public services to the aid programmes which funded them?

As we implement a public data infrastructure for aid, one of the most exciting new possiblities is that this will help us to find out, for the first time, information from citizens about their experiences of how aid is used and their priorities for the future.

The Maji Matone (‘Raising the Water Pressure’) programme in Tanzania  enables  citizens to use their mobile phones to give feedback on the state of rural water supply. The information is then forwarded to the relevant government authorities – thus enabling them to respond quickly – as well as to the media.  This kind of feedback is a great way to improve public services in developing countries.  But it will do nothing to improve the quality of aid unless this feedback about services can also be linked back to the specific aid programmes that supported those services.  If the information coming from Maji Matone about which water points are working can be mashed up with information coming from donors about which of those water points they paid for, then we can find out which donors’ provide the most useful and functioning water points.  It is ironic that the part of this jigsaw that is missing is not real-time  feedback from rural water point users in Tanzania, but the necessary information from donors to connect that feedback to their aid programmes, despite the money and technology at their disposal.  When donors move ahead with detailed geocoding, and publishiing that information through IATI, a big part of this problem will be solved.

The examples so far – from crowdsourcing in disaster relief to citizen report cards in health clinics – suggest that when citizens are able to get engaged, the benefits can be enormous.

A public data infrastructure for aid creates a platform which makes this possible on a large scale. Together with with growing access to mobile phones and the internet, it will change the power dynamics in the aid industry forever.  For the first time, it will be possible on a large scale for citizens to set priorities and give feedback about what is working in development.

Conclusions

The emergence of an open data infrastructure for development has huge potential to enable us to use aid much better.  I’ve proposed ten steps to improve that infrastructure, and to begin to take advantage of the opportunities it offers.   Here they are again, in short form:

1.  Putting a database of aid projects online does not result in user-centered, meaningful aid transparency unless the information is online in a common, machine-readable, reusable format.  Donors, foundations, international organisations, NGOs and aid contractors should implement the IATI standard.

2. Donors should require NGOs and and implementing agencies to implement the IATI standard as a condition of grants and contracts.  Citizens should demand it of charities.

3. Organisations should use the publicly available data infrastructure of IATI to power their websites and for other publications (the ‘dogfood principle’).

4. Helping citizen country citizens to use this data is important, but donors’ top priority should be getting their own data into the IATI system and using that public data infrastructure themselves.

5. Aid spending should be published categorized according to recipient country budget classifications (as well as the agreed international classifications).

6. Forward looking information about aid is administratively challenging for some donors, but hugely important.  If the donors have forward looking information then (apart from a few exceptions) they should publish it.

7. Donors should implement a global system of traceability in aid.

8. All organisations should start to record geographical information, in the agreed common format.

9. We need an international agreement on a common set of standardised ouput indicators.

10.  We need to connect feedback from citizens in developing countries to this public data infrastructure about aid.

 

If you enjoyed this post, please consider leaving a comment below, and perhaps sharing this with other people using the buttons on the left. You can also sign up to have blog posts sent to you by email.

18 Responses to Ten steps for meaningful aid transparency

  • Hi Owen,

    Another great post, and my colleagues and I at Daraja appreciate the mention for our Maji Matone programme.

    On that point first, we are working on making details of the donors funding different waterpoints available through a map, and the raw data we have on waterpoints makes this pretty straightforward. Watch this space. Or rather, watch this one: blog.daraja.org.

    I also have a couple of more general comments, first on the relationship between aid/donor transparency and recipient country budget transparency. At the moment, the two sides of the coin are each being worked on extensively but they don’t seem to join up very well. IATI gets as far as looking at how to match donor budget classifications to recipient country budget classifications, which is important, but doesn’t go further to press recipient governments to make their budgets accessible. The International Budget Partnership / Open Budget Survey work comes at things from the other end. But I’m convinced that more could be done to capitalise on the natural complementarity of these initiatives. There are issues of accountability and legitimacy of course, but I’m sure the IBP would appreciate stronger donor commitments to help making recipient country budgets more transparent.

    Second, I’m concerned that the coping with the complexities of aid budgets (cf the length of your post!) means that all sorts of compromises would need to be made to come up with a system that fits everything, ending up with too many cases of square pegs and round holes. The “funding only what’s measurable” issue is part of this, but it goes further than that, potentially dis-incentivising innovative modes of aid delivery and mis-incentivising by choosing the wrong standard indicators (e.g. school enrollment rather than learning – see http://blog.daraja.org/2011/03/future-dfid-funding-in-tanzania.html). After all, one of the biggest lessons of 40 years of the aid industry is surely that we don’t always know in advance what’s important. I’m worried that standardisation would go against your “learning from evolution” argument.

    Ben

    • @Ben

      Thanks. On the issue of recipient country transparency, I find myself in a slightly ambiguous position. I’m in favour of transparency, of course, but I think that is a matter to be settled between a government and its own citizens. I do not think donors should insist upon it, not least because until they are transparent themselves it is rank hypocrisy. But I do think that aid transparency should be done in a way that can link to budget transparency so that where governments do become more transparent, the information coming from the aid sector can be easily mashed up with the domestic government information.

      On the risk of disincentivising innovation, my hope is the opposite of your fears. It seems to me that the very bureaucratic, controlling systems that many aid workers have to comply with are the consequence of poor information about how aid is being used and what impact it is having. If aid can be more transparent and traceable, and the impact can be more rigorously measured, donors are more likely to be willing to decentralise decision making and allow governments and implementing partners to innovate and find their own ways of achieving results.

      Owen

  • Another post that could have an impact on how development works. Data and information are necessary in order to be able to work in networks or to have functioning markets.

    I was thinking of using something like the Kazaa or limewire (Gnutella) network: the data reside in the standard format with the Aid agency, but can be searched and recompiled on-line.

    Your point no 2 is really fundamental: donors, governments have the duty to provide regulation in order to have a level playing field for information exchange. Indeed: the public data could replace the project document submission and results reporting. Instead of having reports sent, we could just consult them on line. The vision is clear, but implementing is not yet started.

    You could even envisage a time when donors use the on-line available project proposals for running a tender procedure.

  • Hi Owen – great post as usual. I will need to go over it in detail later, but you make a range of good points.

    I just wanted to leave a comment to say that currently, I actually do work in capacity development. Although I would like to think the causality runs in reverse – I work in capacity development because I passionately believe in it. In fact, what I do currently is help Governments collect and analyse information on aid flows, many of which have no information and do no analysis before. Obviously the aim is fully public information in the end, but even just getting better analysis from Government (who contract the vast majority of aid, and are thus need better information to improve the quality of their aid) is an improvement over the status quo.

  • Hi Owen,

    Great post! A few reactions on the AidData blog.. http://bit.ly/ianFYV

    Emily

  • Hi Owen,

    A worthy addition to your earlier post. I agree with you in principle on just about everything you say, but I would add a note of caution on putting it all into practice.

    Maybe you know this already but … while it might be easier than many people might think to do some of these things, that most such statements apply to what one might call the initial working demo. After that we get into the old 80:20 rule. We’ve got 80% of the way there with quite a small effort (bit of code), but to put in place a fully working system that can cope with all the possible permutations etc can take a lot more work. The last 20% of an automated system can be a long hard slog, and is what most systems developers spend their time on. (I used to work in systems design and development.)

    An important part of the solution is to take an iterative approach, e.g. see my post on KISI. I think such a stepwise approach (and I note you have used that metaphor yourself) will be very important in taking forward these aid transparency proposals. In particular donors should lead the way as both (a) a matter of principle, and (b) they have the resources. They should also seek to develop tools which will make it easy for others to follow. Furthermore they should focus on plucking the low-hanging fruit first (step 6 looks the most challenging to me, and I would leave it till last).

    Finally, I think there is one last challenge / benefit to tackling this. As a small NGO we struggle to keep out website up to date: it takes resources which could often be better used elsewhere. Worse as it gets more and more out of date the challenge to update it grows, further putting us off updating it. Now we want to move to a proper Content Management System to reduce this barrier. As your friendly database consultant will tell you, it is one thing to crunch through some data once in response to a specific request, but it is another thing to get it regularly updated. For that we need systems. But once you do have systems in place, then the dog-fooding principle even gets turned around: now we can do things we couldn’t do before. NGO staff might be the biggest consumers of their own data.

    But to get there we’re going to need lots of easy-to-implement tools: pre-configured spreadsheets, plugins for standard accounting tools, scripts that publish straight to a website, standard secure repositories where organisations without their own repository can publish direct to etc.

    I think its great that IATI have taken the important first steps along this journey but there is a long way to go still. It is also just as important to have a vision of where we are going, and that you have provided.

    MJ

    • Thanks MJ. I think we agree completely. This is indeed setting out a vision of where we might go, not because I think we can get there all at once, but because it is useful to have in our minds how all these different pieces will eventually fit together. The great benefit of the open development information architecture that is now being put in place is that these different pieces can be designed and implemented separately, and when they are the information they provide can be used across the system as a whole. I agree with you about the need for lots of easy to implement tools. Now that we have a data standard, it should be possible for those to be developed and used across the entire development system.

  • I tried to add the following comment to the AidData blog below the post by Emily Kallaur.

    Unfortunately the AidData blog won’t accept my comment, so I’ve put it here instead.

    Emily

    Thanks. I think we agree about almost everything, but I am not clear why you are sceptical about having donors classify their aid spending according to local budget classifications.

    I think we agree that:

    a. it is for developing countries to determine their own budget classifications, taking account of international norms but modifying them as they see fit for their own circumstances.

    b. donors should have some knowledge of the context into which their aid is arriving, including government spending in that area. So any donor doing its job properly will know what the corresponding budget code is for any planned aid spending and will have checked the government’s budget plans.

    c. the in-country staff of donor agencies are likely to have a better feel for the nuances of the budget system and of the details of an aid project than donor HQ staff, so ideally it would be them who classify aid projects.

    d. there are advantages to everyone: the finance ministry, the international organisations, line ministries, donors, parliamentarians and the public for all aid spending to be classified according to the local budget code.

    It seems to me to follow that donor agencies should plan to code the budget classification of an aid programme at the same time as they enter the other details of it (eg disbursement dates). In nearly all cases (though not quite universally) that means that the coding would be done by in-country donor staff. It would mean adding two or three extra fields to the online project form which they are completing anyway.

    I’m keen to know why you are sceptical of this. It seems quite straightforward, and proportionate to the need.

    Owen

  • Hi Owen,
    Yes, I think in many countries it might make sense for the local donor office to do this. What I really meant is that it would be difficult for a donor HQ to do this in a systematic, centralized way for all countries. In countries with aid management systems, sometimes it’s the local donor offices and sometimes it’s the government that enters data (or they agree to each enter different parts). However, even at the local level, ensuring that all donors (perhaps dozens) are interpreting complex budget classifications in the same way does take coordination and ultimately the government will have to review the results anyway. And donors who provide aid entirely outside government systems may not be aware of the nuances of budget classifications at all (obviously their aid would not be on-budget, but useful to map it for analytical purposes). So transaction costs might be lower if the government does this particular coding itself. Definitely agree that it is a critical piece of aid information regardless of the group responsible for generating it.
    Emily

    PS thanks for the reply & will try to figure out why the commenting isn’t working on our blog!

    • Emily

      Agreed. But I am reluctant to load this responsibility on to the recipient governments. If they have time and inclination to tell the donors how a particular program should be classified, all well and good. But if they don’t, donors should make the effort to figure out how the spending stacks up against the budget (whether or not the aid is delivered through government systems). That seems to be not a particularly onerous requirement on donors (certainly by comparison with the requirements they impose on others!) – donors should, after all, know something about the government’s activities in the area as part of their due diligence.

      So I stand by my starting point: donors should classify all aid categorized according to local budget classifications through the IATI platform.

      Owen

  • Hi Owen

    A really excellent set of recommendations …

    Something a bit similar, but a bit more specialized: Before I joined ILRI, I worked with a some agricultural knowledge/information people to devise a set of guiding principles for making research results (and data) truly available, accessible, applicable. In the CGIAR, we called it ‘Triple A’, something for organizations to aspire to and work towards. This morphed into http://www.ciard.net. The idea is that ALL the publicly-funded results of research in agriculture should be open and accessible. Not just ‘on the web’, but in proper systems using proper standards with open formats and open licenses that would allow anyone to harvest, view, review, and re-use the results. While there are many well-meaning efforts to help others publish their research, we think that the people producing the research need to put their own houses in order … and maybe come up with something useful that our colleagues in Ethiopia, for example, could use to do the same, or better, for their research. This is what we are implementing at ILRI – with some successes – though we still struggle to get data organized and accessible and scientists do love to publish in ‘high-impact’ (inaccessible) journals!

    By the way, when I first came to Ethiopia I also met Gary and was even involved in early conversation and planning on his database. We tried to envision his database as something that would harvest the existing systems you mention (also from systems in donor HQs, some of which we might have been able to harvest from). We found something similar in Washington that was already used in a national ministry that we could perhaps use or adapt, but it wasn’t do-able somehow. So … in the time-honored way, web people were hired to build a new system (which is the usual thing, mashing content from multiple existing systems is not business as usual and has a lot of transactional costs, in terms of processes, negotiations, and the like. Better to just do something, tangible). The result was intended to be open and to use open standards and be able to draw (and exchange) data from similar existing systems …

    I think the key is that we each put our stuff online in ways that maximize its potential re-use by others. But this is so unlike normal institutional behavior! It means giving away much control, how might it be used? what if ‘they’ don’t use my logo properly, or at all? Too many ‘what if’ questions ….

    If we could somehow measure and rate organizations on the accessibility and re-usability of their work, we might force some behavior changes. At ILRI, we would like to be known not just for the quality of the research but also for how we get this knowledge (and data and code) out in the open and re-used, but it is not something that the people who fund us, so far, have valued very highly. In general, they love well-designed logo-rich products, if possible in those high-impact journals, as this suggest our science is top quality. If only they would ask for us to demonstrate our commitment to high quality AND wide accessibility!

    Melkam Fasika !

    Peter

  • It would be interesting to hear your views on how donors should treat governments which are known to by systemically corrupt. In Tanzania, there have been no criminal prosecutions of any of the senior politicians and civil servants involved in the numerous cases of grand corruption.

    Time after time the judiciary and police top the rankings of corrupt organizations in our country. There is no doubt that billions of dollars of aid have been stolen – simply stolen. Those who control a completely broken system like this will have no difficulty at all in faking transparency.

    Donors can eat their own dog food and bar code their aid as much as they want, but vast amounts of aid will continue to disappear through hard and soft corruption. Transparency is meaningless in this context.

    As I write, government agencies are spending hundreds of millions of dollars on huge city centre office buildings in Dar es Salaam and on computer systems which will never work. Will ‘transparency’ ever reveal the waste and theft in relation to these projects.

    Donors and the World Bank continue to fund these huge, wasteful expenditures which have nothing to do with alleviating poverty and everything to do with enriching the officials involved. On top of this, multi-national gold mining companies and others pay even more in backhanders to avoid taxation. And this is before we even consider the ‘ask no questions’ activities of the Chinese, whose activities are enriching the local elites even more and who will see your proposals.

    Let’s stick with our point about systemic corruption for now, though. How do you propose to deal with governments who can, will fake transparency? They already are, of course.

  • Hi Tanzania Watch,

    Good to see you re-emerging. Tanzania’s Twitterverse hasn’t been the same without you, though it’s grown a lot since you departed. We would love to have you back.

    Why take a shot at transparency though? Surely transparency is one of the few things that can make a difference to corruption. Yes, transparency alone may make little or no difference, but it’s an important building block in the slow and difficult process of constructing an environment that exerts meaningful accountability pressures.

    Your particular focus has always been on the judicial system, which is an equally important part of a strong accountability environment. But a strong and independent judiciary is no more a magic bullet than transparency – how do you maintain an independent judiciary without strong media and parliament, without a watchful citizenry, and without transparency? So why take a shot at your natural allies?

    Ben

  • Hi Ben and thanks. Transparency, like motherhood, has to be a good thing. But as we said our major concern is whether or not real transparency can be achieved in systemically corrupt systems.

    We should emphasize that we are not arguing against aid per se, but we do have a huge problem with funding via budget support for reasons which don’t need to be spelt out. Another $500million of aid is about to be injected into Tanzania’s coffers. Believe us, most of this will be stolen, wasted or mis-used. The only reason that donors have given in is that they have their own % of GDP based targets to meet.

    Isn’t the solution to all this very straightforward?

    Until the government can demonstrate that it has fixed its criminal justice system – by, for example, locking up some of the criminals currently lounging in huge villas on the beach fronts of Msasane, Jangwane and Kunduchi – give the funding directly to organisations which are transparent and which can be held to account.

    There are many NGOs which can meet this requirement, so why do donors persist in handing hundreds of millions to a government whose ministers and senior officials make Chicago mobsters seem like saints?

    http://www.nation.co.ke/News/africa/-/1066/1162186/-/12lc5pj/-/

Follow Owen




Email subscription

I want to receive new posts by email

Owen on Twitter
Owen on Google Plus