Poverty Reduction Blog Tag: Data
Posted on April 3, 2014 by Alicia Phillips Mandaville, Managing Director of Development Policy
It seems that everyone is talking about how data will shape our global future. It is a beautiful and unprecedented level of enthusiasm for data—bring it on! But bring it practically.
While we are getting excited about what data can do for development, let’s also get excited about what we are finally in a position to do about development data. For too long, we have lacked credible numbers about many of the things we care most about—including comparative data on governance—and now people are finally starting to take note. But what can we actually do? As a first step, MCC and Global Integrity (with support from the Omidyar Network and The William and Flora Hewlett Foundation) are convening a group of global governance data users, producers and funders who are trying to identify collective action solutions to the way the current state of play affects the availability and quality of governance data.
MCC relies on independently produced, third-party data to drive the annual process of selecting country partners for large-scale grant investments in economic growth. We are painfully aware of data gaps, especially with regard to measurements of countries' efforts to fight corruption. For MCC’s purposes, the data on our scorecard remains the best available measure of anti-corruption efforts that covers all low and lower middle income countries. However, no single data set can answer every question—particularly for something as complex as corruption.
At its heart, these data challenges are a collective action problem. Plenty of people want more and better data, but no one is really doing anything about it yet. Until the field of governance measurement as a whole is more coordinated, we will have gaps and overlaps in our collective knowledge. Heck, at this point, we still battle huge gaps and overlaps with respect to what data is available in a machine-readable format (what can I suck into my computer without first tediously rearranging and cleaning enormous spreadsheets).
The consequences of this are a big deal to MCC, but they extend way beyond us. MCC has experienced firsthand how frustrating it is to want to compare national-level education or health outcomes (like literacy or maternal health at delivery) and find the data lacking. It would be terrible if the thing that undermined global focus on governance issues was inadequate data. So we're trying something new.
In mid-April, MCC and Global Integrity are co-hosting a two day effort to convene global organizations that rely on governance data (governments and donors), organizations that produce governance data (largely third-party NGOs) and organizations that are working to enable improvements in the quality and availability of governance data (philanthropic foundations and academics). If we can get the people who have a data problem (users) in the same room with the people who can solve the problem (data providers and enablers), maybe we can make some real progress. We've been excited by the enthusiasm we've met so far.
What will come of this? We aren’t sure. But no matter what, we'll end with much greater clarity between the users and producers of governance metrics (in part through some pre-event surveys that will be public shortly), and that's no small thing. But could we see a commitment to collectively move governance data into more easily useable spaces? A sort of alliance of actors who create or rely on the quality of the information available? What would that look like anyway, and what would it do?
A first step might be as technically wonky as agreement about the formats that data gets published in. Or a public calendar of when new data is available or being used. And while that sounds unexciting to outsiders, in the long run these are exactly the kinds of first steps that enable governance data to be broadly and regularly used by more than just MCC. Whether you care about measuring, monitoring, ranking, or investing – you have to start with useable data.
This kind of opportunity doesn’t come along all the time—people are talking about and calling for data! And not just some data—global data! At MCC, we will continue to celebrate the fact that people are finally paying attention to the promise and potential of development data. That environment makes us keener than ever to get people focused on practical questions attached to resolving collective action problems in the data world. And, of course, keen to hear your perspective and ideas about next steps and solutions!
Posted on March 12, 2014 by Alicia Phillips Mandaville, Managing Director of Development Policy
Yesterday, the Center for Global Development published a data-savvy critique of MCC’s control of corruption selection indicator. They bring to bear some serious empirical analysis, and after reminding the reader that the indicator is a hard hurdle that acts as the sole difference between passing or failing the MCC scorecard for some countries, they raise a number of tough questions about why we use the data that we do. The authors point to the difficulties in measuring corruption accurately, empirical work that shows weak correlation between corruption and development outcomes and the indicator’s slow, opaque relationship with policy reform efforts—and conclude that MCC should deeply question how it can rely on this data as a hard hurdle.
I love this. Seriously.
In January, I promised I would discuss what constitutes a responsible use of data for development or foreign assistance purposes. This is a perfect opportunity to talk about the most fundamental principle: know thy data.
The CGD paper is constructive because it unpacks what is actually rolled up in the data that we rely on for the corruption hurdle—and it does so objectively and with no assertion that this is particularly unfair to any individual country. Rather, they are talking about fundamental data content and behavior. It's technical and it's detailed. It requires math. It’s the stuff most people would prefer to skip over.
But if decision making about a country rests on that data, and if you care about real progress on the measured issue itself, the math matters.
I have been working with this data for years now, and understanding what is and isn't measured—what annual composite data can and can't tell us about any one country—has been a critical part of building a holistic approach to investigating and briefing MCC’s Board of Directors on anti-corruption and accountable governance in candidate countries. That’s not unique to this dataset. What we do now is something we would need to do for any new or improved indicator measuring corruption or accountability.
Which is another reason I am glad to see this paper: It suggests alternative data sources we could look at and is upfront that none of the suggested data is yet available for every country. That isn't just a problem for us. For MCC to use a data set as a hard hurdle—or for others to seriously consider using a data set to measure progress against global development goals—that data set must actually cover all low and lower middle income countries at a decent (preferably annual) frequency. At present, very few anti-corruption measures or proxies do. That's a subject that—as people debate the possibility of a governance-focused goal on the Post-2015 Development Agenda—the world needs to come back to: Why do we still have the same predictable gaps in governance data? And it's a topic you'll hear more about from us.
In the meantime, we have built a practice around making sure MCC remains a responsible user of development data. If you look at the annual Selection Criteria and Methodology Reports, you will see that the section on supplemental information has grown over time. In 2012, we introduced a public guide to supplemental information that includes reference to country performance on international initiatives (like the Extractive Industries Transparency Initiative or Open Government Partnership) that weren't fully operational when MCC got started. And if you look at our on our approach to corruption, you will see we've built a thoughtful methodology for tracking corruption concerns.
My colleagues and I sincerely welcome the questions raised by this paper and look forward to participating in the conversation.
Posted on March 4, 2014 by John Underwood, MCC chief economist
MCC watchers pay a lot of attention to how our Board of Directors selects countries. Performance-based selection is one of our signature features—but it’s just the first step in an exacting process that MCC and partner countries undertake before taxpayer money is ever spent in the country. The process isn’t easy, and money doesn’t always flow at the end of it. But as MCC’s chief economist, I see it as a real strength of the institution.
This is what happens after MCC’s Board selects a country as eligible for assistance—based on a commitment to good governance and investing in sound economic and social policies—but before we fund projects:
1. Undertake a joint search for the most likely binding constraints to private investment and economic growth. I lead our team of MCC economists who, together with our partner country colleagues, undertake a constraints analysis. The results, informed by and tested through broad in-country consultations, enable us to jointly select activities that are most likely to promote sustainable poverty-reducing economic growth. The binding constraint in many MCC countries is in infrastructure, particularly transportation and energy. Governance issues are also common. Education comes up in several cases, notably in countries in the lower middle income category, representing situations in which countries will at best only slowly move further up the income scale and create what people want—jobs—without addressing education quantity and quality. The table below shows MCC’s country-by-country constraints analysis findings to date:
Along with the constraints analysis, countries conduct a social and gender analysis and look for private sector investment opportunities. Both contribute to the constraints analysis findings. In addition, the social and gender analysis looks for barriers that may inhibit groups from benefiting from the proposed investments. The investment opportunities analysis explores possibilities to directly or indirectly leverage private sector investment. Both provide valuable data for the next step.
Identify a program to address one or more binding constraints. The partner country, with MCC collaboration and further in-country consultation, undertakes further work to get at root causes behind the binding constraints to growth. The aim is a coherent program logic that explains how policy and institutional reform and investments will help address the constraint. MCC uses cost-benefit analysis to measure the likely impact of proposed projects. It’s a straightforward comparison of costs and benefits; the costs are the MCC-funded grants and related costs funded by the country or other donors, and the benefits are increases in incomes of the country’s targeted households and firms. MCC analyzes proposals as investments, with payoffs going to households and firms. We only include benefits when there is evidence to support the logic and look at who benefits across the income spectrum.
The cost-benefit tool allows a back and forth between country project teams and MCC to improve the cost-effectiveness of projects, notably by looking for cost savings while retaining the benefits. MCC expects projects to pass a “hurdle rate” of at least a 10 percent expected economic rate of return (ERR). As part of project preparations, the country works with MCC to set out the framework for monitoring and evaluation to help keep projects on track during implementation and for careful independent evaluations after completion.
The rigorous combination of the constraints analysis, social and gender analysis, investment opportunity analysis, program logic development, project cost-benefit analysis leading to an ERR, and planning for monitoring and evaluation helps ensure that MCC will support countries doing the right things and doing them the right way.
Selection may be the most well-known way we use evidence in our decisions, but the demanding, data-driven project development process is just as much a part of MCC’s DNA. I hope it will get the attention it deserves and ultimately benefit from receiving your input on how it is working.
Thanks to Sandra Ospina and Natalie Kottke for contributing to this post.
Posted on February 20, 2014 by Andria Hayes-Birchler, Senior Development Policy Officer
Hillary Rodham Clinton just launched a global review of data on the advancement of women and girls. The former Secretary of State (and former chair of MCC’s Board of Directors) is using her platform at the Bill, Hillary & Chelsea Clinton Foundation to partner with the Bill & Melinda Gates Foundation on No Ceilings: The Full Participation Project, which aims to gather and analyze data on the progress of women globally. I am thrilled that she is focusing on two issues of importance to MCC—gender parity and data—and hope it paves the way for more and better data across development decision-making.
The project aims to track global progress of women and girls since the 1995 United Nations Conference on Women in Beijing. In the nearly two decades since the conference, have women advanced in education? Are they serving as elected officials more frequently? What about women’s economic participation: Are there fewer women living in poverty? Have women’s wages increased in an absolute matter? How about relative to men? To answer any of these questions, one needs high-quality data and the capacity to analyze it well, and this is exactly the challenge No Ceilings hopes to tackle.
At MCC, we rely on a huge amount of third-party data for making decisions about which countries we work with, which investments are most likely to lead to economic growth and poverty reduction (and for whom) and for measuring and understanding our results. My colleagues and I are deeply interested in ensuring high-quality data exists and that development stakeholders use that data responsibly. We know how powerful data can be in driving decisions. And we know how frustrating it can be when there isn’t good data or the data is weak.
This new initiative could advance the data in development conversation; particularly since it:
- Brings accountability to global promises. In 1995, the world came together and promised to advance women’s empowerment. Without data on women’s literacy rates or incidences of violence against women, for example, it is impossible to know if there has been progress on these promises. Data help provides answers.
- Has an eye on post-2015 goals. As the Millennium Development Goals race towards their 2015 target date, the global community will need to come together towards new post-2015 goals. By highlighting where progress has (and hasn’t) been made towards women’s empowerment over the past two decade, No Ceilings has the potential to inform where the global community can best focus the next wave of commitments.
- Is likely to serve as a “gap analysis.” Although the project primarily aims to analyze existing data, it is likely to highlight all the areas where data is low-quality (or simply non-existent). By identifying the unmet needs for data, No Ceilings has the potential to inspire fresh efforts at capturing new data, much like MCC’s selection scorecard has helped development stakeholders examine the quality of global policy data over the past decade.
- Uses traditional and non-traditional data sources. With Google in the mix, it is likely No Ceilings will have access to data that hasn’t traditionally been explored by development stakeholders. I look forward to seeing if new data, indicators or ideas comes out of the data review and analysis.
More than anything, we know that for women and girls to count in economic development projects, they must be counted. Their progress in education, politics and economics must be counted. And as MCC seeks to reward governments that promote women’s economic participation—and ensure women benefit from MCC compacts—this data is a vital tool for tracking progress. I’m eager to see No Ceilings help us do just that.
Posted on January 15, 2014 by Alicia Phillips Mandaville, Managing Director, Development Policy
The start of a new year seems to prompt an awful lot of writing about how the data revolution will change everything—especially in the developing world. It will be bigger than the industrial revolution. It is already disruptive. And the applications and devices that humans can design to use this data are projected to reduce poverty, liberate people, halt the spread of disease, and alter the state-centric nature of the international system. The more disruptive the better! Vive la Révolution!
It’s easy to get caught up in this, as (full disclosure) I am. The availability of machine-readable, comparable information is already changing people’s lives in very practical ways. Data has even become less nerdy and more exciting to talk about: We can refer to “a disruptive future,” and plenty of people think that future kind of looks like an iPhone. Using technical terms in everyday professional conversations is becoming the norm. But underneath the comfortable arm waving about this bright new future, there are some quiet places that have not seen this change.
At a time where people are waxing eloquent about the power of big data to make consumer goods and services ever more tailored and ever more rapid, the world still lacks reliable, comparable country statistics on basic economic, governance and human development outcomes across much of the developing world. UNICEF estimates that one in three children have not been registered and therefore simply do not exist in statistical terms. Education outcomes are often estimated by models based on five-to-10-year-old data. As a proxy for accountable governance, budget transparency data covers only about half of the more than 190 countries in the world.
And the closer you look, the more you find that even the data we have considered reliable has internal flaws that can make it hard to trust (see Mortan Jensen's controversial book Poor Numbers). Unlike “big data”—where the law of large numbers more or less evens out the errors of any individual data point—cross-country data comparisons are typically small enough that even a handful of inaccurate data points can alter the outcome.
The first challenge here is obvious. If we want to realize the potential of the data revolution in the world’s poorest countries, we need more and better data. Period. And people are already both demanding it and trying to create it.
But there is a second, less-visible challenge: ensuring that data is used responsibly. Foreign aid and foreign assistance are fields where much of the data we want to use is just beginning to be collected or fraught with challenges. But while development professionals grapple with how to work appropriately with some serious data gaps, we are surrounded by popular examples from other fields of how reliable big data can be: Nate Silver's 2012 election predictions, Target's marketing algorithms that can tell you are pregnant before you tell your friends and even a Brad Pitt movie about data—seriously! It can be tempting to think our world is the same—but it isn’t yet.
So if we are using development data, how do we know we are using it responsibly for policy making and aid allocation? That's not an often-asked question, but I think it should be. Are there cross checking metrics? What would that even look like?! Is transparency the answer? When someone corrects a data error, how should decision makers react (à la the Reinhart and Rogoff data controversy)?
Over this year, focusing on the responsible use of data is a theme I'll come back to again and again: things worth watching and learning from, characteristics of the responsible (and irresponsible!) use of development data and efforts to fill data gaps to enhance aid effectiveness. I hope others will too.
Posted on May 16, 2013 by Sheila Herrling, Vice President for Policy and Evaluation
On April 29th at the G8 International Conference on Open Data for Agriculture, the Millennium Challenge Corporation (MCC) unveiled a new evaluation data catalog to house all the data collected through our independent evaluations. Right now, the public can view metadata from agriculture programs in Armenia, Ghana, El Salvador, and the Philippines on the catalog at data.mcc.gov/evaluations, including descriptive statistics for surveys of an estimated 5,000 households in Armenia, 9,300 households in Ghana, 1,700 individuals in El Salvador, and 2,400 households in the Philippines.
The data catalog is designed to contain all of the information that documents and describes MCC-financed independent evaluations, including information on evaluation questions, the types of surveys conducted for the evaluation and the population of interest, questionnaires, sampling methods, and descriptive statistics for household- and individual-level data. The data catalog is fully searchable down to the variable level, allowing for comparison across datasets. In addition, as microdata for each survey is reviewed by MCC’s Disclosure Review Board and is approved for public release, the catalog will host public-use datasets and statistical analyses files for replicating the independent evaluator’s results or conducting separate analysis.
The launch of the catalog is just the beginning of a series of planned data releases. We aim to release as much of our independent evaluation data to the public as possible. We’ve developed an institutional process to enable us to do this over the coming months. It is a labor-intensive effort, but that’s a small price to pay for pushing the boundaries of transparency and accountability to get this huge stock of data into the public domain. And we are delighted to be ahead of the curve on President Obama’s just-released Executive Order on Open Data Policy.
While publishing the data is a big deal in and of itself, the really big deal will come in seeing how others use it. We know – and welcome – that it will be used as another accountability check on us and our partner governments. We hope it also will be used by other investors to learn from our experience on how to increase the impact of the dollars they invest. For example, the agricultural data we are releasing may help us better understand why some farmers adopt improved practices more quickly than others, which can lead to program improvements to maximize impact, increase incomes and expand productivity.
Still, it is the unknown uses – the things we never imagined our data could be used for – that will likely prove to be the most exciting. Finance institutions, for example, looking to spur agricultural growth may gather information needed to develop innovative new products for smallholder farmers. Companies that want to evaluate the risks and benefits of operating in certain locations may find market information that is useful for evaluating risk and catalyzing new investments. Governments and civil society organizations can also analyze this data to drive forward their own complementary development and social programs.
MCC is opening our data because it is the right thing to do: American taxpayers deserve to see this part of their investment. But we are also opening our data because it is the smart thing to do. Information and data are tremendous strategic assets. They can help us enhance policies and practices to more fully contribute to economic growth, strengthen democratic institutions, improve the impact of our work, and inspire entrepreneurship, innovation and scientific discovery in the field of development and beyond. Follow our efforts and give us your feedback!
- April 2014
- March 2014
- February 2014
- January 2014
- December 2013
- November 2013
- October 2013
- September 2013
- August 2013
- July 2013
- June 2013
- May 2013
- April 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- October 2012
- September 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- May 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- September 2008
- July 2008
- June 2008
- April 2008
- March 2008
- February 2008
- January 2008