Martin Tisne, program manager for the Transparency and Accountability Initiative, a research effort set up about a year ago by eight major funders to inform their work, suggested May 20 that considerable additional research is necessary, not only to understand the impact of FOI laws, but to know how change occurs.
His remarks sparked some reaction around the costs and burden of evaluation work on project sponsors. He spoke at The First Global Conference on Transparency Research held May 19-20 at Rutgers University-Newark, N.J. (See related report in FreedomInfo.org.)
We understand that transparency can contribute to greater responsibility for the state, lower corruption, build engagement spaces, and improve budgets, he said, “What we really lack is: under what conditions that takes place.”
The evidence of impact is “very uneven, very scattered,” Tisne said, with a lot about the effectiveness of intervention, not much assessing impact or examining what makes more information accessible and useable. “We’ve been guilty as assuming that information will lead to accountability” and that accountability will lead to better governance.
The big gap, he said, “… is how we can help better articulate and understand theories of change….” Very few organizations articulate their assumptions about how their projects will effect change, but they should, he recommended.
Other areas of need, Tisne said, include finding the best research methodologies that will help identify the best projects, establishing good baselines, embracing “complexity aware” mixed methods, and doing long-term studies. More attention also needs to be paid to what the research methodology is for, he said.
More Documentation Needed
Eszter Filippinyi, program officer on right to information for the Open Society Foundations, said the emphasis needs to include not just the accessibility of government information but also “what we do with this information.”
Praising the academic research, she noted as an example that one study of the political forces around passage of FOI law might influence future strategic choices.
She said activists have learned a lot in their work, “but not a lot has been documented,” and she encouraged more collaborations between activists and academicians. Further work on theories of change could help civil organizations that often rely on assumptions. The examination of problems after passage of laws is necessary, she said, citing the lack of enforcement of pro-transparency decisions in Peru. A lot can be learned from comparative experiences, she said.
Funders of transparency projects generally are moving to a more sectoral approach, asking how it can tie in with work in the areas of health, education, natural resources, and the environment, according to Filippinyi.
Reactions Highlight Complications
Commenting on their remarks, Yamini Aiyar, Director, Accountability Initiative, India, noted that monitoring impact with information from the community can be unreliable. She said the exercise of writing a theory of change for a project was difficult but useful.
Theories of change, said Archon Fung, of the Harvard Kennedy School, don’t always bear out, so a variety of hypotheses can be tested. For example, he observed, the theory that “armchair analysts” would use the considerable data put up about U.S. government spending to stem the economic crisis was “flat out false.”
Burden Seen
Laura Neuman, Associate Director of The Americas Program at the Carter Center, Atlanta, Ga., voiced the concern that donor reporting requirements have grown to be quite burdensome and that money for evaluative research is hard to get. The additional work that Tisne suggested would be “incredibly onorous,” she said.
Tisne said he was looking for short descriptions of a change strategy, that donors hope to encourage a culture of learning and to establish long-term relationships of groups and academic researchers. “We’re not looking at tick boxes; we want more change, change at scale, and we want impact,” he said.
Case Studies Criticized
Sanjeev Khagram, a professor at the University of Washington’s Schools of Public Affairs and International Studies, commented that most existing case study research is “woeful and inadequate.” The promise of more collaborative activist-academic work would be to form a collective research exercise that “… may be able to tease out” causal factors. “If we continue to do it in this piecemeal way, we don’t add up to something greater,” he said, suggesting that academics should be “embedded” with projects “on the front lines.”
David Miller, a professor of sociology at the University of Strathclyde, said academics don’t look at impact and suggested the need for more “activist scholars.”
Greg Michener, a political scientist from Brazil, said academics need to share their research more with others, for example, taking advantage of their legitimacy to reach out to potential FOI supporters such as those in the business community.
Marcos Mendiburu, World Bank Institute FOI expert, added a ”note of caution,” stating that it can be hard to find at researchers in developing countries and hard to gather data.
Filed under: What's New