How to Measure Openness?

16 November 2005

Towards an International Index

Government transparency ranks as a fundamental human right and an essential element in developing effective democratic governance. Nonetheless, international freedom of information advocates face a daunting challenge in quantifying and evaluating government openness and access to information in different nations.

A wide range of researchers have pioneered the development of indexes for measuring and assessing openness around the globe. Some of these efforts use surveys and polls to establish the subjective perceptions of individuals, advocacy groups, and officials about their own and other governments. Others combine evaluations of experts, researchers, and journalists, observing as outsiders the legal and practical realities of each society. Several more economically-oriented approaches examine aspects of regulation and corruption by looking specifically at market and business-related factors. Still others, including Sheila Coronel in The Right to Know: Access to Information in Southeast Asia, rely on a set of objective criteria tested by approaching directly the government being measured.

This debate will be critical as civil society groups and democratic advocates around the world move forward in appraising progress and demanding reform on openness issues in their own countries and in international institutions. By laying out some of the options that are now available to tackle this important problem, freedominfo.org hopes to begin a conversation that can lead to productive contributions and an ultimate consensus on quantifying the successes and failures of freedom of information around the world.

World Bank

Over the past six years, a number of researchers at the World Bank Institute (WBI), the World Bank’s research arm, have evolved a model of aggregate governance indicators to assess the quality of governance in a manner more precise than the anecdotal evidence on which most policy-makers ordinarily rely.

  • Daniel Kaufmann, Aart Kraay, and Pablo Zoido-Lobaton “Governance Matters” and “Aggregating Governance Indicators,” companion paper (October 1999)
    In 1999, Kaufmann, Kraay, and Zoido-Lobaton derived their method of aggregate governance indicators in an attempt to combine the numerous cross-country indices that were then being used to measure different aspects of governance, with the advantage of thereby being able “to quantify the precision of the both individual sources of governance data as well as the aggregate governance indicators.”Definition of Governance: the set of traditions and institutions by which authority in a country is exercised, specifically:

    • The process by which those in authority are selected and replaced
      1) VOICE AND ACCOUNTABILITY
      2) POLITICAL STABILITY AND ABSENCE OF VIOLENCE
    • The capacity of government to formulate and implement policies
      3) GOVERNMENT EFFECTIVENESS
      4) REGULATORY QUALITY
    • The respect of citizens and state for institutions that govern interactions among them
      5) RULE OF LAW
      6) CONTROL OF CORRUPTION

    In analyzing the empirical data they collected, the authors focused on the six aggregate indicators listed above, representing basic principles of governance. Each of the indicators used are subjective, measuring the perceptions of individuals as to the quality of governance. This data is derived from two different types of sources: polls of experts, which represent country ratings from commercial and other organizations; and surveys of citizens, conducted by international organizations or other non-governmental organizations.
    The authors admitted the drawbacks of their model, in particular that a broad range of different types of sources and variation among the indicators generate imperfect conclusions, as does the fact that certain of the indicators only cover a particular sample of countries–some larger, some smaller or more focused–and that different units of measurement are used in each source. They conclude that, despite their initial instincts to the contrary, governance cannot be measured with precision and the statistically significant differences, particularly among the majority of countries that fall in the center of the spectrum, make it difficult to confidently distinguish the countries. Nonetheless, aggregate governance indictors can be useful for translating various data sources into a coherent framework, and, although still imprecise, the aggregate indictors cover a larger sample and are more reliable than any one indicator alone. From their study, the authors conclude that governance matters: they show “empirical evidence of a strong causal relationship from governance to better development outcomes.”

  • Kaufmann and Kraay, “Growth Without Governance” (2002)
    In a subsequent study entitled “Growth Without Governance,” Kaufmann and Kraay looked specifically at a feature of corruption as it affected the governance indicator data, predominantly in Latin America and other transition economies: the phenomenon of state capture–which occurs “[i]f the fruits of income growth largely accrue to an elite that benefits from misgovernance, then any possible positive impact of income growth on governance could be offset by the effect of the elite’s negative influence.” The implications lie in matters of reform, and particularly in identifying and targeting of country-specific needs which, in the case of states where growth is inhibited by capture, require “much more emphasis must be placed on promoting mechanisms of external accountability, voice, participation, and transparency.
  • Kaufmann and Kraay, “Governance Matters III: New Indicators for 1996-2002 and Addressing Methodological Challenges”
    Kaufmann and Kraay updated their research in 2003 with “Governance Matters III: New Indicators for 1996-2002 and Addressing Methodological Challenges.” The later reports greatly increase the number of data sources as well as the number of countries included in the study, including up to 199 countries for the 2002 data. By means of illustration, the 2003 paper looks at the U.S. Government policy on the Millennium Challenge Account (MCA), which announced rules for allocation of funds according to governance-based factors, including the degree to which potential recipient countries “govern justly,” “invest in people,” and “promote economic freedom.” Kaufmann and Kraay in their conclusions caution against attempting to group countries based on levels of governance rather than viewing them in relation to one another, because of the relative imprecision of the indicators. In addition, the authors assess for the first time any trends over time appearing in their aggregate data covering 1996-2002, and conclude that although “[i]nterpreting these trends is difficult … we can state with some confidence that there is little if any evidence of improvements in global governance over the period we consider.”
  • Roumeen Islam, Do More Transparent Governments Govern Better? (June 2003)
    Islam sets out to delve deeper into how the availability of economic information and the legal framework for access to information affects the quality of governance in a given state; she further examines how restrictions on information–particularly limitations on media freedom–can reduce the quality of governance. Building on scholarship examining the way in which available economic data improves the function of economic markets, Islam creates two new mechanisms: a “Transparency Index” and an “Access to Information Index.” The Transparency Index looks at 11 representative variables from real, fiscal, financial and external sectors and evaluates each of 169 countries in terms of the availability of economic data in comparison to an ideal, or “desirable,” frequency level. She subsequently looks at an Access to Information Index, based on the existence of a FOI law. It is unclear from the methodology presented whether she considers any factors beyond the existence of statutes, although she discusses in her analysis the importance of subjective factors–including implementation and FOI practice–in the question of who governs. Islam concludes that both the transparency index and the access to information index show a positive correlation with the quality of governance (based on KKZ) and, therefore that “there is a close relationship between better information flows and how fast economies grow.”


Center for Public Integrity

The Center for Public Integrity is a nonprofit, non-governmental organization that sponsors and promotes investigative journalism and reporting on public policy issues in the United States and around the world. The Center receives funding from the Open Society Institute (OSI), among other organizations and individuals.

  • As part of its Global Integrity Project, launched in 2001, the Center produces in-depth country reports, focused “on the existence and effectiveness of mechanisms that prevent abuse of power and promote public integrity, and on the access that citizens have to their government.” Reporting for the country reports is conducted by in-country teams of independent social scientists and investigative journalists who observe both legal restrictions and practical realities of access and transparency on the ground.
  • The Center has also developed a quantitative assessment in the form of the Public Integrity Index. Countries are rated based on six broad categories: Civil Society, Public Information and Media; Electoral and Political Processes; Branches of Government; Administration and Civil Service; Oversight and Regulatory Mechanisms; and Anti-Corruption Mechanisms and Rule of Law. These categories are further broken down into 80 separate indicators and 238 subindicators, which together seek to quantify the existence of public integrity mechanisms, the effectiveness of these mechanisms, and citizens’ access to public information and ability to hold their government accountable. (View the Indicators) The question of access to information falls within the first major category and includes two indicators: whether citizens have a legal right of access to information, and whether access to information functions effectively in practice. Subindicators focus on the legal availability of government records and the opportunity of appeal for government denial of access.

Freedom House

Freedom House is the oldest nonprofit, non-governmental organization dedicated to monitoring and promoting human rights and democratic freedoms around the world. Founded in 1941 by Eleanor Roosevelt and others, Freedom House has been publishing its standard-setting comparative survey on Freedom in the World since 1972. Freedom House has also been conducting a global assessment of press freedom annually since 1980.

  • Freedom of the Press 2005
    Freedom House’s annual survey of media independence covers transparency and, in particular, considers as part of its methodology whether states have freedom of information legislation and, if so, whether journalists are able to use it to gather information from the government. The data comes from overseas correspondents and traveling staff, findings of several other human rights organizations, geographic and geopolitical specialists, reports from governments and multilateral bodies, and various domestic and international news media.

    The Freedom of the Press survey calculates numerical scores for each country and, based on this score, designates countries as either “Free,” “Partly Free” or “Not Free.” The total score is given out of 100 points, and includes 30 points for criteria related the Legal Environment, 40 for the Political Environment, and 30 for the Economic Environment; lower scores indicate greater freedoms. Within the legal category, 2 points may be awarded based on the question, “Is freedom of information legislation in place and are journalists able to make use of it?” Under the political criteria, there is also the potential for 2 points to be given based on the question, “Is access to official or unofficial sources generally controlled?” In some of the individual country reports presented, there is mention of the role of access to information laws and practices with regard to journalistic freedom.

  • Freedom in the World 2005
    Freedom House also publishes a survey of political rights and civil liberties annually. Each country is given a numerical rating and a freedom designation based on two categories, political rights and civil liberties. This survey does not attempt to rate governments or government policy, but rather “the real-world rights and freedoms enjoyed by individuals,” and therefore reflects more broadly the interaction between governments and nongovernmental actors.

    The methodology of Freedom in the World is largely based on established standards in the Universal Declaration of Human Rights. A team of regional experts and scholars evaluates a broad range of data from governmental sources, non-governmental organizations, and the media. Within the category of Political Rights, the survey checklist covers Electoral Process, Political Pluralism and Participation, and Functioning of Government (which includes the question, “Is the government accountable to the electorate between elections, and does it operate with openness and transparency?”), and under the heading of Civil Liberties the inquiry considers Freedom of Expression and Belief, Associational and Organizational Rights, Rule of Law, and Personal Autonomy and Individual Rights.

PricewaterhouseCoopers / Kurtzman Group

  • Opacity Index
    This index measures countries’ degree of opacity–“the degree to which they lack clear, accurate, easily discernible and widely accepted practices governing the relationships among businesses, investors, and governments, which form the basis of most small scale, high frequency risks.” The Opacity Index compares 48 countries based on 65 objective variables and data from 41 different sources. The methodology measures opacity based on five distinct components: business and government corruption (costs from corruption, based on Transparency International and other sources); ineffective legal system (overall function of legal system, effectiveness in resolving disputes and protecting businesses); economic costs of doing business (including losses from bureaucratic red tape, non-transparent taxation, and costs from organized crime and terrorism); inadequate accounting and governance practices (degree to which accounting and banking laws in accordance with international standards); and harmful regulatory structures (security of capital investments).

    The final score is derived by rescaling all of the above categories and taking the simple average of the five sub-indices; separately, the scores for each individual sub-index can show companies where the threats lie in each country and whether they might be able to seek legal redress. The writers of the Opacity Index conclude that, which few exceptions, “higher levels of opacity strongly correlate with slower growth and less foreign direct investment in all markets.” In fact, the analysis shows, for every 1-point increase in a country’s opacity based on the Index, per capital increase falls by $986.Download the Opacity Index 2004

Transparency International

Transparency International (TI) is a non-governmental organization dedicated to combating corruption on a global scale. Internationally, TI “raises awareness about the damaging effects of corruption, advocates policy reform, works towards the implementation of multilateral conventions and subsequently monitors compliance by governments, corporations and banks.” The TI network also includes more than 85 national chapters around the world that monitor institutions and advocate reforms within countries.

  • In the 2003 version of its annual Global Corruption Report, Transparency International focused its analysis on the right of access to information as an integral part of the struggle against corruption. The report includes regional assessments, data and research, and shorter reports on different right to information issues.
  • TI’s conclusions are based in part upon its Corruption Perception Index (CPI), a composite index first introduced in 1995 as a means to “captur[e] the degrees of corruption perceived by international and domestic business communities.” The CPI draws on a number of different surveys from institutions that poll individuals and decision-makers about the extent to which they believe public power is misused in their countries.

Inter-American Commission on Human Rights

  • Annual Report, 2003 (Office of the Special Rapporteur for Freedom of Expression)Within its Annual Report, the Office of the Special Rapporteur includes a “Report on Access to Information in the Hemisphere.” This section outlines the IACHR’s framework for the establishment of access to information regimes that comply with the requirements of the convention on access to information. In addition, the report examines the status and success of access to information laws and practice in the member countries.The findings in this survey are based on official questionnaires issued to each of the permanent missions of the OAS member states, requesting information about constitutional and legal provisions as well as implementation and actual procedure relevant to access to information; the analysis also includes some additional research by media organizations and NGOs. In particular, the questionnaire asked about constitutional provisions, laws, regulations, and pending measures that protect or restrict the right to access information; jurisprudence of tribunals allowing or denying access to information; campaigns to educate the public about the right to information; and systems for submitting requests for public information. Although the resulting report does not draw any subjective conclusions about the standing of access to information rights in the IAC countries, the objective data provides a valuable guide for using official sources to evaluate information freedoms.Visit the Inter-American Commission on Human Rights website

Philippine Center for Investigative Journalism/ Southeast Asian Press Alliance

  • Access to Information in Southeast Asia (Sheila Coronel, ed.)
    Buy the book

    In the book Access to Information in Southeast Asia, published in 2001, editor Sheila Coronel presents a survey of eight countries in Southeast Asia that indexes and compares the availability of public records in each country. The survey looks at the freedom of information laws and the accessibility of 45 different categories of public records in each country, and on this basis ranks the countries according to their openness. The appendices of the book include a cross-country comparison of access to various types of records, as well as country-by-country summaries (for the eight Southeast Asian states) that include the same categories, but also describe the agency or government body that maintains each type of record and the availability of such records from non-governmental sources.The categories of records assessed include:
    1. Macroeconomic data (e.g. GNP, GDP, balance of payment, current account deficits)
    2. General social data (e.g. literacy rate, poverty rate, infant mortality rate, employment rate)
    3. Population census data
    4. Environment data (e.g. forest cover, air/water quality, coral reef destruction)
    5. Copies of laws
    6. Copies of government directives & circulars
    7. National government budget (revenues and expenditures)
    8. Local gov’t budgets
    9. Military expenditures
    10. Gov’t loans & contracts
    11. Military loans & contracts
    12. Official audit reports of gov’t agencies
    13. Records of congressional or parliamentary proceedings
    14. Reports of official investigation on the conduct of gov’t officials
    15. Police investigation reports
    16. Military/police intelligence reports
    17. Credit investigations
    18. Court records
    19. Resume of gov’t officials
    20. Bank records of gov’t officials
    21. Election contributions & expenditures
    22. Registration of other forms of property of gov’t officials (aircraft, yachts, cars)
    23. Financial disclosure reports that show assets and liabilities of gov’t officials
    24. Corporate registration records
    25. Financial statements of publicly listed companies
    26. Financial statements of companies not listed on the stock exchange
    27. Corporate tax records
    28. Business licenses & permits
    29. Civil registry records (e.g. birth, marriage, divorce, death records)
    30. Gov’t service records
    31. Military personnel records
    32. Academic records
    33. Land registration records
    34. Real estate tax records
    35. Licenses & permits (license to own and carry firearms)
    36. Vehicle registration
    37. Driver’s license
    38. Alien information (e.g. date of entry, manner of arrival; addresses; occupation; age)
    39. Voter registration records
    40. Medical records
    41. Income tax returns
    42. Industry or professional listings/directories (e.g. bar associations, chambers of commerce)
    43. National ID records
    44. Professional licensing record
    45. Civil service exams and related information

    View the matrix:
    “Availability of Public Records (A Cross-Country Comparison)”

Be Sociable, Share!
  • Facebook

Tags: , , , ,

Filed under: Latest Features