Surveys as Tools

By
Oct 3, 2003, 18:02

Email this article
Printer friendly page

TI Source Book 2000

Chapter 28: Surveys as Tools--Measuring Progress

Are we winning? Are we losing?
I can't tell,
I just keep punching
'til I hear some kind of bell...
-- Rock Follies, 1977
Source Book Table of Contents Previous Chapter Read .PDF Next Chapter Image to Access Table of Contents, Previous and Next Chapter, or Download .PDF File

Corruption is perhaps the most under-reported crime there is. Kevin Ford has observed that:

[An act of corruption] is generally conducted in great secrecy. All parties involved in the immediate transaction (the bribe taker and the receiver) are usually satisfied with the result and recognise the possibly very negative consequences of revealing their own role in such criminal conduct even if they are not satisfied. Meanwhile the victims of corruption, which are usually the general public and society at large, are either blissfully unaware of specific acts of corruption or so inured to such corruption that they have become indifferent to it. [1]

Given such secrecy and commonality of interest among its perpetrators, corruptionlevels are extremely difficult to measure. There is no such equationas the one that enables us to assess the size of an iceberg by the volume ofthe tip that we can see protruding above the waves.

Yet without measurement, it becomes extraordinarily difficult either to assessareas of particular difficulty or to ascertain whether reform programmes are,in practice, having a positive effect. Surveys of some description are nowrecognised as an essential tool in the context of containing corruption andillegal conduct, however fraught with difficulties they may be in their designand execution.[3]

It is, of course, unrealistic to expect people to admit to criminal or evensocially undesirable conduct in the course of a survey, if at the same timethey are exposing themselves to risk. There are, however, some categories of people that canbe surveyed effectively, those where individuals see themselves as "victims" rather than aswilling conspirators, as being exposed to "extortion" rather than as participatingin consensual corruption.

Thus, international business (despite its proven role as a prime moverin grand corruption)[4]has been willing to be polled. So, too, have localbusinesses, and ordinary people (as customers of government services).However, although these groups of people know something about corruption levels, thisknowledge, beyond that based on their immediate experience, is decidedly unreliable andreally falls into the area of "perceptions".[5]

As a general rule, the broader and the less specific the survey, the more contentious its results.The fact that perceptions can lag well behind realities should also be taken into account, especiallywhere things are in reality getting better but public perceptions are based on anecdotalexperiences from the past.[6]

Surveys, too, can give credence to what may appear to be extravagant claims, or assertionsthat the respondents are not equipped to make. For example, in a recent national poll in thePhilippines it was reported that over half the respondents claimed that more than 50 per centof the funding spent on roads was wasted. Nearly two-thirds thought that more than 40 percent of the funds involved were lost in the process of collecting taxes, 30 per cent in providingfree books to children in schools and 23 per cent in installing modern equipment in governmentoffices.[7]This is not to say that the claims were necessarily wrong, but it seemsunlikely that so many of the respondents were in possession of evidence that would establishthese losses as fact. They were, and could only be, "perceptions".

On the other hand, there is the ever-present question of who it is who "owns"the data? If the results of a survey are less than flattering, and if the surveyis "owned" by the agency whose performance is being criticised, the temptationto suppress--or worse still, to massage--the results can be irresistible.This temptation can prove to be overwhelming where an agency is given"performance targets" to reach, and where the survey data will be used tojudge its management's performance. It is therefore highly desirable thatindependent people be involved in the design and implementation of surveys,and that the results are publicised and "shared" with the community whichhas been surveyed. It is the honest and professional collection of data coupledwith the subsequent transparent presentation of the results that give the citizen "a voice".

The results of surveys should--as an important element of a national anti-corruption actionplan--be publicised and put on web sites, rendering the results available to a wide public andpresenting a clear challenge to the political will of a country's leaders.

International surveys

International corruption perceptions surveys have been pioneered by Transparency International, but not without a degree of controversy. As TI has always acknowledged, these surveysare of perceptions, not necessarily of reality, and have value as they reflect the views of theinternational business community, whose investment decisions impact on a national economy.The higher the perceived level of corruption, the less inclined are foreign investors to make nomore than very short-term (and high return) commitments to a country.[8]

In particular, TI's Corruption Perception Index (CPI) has served to highlight the issue of corruptionin a number of countries, and is credited with raising the level of attention accordedto the problem by some governments. It has also been strongly criticised for being "anti-South",as focusing on only one side of the problem (the bribe-takers as opposed to their internationalbribe-givers), and as giving, at times, a depressing message to governments who aremaking sincere efforts to combat corruption. The methodology developed to amalgamate aseries of different surveys has been under continuing review and has not yet fully achieved aconsensus among experts in the field.[9]Yet, the fact that countries must be covered by at leastthree different surveys means that the resulting index itself is comparatively robust.

The CPI sends out a simple message to governments who fare poorly. Whether the results bejustified or not, the Index reflects the perceptions of business people who are making dailydecisions that directly affect their countries' economies. Action is called for, either to dispelthe misunderstandings, or to remedy the complaints. In the words of The Economist, governmentsignore the results at their peril.

The CPI has also helped address the "them and us" syndrome: a tendency by some developingcountries to blame only the foreigners for their domestic ills. By focusing on corruption "athome", the CPI stimulates domestic debate and fuels programmes to combat it.

A second, more recent "Bribe Payers Index" (BPI), was conducted by the internationally-renownedGallup International on behalf of TI in 1999. It surveyed a cross-section of élites ina number of emerging markets to ascertain, from those most likely to know (businessmen,bankers, lawyers etc.), from where the bribes were most likely to come. Thispainted a depressing picture of the involvement in corruption of the exportersfrom many of the world's leading exporting countries. The data from this surveycan be regarded as being very reliable since it targeted a specific éliteaudience, and an audience that is prepared to be surveyed again to measurethe extent and pace of any changes.[10]

National and sectoral surveys

What can national surveys hope to achieve? Surveys of the views of citizens would appear tobe of limited value unless they are conducted with high professional standards, and/or, are targetedat particular sectors (such as the customers of health clinics etc. to determine their expe-riences).When they are, the resulting data can be extremely useful both in establishing a baseline against which to measure future change, and to determine what is going wrong. Whenthe results are published, they can also raise public awareness and generate public debate.[11]By flushing out the facts, they can depoliticise corruption which is too often a political football.

In particular, the surveys can:

  • Promote institutional reforms--Measuring the economic and social costs of corruption can help identify priority areas for reforms, and establish quantitative benchmarks to gauge the success of institutional reforms. Agency specific data focus the debate on institutions, not individuals. Information on the underlining institutional structures help explain why some institutions are more vulnerable to corrupt activities and behaviour than others.
  • Stimulate a technocratic and focused debate on concrete action--Survey data and analysis helps build coalitions among key stakeholders by encouraging their positive participation, stimulating a technocratic debate on concrete reforms and promoting collective action.
  • Help identify problem areas and priorities for reform--Survey instruments comprise closed, indirect questions that maximise response rates and facilitate a systematic analysis of the data. Rigorous empirical analysis leads to a non-political debate on concrete reforms to combat corruption. Survey questions emphasise experiential (i.e. empirical) rather than perception data. Both experiences and perceptions are helpful data sources for rigorous analysis. Concrete data on new and old subjects is a powerful tool for anti-corruption strategies.
  • Give the community a "voice" and strengthen local ownership--Surveys are implemented by independent and technically capable local NGOs and survey firms, capitalising on and strengthening local knowledge and expertise. Survey results can be shared with stakeholders and widely disseminated to energise and empower public opinion and build momentum for reform.

Governance and anti-corruption diagnostic studies implemented by the World Bank nowinclude three surveys carried out simultaneously, which focus on public officials, householdsand enterprises, respectively.

  1. Public Officials Survey--The purpose of this survey is to understand institution-specific determinants of corruption (including bribery, nepotism, political interference, embezzlement, etc), discretion/informality, performance, and governance. Survey results inform the policy dialogue on such links as those between governance and poverty alleviation and political and values/cultural differences.
  2. Enterprise Survey--This studies the business environment, with a special emphasis on the effects of public sector governance and corruption on private sector development. It examines firms' roles as users of public services, as being subject to various forms of regulation and as clients for licences and permits. Special attention is paid to the judicial system.
  3. Household Survey--The purpose of the Household Study is to capture the citizens' experiences with, and perceptions of, corruption in their daily lives in both the public and private sectors. Citizens are surveyed in their roles as users of public services, as the subjects of regulation, as clients for licences and permits, and as customers for such services as education, health, water, electricity and housing. Special attention is devoted to social services such as health care or education. [12]

An in-depth diagnostic is not the goal but rather a means of developing anti-corruptionprogrammes that focus on reform and collective action.

However, all these surveys need to be conducted regularly if progress is to bemeasured. Furthermore, in democratic societies, people will want to knowwhether the government is actually being effective, particularly if they arenot seeing immediate change in their daily lives. If surveys show that reformsare, in fact, starting to work, then they can play an arguably even moreimportant role by building public support for reform programmes andthereby adding to their dynamism and impact. It has long been a truism thatan apathetic and disbelieving populace can fatally undermine the best-intendedreform programmes.

Six other approaches may be of particular interest:

(a) Report cards

A particular methodology pioneered by the Public Affairs Centre of Bangalore,is the use of "report cards" on public services. These involve the interviewingof the "customers" of various public utilities to ascertain what in theway of "extras" they are being required to pay in order to get their legitimateservices. The resulting "report cards" are then discussed with managersresponsible for the various services and published in the press and on radio.Various NGOs around the world are being trained in Bangalore to use thisapproach, including members from a number of TI national chapters.

(b) "Big Mac" surveys[14]

A different methodology has been developed by Poder Cuidano, TI's nationalchapter in Argentina. This methodology involves, first winning the cooperationof a particular Government Ministry or City Mayor, and then carryingout an analysis of the prices paid or charged for various services. For example, it was foundthat identical supplies were being purchased by different hospitals at very different prices.Whether due to inefficiency or to corruption, the waste and extravagance was clear. Once theoffenders had been identified through comparing the prices, official action was taken and thepurchase prices dropped dramatically.

In another Argentinian "Big Mac" survey, a comparison of the costs of school meals and theprices being charged for them, revealed great discrepancies. Within a short time of the publicationof the differentials, the suppliers who were overcharging had dropped their prices andbrought them all into line.

(c) Before and After surveys

In Milan, TI-Italy has tried to measure corruption through research on the cost of corruptionby adopting a "before and after" approach.

As is well-known, the Pool of Milan Magistrates launched their "Clean Hands" investigationon corruption in the early 1990's. A large group of politicians and businessmen were convictedof mismanaging public funds and of providing illicit finance to political parties and politicians.Civil society strongly supported the Pool and induced a change in the next municipalelections. Except for a few highly-respected individuals, none of the old City guard was reelected.

In these circumstances, the TI National Chapter felt able to measure the cost of different publicinvestments "before and after" in several areas. The results were striking. The cost of publicinvestments in Milan before and after the "Clean Hands" campaign were:

  • direct investments per year increased by nearly 400 per cent;
  • total municipal debt was reduced by ten per cent;
  • the construction cost of the underground railway more than halved;
  • the new international airport (built in three years) had an actual total cost of less than half the previously-estimated cost;
  • City-owned companies moved from an annual loss to a substantial profit; and
  • municipal tax was limited to five per cent, as against the six-to-seven per cent of other comparable urban areas.

These figures helped to make people aware of the real cost for the taxpayers versus an otherwisesomewhat abstract concept of corruption.

About one year later an independent poll company undertook a poll for TI-Italy on howMilan's citizens felt about corruption. The results showed that:

  • over 76 per cent felt that that state was not adequately protecting its citizens from the danger of corruption and intimidation, as they believed that criminals had taken over the state and its institutions;
  • over half (52.2 per cent) thought that unlawfulness and thefts were rarely punished and a further 42 per cent thought that this proposition was "largely" true; and,
  • nearly 93 per cent thought that the corruption network in Italy had not yet been totally uncovered, or that there were still some areas to be explored.

Yet, there was some comfort in the level of citizens' awareness, as nearly 85 per cent consideredthe costs of corruption to the ordinary citizen to be very high.[15]

Subsequent tracking of the costs of public procurement, after initial gains had been made,revealed a steady increase in the costs of some public procurement. Not, it seems, from corruptioninvolving large private sector companies, but rather businesses that had started toform illegal cartels and to rig the tender processes for smaller contracts. In total contrast, theinterest of international firms in competing in the new and more open market place has meantthat the decline in costs for major public contracts has continued.[16]

(d) Value for Money

The guiding thesis is that corruption drives up prices while driving down quality and performance.Thus, when assessing a public transaction on the basis of "value for money", if thereare gross distortions it can be due either to corruption or to rank inefficiency. In either case,firm action is needed if systems are to be strengthened and repetition avoided. "Gross" distortionscan be detected quite readily, particularly given the availability of agencies around theworld who can quickly and inexpensively provide information on prevailing prices of goodsand services.[17]

(e) "Mirror statistics"

Mirror statistics are a form of civil society monitoring that has been developed in Bulgaria. Itinvolves a comparison between the goods exported according to Bulgarian documents toneighbouring Romania, for example, with the Romanian import documentation, and viceversa. This promising initiative requires cooperation from both customsauthorities. As well as comparing national statistics, the daily exchange ofinformation and entries in registers on both sides of a border enables Bulgariancustoms officers and those in neighbouring countries to compareinformation about the vehicles carrying goods in "high-risk" categoriesacross the frontier. When this was applied to the Bulgarian/Romanian frontier,it revealed a variety of customs frauds , in particular involving cigarettes.Trucks loaded with cigarettes for export from Bulgaria were shown never tohave crossed the border, or else to have done so loaded with toilet paper andthe like (as recorded in the records on the Romanian side of the border). Inthis way the duty-free goods remained in Bulgaria with the duty unpaid.[18]

(f) Monitoring major projects

Finally, there is the question of whether it is possible for civil society, with its necessarily limitedresources, to monitor the implementation of very large projects. The approach that isemerging suggests that it is. Civil society should only aim to tackle tasks on which it candeliver useful results. Thus, where the scale of a particular project is beyond its reach, civilsociety can do either or both of two things. It can monitor selected key elements of the enterprise,and/or monitor the entire project by focusing on the "system" and the way it is functioning,thereby ensuring that the processes by which the tenders are awarded conform to thelegal and administrative requirements.