PART ONE -INTRODUCTION
Jan Bakkes
Indicators are always a compromise. Their design needs to optimize between relevance to the user, scientific validity, and measurability. O'Connor, writing about accuracy and precision of indicators, once captured this spirit in the requirement for indicators to be 'optimally inaccurate' .Moreover, further development of sustainability indicators concerns as much the organization of the process that the indicators are part of (the process of setting priorities and goals, of designing programmes and allocating responsibilities, and monitoring the results) as it concerns the proper design of the indicators. Consequently, because these steering processes do differ between different situations (for example, between different forms of government), research on sustainable development indicators cannot aim for universally applicable methods. In short, sustainable development indicators are, by definition, imperfect, process bound and not universally applicable.
However, when practitioners formulate what methodological difficulties they meet in enhancing the power and sharpness of the tools they use as sustainable indicators, common issues do emerge. This is what this chapter reflects. It is based on meetings, including those by the SCOPE project on indicators (Moldan and Billharz 1996; see also Box 5A) and on reports using indicators for sustainable development. Its rationale lies in the observation that few resources (including time) are available for research on sustainable development. Therefore, a research agenda would be useful as a means to concentrate available resources.
This chapter is a structured overview of issues requiring research in the development of sustainable development indicators to date. In order to provide more structure, these issues have been organized into the following three groups, even though there are important linkages between the issues. First, research is needed to develop indicators for concerns that seem to be completely lacking in present-day methodology. Secondly, efforts are needed for topics where indicators exist, but are inadequate. Last, research is necessary in order to successfully use the indicator designs in practice.
THEMATIC COVERAGE
Many sources report the need and the difficulty to provide indicators for 'human aspects' of sustainable development. Upon closer observation, this is a mixture of issues related to the well-being of individuals (such as health, education, absence of poverty) and issues related to social capital (which is far from universally defined but includes aspects such as rule of law, security of tenure, trust, social networks, access to information, adequate institutions and absence of corruption). Both clusters are now increasingly recognized as critical elements of transitions towards more sustainable societies (see United Nations 1997). Both are difficult to capture in one, or a few, quantitative indicators. It may be easier to characterize the status of a community in tenus of a pattern rather than a number.
If we wish to apply numerical indicators, however, there is almost nothing available in the area of social capital, beyond anecdotal material. At the same time, there are relatively well established indicator concepts, and even operational indicators, in the area of human capital. Some examples include: infant mortality, disability adjusted life expectancy (introduced by Murray et al.1993) the three poverty measures in the CSD list (United Nations 1996; Ravallion 1994) and the human development index (UNDP 1996).
From the perspective of measuring progress towards sustainable development, research issues, in the domain of social capital, would include the question whether there is, conceptually, such a thing as social capital that can be measured on an equal footing with man-made, social and environmental capital (Serageldin 1996). Even if that is not so, or if the question remains unanswered, measures of specific aspects can be developed as indicators for partial progress. When targeting research on indicators of partial progress, it is advisable to start with relatively unambiguous issues such as security of tenure. In contrast, access to and use of information, which is often named as a candidate indicator in this area, poses an example where further thought is required before an indicator can be operationally defined and be interpreted in its cultural context.
The absence of non-trivial indicators for the institutional dimension of sustainable development may be one of the two largest gaps in current indicator programmes. (The other significant gap is the absence of indicators of policies. ) An obvious topic for research is to test whether or not semi-quantitative rating systems, used in businesses, can be applied in the context of sustainable development.
Many criticisms about currently used or proposed indicators (for example, Fues 1996) have to do with the fact that they reflect simple averages or totals, masking the distribution of the chosen measure over the population. For some issues in sustainable development, skew distribution is at the heart of the problem and an indicator of progress should focus on this problem. The highly uneven distribution of land ownership, for example, is an impediment to development (compare Squire and Deiniger 1997). For other issues, skew distribution is only an 'aggravating' factor. However, this can be to such an extent that the problem -or progress, for that matter - is invisible if averages are used to measure it (e.g. exposure of the urban poor to multiple pollution, or expansion of agriculture on vulnerable land).
Research in this area should aim at producing aggregate indicators that are much closer to the real problem and thus, much more suitable to evaluate the effect of policies. Such aggregate indicators build on the combination of more detailed information and rules for generalization. The underlying detailed information may involve a traditional stratification in terms of social groups or industrial sectors, or spatial distribution (e.g. drainage basins within a country, or distinguishing between hillsides and valleys or mapping of ecosystems relative to potentially threatening human activities - compare Box 2C; Bryant et al. 1995; Ten Brink 1997) or a combination of these aspects. An example of indicator related research, where both spatially and sectoral detailed information was combined, is the analysis of Chomitz on the different relationship between poverty and pressure on forest margins, resulting from either growing cash crops or subsistence agriculture.
The rules for generalization should allow the eventual indicators to capture the findings of such detailed analysis at the more general level of reporting, such as countries or regions. An successful example from international negotiations on pollution control is the mapping of critical loads (see Box 3S). Although this type of indicator initially requires extensive sets of basic data, it is currently being applied successfully outside the high-income regions. Those indicators from the CSD set that are of no use without distribution information are an obvious priority target (e.g. BOD in Water Bodies, or Maximum Sustained Yield for Fisheries).
A second topical area that urgently needs good overview indicators is policies. (They are 'Responses' in the Driving Forces-State-Impact-Responses framework.) Little methodological progress is reported here, especially with regard to indicators that would allow for the summarization of a government, enterprise or sector position concerning sustainable development policies. Typically, the most comprehensive effort thus far with regard to environmental policies (World Bank 1997) aims at organizing knowledge about good practice, rather than at measurement. In this area, specific difficulties need to be tackled. Efforts must be made: to avoid a bias towards curative policies, as opposed to preventive (Fues 1996); to distinguish between efforts and outcome; to aggregate and compare the very large array of measures that often address the same issues in different situations.
Three routes for further development appear to exist. First, within a limited context, the costs of measures have been used successfully as indicators of effort. If this is expanded to include the full social costs, it can produce a useful overview indicator for comparing policy options within a country, region or enterprise.
Second, the total level of subsidies, including 'tax subsidies' , on material input in an economy has been proposed as an indicator of non-sustainability from an environmental and resource perspective (de Moor 1997; de Moor and Calamai 1997). This has obvious difficulties, for example, the fact that fertilizer subsidies cannot be compared between countries without taking into account whether the problem is the excess or shortage of nutrients. But because the issue of' subsidies' will get more political and research attention in the coming years, there is greater potential for the development of indicators for policies towards sustainable development.
The third, and probably the most promising, avenue for improving indicators of policies, is to accept that policies are indeed 'responses' and need to be presented in this dynamic context (see chapter 4 in this book). The research needs are, then, in the domains of modelling (which is also, for other reasons, increasingly recognized as a related area that requires attention), presentation, and the study of governance.
Trade in natural assets has been named as another issue of sustainability that has not been sufficiently covered in present indicator sets. Interestingly, it is an important component in the estimation of the successful 'Genuine Saving' indicator. Recent work based on genuine saving led to the realization that it is probably the money flow that deserves the most attention in future research into indicators related to trade in natural resources such as wood and oil (Hamilton 1997). More specifically, this is about rent capture (the fraction of the revenue that flows back to the exporting country) and the fraction of revenues that is actually invested in the exporting country.
ADEQUATE REPRESENTATION
One issue that affects indicator design is relevance. It asks the basic question whether or not an indicator measures the right thing. Obviously, this refers to the design process of each individual indictor in each individual programme. However, there are a few related examples where a systematic effort can provide results that can be used more widely. In ecological and pollution research, 'budget methods' or 'substance balance sheets' have been successfully used in order to systematically determine the scope of the problem before indicators are proposed (see an example based on the Baltics: Rahm and Danielsson 1995). Environmentally-oriented methods are emerging for the rapid assessment of relevant social, economic and environmental concerns in large river basins (TIAER 1996). Again, these methods are meant to be applied before the indicators are proposed.
Another example is the weighting of life-expectancy by disability (Murray et al 1993). This example prompts the question: what environmental parameters should be chosen as environmental health indicators, if any at all. These screening methods clearly have great potential in guiding indicator design, but it is also clear that effort is needed to make them cheaper and more efficient.
Specifying the significance of a given indicator also requires continuous effort. The methodology sheets of the CSD indicators programme demonstrate how a specification may look (United Nations 1996). They also demonstrate that the unclear significance of a given indicator is a significant weak point of many currently proposed indicators. Part of this can be resolved during the testing and use of indicators. Also, it has to be recognized that perceived relevance in public opinion is an important and valid consideration in the selection of indicators. In addition, it is useful if description formats, like the CSD methodology sheets, are tested and refined further in order to guide developers. Finally, in addition to the case studies commenced by the Bellagio group, a comparable evaluation of relevance in existing indicator programmes could help to boost work in this area.
The use of stock measurements (such as capital or habitat) is increasingly recognized as a way to place more conventional flow variables (such as savings or emissions) in the context of sustainable development. In particular, the aggregated estimation of man-made, human and environmental wealth has proved to be a powerful concept. However, a host of methodological problems has to be resolved before wealth estimates can be used as operational indicators. Most importantly, an approach towards establishing critical lower levels of the different capital forms needs to be worked out and tested (Serageldin 1996). (Also, in order to make wealth estimates suitable for comparison with different sustainable development strategies, the resource valuation used in wealth estimates has to reflect the sustainability of the use of the resource. If not, the indicator will, by definition, be biased in favour of seeking short-term revenues from natural resources, such as land and forests.) This means that initial shortcut methods, based on market prices, have to be refined or replaced. In other words, classical issues such as green accounting have to be revisited but from a slightly different perspective.
At a macro level, a good example of flow measurement, as an operational indicator that derives much of its significance from a background stock measurement, is the combination of genuine saving and national wealth. It also serves to illustrate that further development is necessary in order to make the measurement flows (e.g. expenditure on education) and the measurement of the related stock (educational attainment as an element of human capital) compatible ( e.g. both covering higher and lower education, and expenditures at all administrative levels and by all sectors). The research required to specify critical lower limits of different forms of capital relates to a more general issue. Indicator design is a construct that includes a reference value (e.g. target, historical value, threshold for effects). Even though targets do not feature in many of the current programmes that develop indicators for sustainable development, the topic cannot be avoided for long when it comes to the application of the indicators (see chapter 3 in this book). A potential area of research is to determine the various ways in which targets and threshold values and the like can be underpinned. The aim is to facilitate communication about targets.
Linkages between the various issues are often mentioned as an aspect, specifically requiring better coverage in indicator schemes. It is argued below that this is largely something that has to be addressed by using an assessment framework and models, rather than by using individual indicators. However, two or three topics exist where additional research can help to quantify linkages directly via the indicator concepts. These are: marginal environmental efficiency (how much environmental pressure for each additional unit of GDP) (Dufournand and Rogers 1994 ); resource intensity (see Box 4C); and possibly, valuation (the 'exchange rate' between human, environmental and man-made capital) (Serageldin 1996).
Of these three topics, resource intensity is developing as a theme in sustainable development strategies and therefore, warrants its own indicators. Not unexpectedly, research is most needed in aggregating resource use by expressing all material inputs in a sensible way to a common finite resource - in most cases, energy or land (Wackernagel and Rees 1996). Resource intensity is the most probable indicator used by enterprises and in certification programmes, such as ISO 14000 (Pfliegner 1996).
This requires a relatively high degree of standardization because the indicators have to be used by many organizations, also outside industrialized countries, at a reasonable cost. On the other hand, in order to use resource intensity for monitoring progress towards sustainable development at a national scale, research is needed in order to eventually expand the indicator beyond the manufacturing industry where, usually, the first and most obvious efforts have been concentrated.
Furthermore, users need accompanying information with the indicator value. In addition to a statement about significance, the origin or development of an indicator value needs to be expressed concisely. For example, the potentially destabilizing effect of poor data availability on an index needs to be assessed and expressed. The risk is particularly serious when the index is constructed from a few components, such as the Human Development Index. But the research needs go far beyond conventional statistics and include concept validity (ways to classify how widely the underlying model is being accepted) and designing uncertainty ratings for complex studies (Constanzu, Functowicz and Ravetz 1992). An obvious beginning is to develop simple and effective methods to show the scientific and cultural perspective on which the selection and estimation of the indicator is based (Rotmans and De Vries 1997; see Wang 1995).
Indicators that summarize the whole array of sustainable development issues in a country, region or urban area are still not operational. It is now being realized that such indicators only have a role to play when supported with a system of more detailed and more analytically powerful measures. However, 'portfolio indicators' are still greatly needed. The best candidates on which to focus research are the combination of national wealth estimates and genuine saving, and aggregate resource intensity. Specific research issues for both were mentioned earlier in this chapter.
Finally, indicators simplify reality. Because of this, it is important that reporting organizations and those commissioning their reports remain alert to possible systematic distortions in the indicator systems. For example, valuation and weighing may introduce circular reasoning because the resulting indicator values may influence the setting of weights in subsequent reporting. With all filtering of information, this risk cannot be avoided completely. Therefore, in addition to extensive peer review, basic measurements and their thorough and critical analysis remain important. Moreover, as indicator-based reporting on progress towards sustainable development develops into real monitoring, comparable measurement over time is indispensable, especially while current data collection is insufficient in many areas of the world (UNEP 1997b). Unfortunately, primary measurements and data collation are costly.
In addition, as experience shows, indicator-based studies on transitions towards sustainable societies require not only looking forward, but also require looking backward for about an equal length of time. In other words, in order to understand and illustrate the challenges, a historical time series for key indicators is required, typically going back to at least 1900. This is definitely a research effort where work can begin immediately.
ASSESSMENT
In many ways, models have a role to play in the improvement of indicator systems. Obviously, all indicator systems are built on an implicit or explicit model, even if it is a mental model, such as the model that led us to give the educational level of women a place in the CSD menu of indicators. In addition, such a model explains the linkages between the various indicators, including keys to aggregation and selection. (In fact, this is one valid definition of a model.) Further development of models, including computer models, can boost the significance of present-day work on sustainable development indicators in a number of mutually reinforcing ways.
First, greater transparency of underlying models would help decision-makers to accept, use and possibly change the currently proposed indicator sets. In fact, this is a good response to the repeated request to have indicators better reflect linkages between issues. Also, the repeated, yet contradictory, requests for the refinement of the CSD indicator framework can be addressed in a productive way by showing the varying roles the indicators have in different models. The most promising development is interactive or quasi-interactive computer models that allow the user to change many of the underlying assumptions and selection steps. As model-based support to international negotiations shows (Hordijk 1991) this is a particularly productive setting to explore targets.
Second, models are indispensable when it comes to using indicators to explore future problems and the links to present-day policies and strategic choices. The newer generations of models that are being developed are better suited to the diversity found within many societies, environments and indeed, ideas about the future. Research, in particular, is required to provide these models with the same richness in the social and economic spheres, as they currently have in the environmental and resource domains. In addition, special effort is needed to make these tools more widely accessible and adapt them to the requirements of assessment processes in different regions.
Third, studies with more elaborate models can help to identify better indicators. A focused research effort should be able to develop meaningful indicators that are suitable for efficient analysis of policy options and for world-wide monitoring, especially with regard to concerns about the use of land, water and energy (see UNEP 1994).
Presentation techniques are closely related to the very purpose of indicators and thus, indicators need to be developed further. Issues for the next few years include the relationship between indicators, models (as discussed earlier) and spatially explicit information. Also, as indicators are developed at different levels of generalization in order to meet different user demands, techniques will be needed to present them in a layered and coherent way. One successful example is the development of the 'safe landing' approach in support of climate change negotiations (see the original methodology: Alcamo and Kreileman 1996). It shows how presentational difficulties were resolved by designing a new type of indicator. 1
Finally, there is a need to use the large sets of seemingly atomistic indicators, similar to the current set of the CSD indicators. The freedom to chose indicators from the 'menu' is necessary. Yet, there also has to be some way to allow for an assessment by the user, whether or not the set of indicators selected maintains the broad coverage of the framework. This potentially sensitive process could be aided by an objective methodology or guidelines for scoping. These need to be developed and tested.
NOTES
1 The problem was the confusing multitude of scenarios, policy options and different views on assumptions. Guided by technical workshops with delegates from climate negotiations, the results of analysis with elaborate impact models were then used to construct 'safe corridors' for the development of global emission levels from the present up until 2100. This encompasses all possible paths towards stabilization by the end of the next century . The upper and lower boundaries of the corridor are defined by hard physical and technical constraints. The chosen path within the corridor is determined by the degree of risk that one is willing to take.
REFERENCES
Alcamo, Joseph and Eric Kreileman (1996). Emission scenarios and global climate protection. In: Global Environmental Change. Vol. 6 (1996), no.4, pp. 305-334.
Billharz, Suzanne and Bedrich Moldan (eds.) (1996). Scientific Workshop on Indicators of Sustainable Development. Wuppertal, Germany, November 15-17, 1995. SCOPE Report.
Brink, B.J.E. Ten (1997) In: The Future of the Global Environment: A Model-based Analysis Supporting UNEP's First Global Environment Outlook. National Institute of Public Health and the Environment (RIVM) and United Nations Environment Programme (UNEP). Bilthoven and Nairobi, UNEP/DEIA/TR.97-1 and RIVM/402002007. See Chapter 5.
Bryant, D., E. Rodenburg, T. Cox, D. Nielsen (1995) Coastlines at Risk: An Index of Potential Development-Related Threats to Coastal Ecosystems. World Resources Institute, Washington, DC.
Chomitz, K.M., and C. Griffith (1996) Deforestation, Shifting Cultivation, and Tree Crops in Indonesia: Nation-wide Patterns of Smallholder Agriculture at the Forest Frontier: Research Project on Social and Environmental Consequences of Growth-Oriented Policies. Working Paper No.4. The World Bank, Washington, D.C.
Constanza, Robert, Silvio Functowicz and Jerome Ravetz (1992) Assessing and Communicating Data Quality in Policy-Relevant Research. In: Environmental Management 16 (1) (1992): 121-31.
Dufournud, Christian M. and Peter P. Rogers (1994) Preliminary Discussion Paper on the Concept of an Environmental Elasticity. Presented at SCOPE Scientific Workshop on Indicators of Sustainable Development. Wuppertal, Germany, November 15-17, 1995.
Fues, Thomas (1996) Indicators for Sustainable Development: Statement to the CSD 1996. German NGO Forum on Environment & Development. Bonn, Germanv.
Hamilton, K. (1997) in: The World Bank. Expanding the Measure of Wealth: Indicators of Environmentally Sustainable Development. Rio+5 Edition: draft for discussion. The World Bank, Washington D.C. See Chapter 3.
Hordijk, L. (1991) An integrated assessment model for acidification in Europe. Academic thesis, Amsterdam Free University.
Moor, A.P.G. de (1997) Perverse Incentives: Subsidies and Sustainable Development: key issues and reform strategies. Institute for Research on Public Expenditure. Commissioned by the Earth Council. The Hague, The Netherlands and San Jose, Costa Rica.
Moor, Andre de, and Peter Calamai (1997) Subsidizing Unsustainable Development: Undermining the Earth with public funds. Institute for Research on Public Expenditure. Commissioned by the Earth Council.
Murray, Christopher et al. in: World Bank (1993) Investing in Health. World Development Report. Oxford University Press, Washington, D.C.
Pfliegner, K. (1996) ISO 14000: Environmental Management Standards and Implications for Exporters to Developed Markets. UNDP Private Sector Development Programme. UNDP, New York.
Rahm, Lars and C. Danielsson (1995) Interlinkages in environment assessment - the demand of proper statistics. In: Bulletin of the International Statistical Institute, Proceedings of the 50th session, August 21-29, Beijing, China.
Ravallion, M. ( 1994) Poverty Comparisons. Fundamentals in Pure and Applied Economics, Vol. 56, Harwood Academic Press, Switzerland.
Rotmans, J. and H.J.M. de Vries (1997, forthcoming) Perspectives on global futures. Cambridge University Press, Cambridge, UK.
Serageldin, Ismail (1996) Sustainability and the Wealth of Nations: First Steps in an Ongoing Journey. Environmentally Sustainable Development Studies and Monograph Series No.5. The World Bank, Washington, D.C.
Squire, L. and Deiniger, K. (1997) New ways at looking at inequality and growth. DEC Notes No.28, The World Bank, Washington, D.C.
TIAER, Texas Institute for Applied Environmental Research (1996) Proceedings from the UNEP Workshop on Rapid Integrated River Basin Assessments. February 20-22, 1996, Stephenville, Texas, USA.
United Nations (1996) Indicators of Sustainable Development Framework and Methodologies. United Nations, New York.
United Nations (1997) Global change and sustainable development: critical trends. Economic and Social Council, Commission on Sustainable Development. E/CN.17/1997/3.
UNDP (1996) Human Development Report 1996. Oxford University Press, New York.
UNEP (1997a) Global Environment Outlook. United Nations Environment Programme (UNEP), Nairobi. Oxford University Press, New York and Oxford.
UNEP (1997b) Report on the Meeting on Integrated Environmental Assessment/Global Environment Outlook (IEA/GEO) Core Data Group. DPCSD Office, New York, 22-23 January, 1996. UNEP, Nairobi, 1997. UNEP/DEIA/MR.97-1. Reprint of UNEP/DEIA/MR.96-3.
Wackernagel, Mathis and William E. Rees (1996) Our Ecological Footprint: Reducing Human Impact on the Earth. New Society Publishers, Gabriola Island, B.C., Canada.
Wang, Rusong ( 1995) Sustainability Indicator in China: A Theoretical and Practical Concern. Presented at SCOPE Scientific Workshop of Indicators of Sustainable Development. Wuppertal, Germany, November 15-17, 1995.
The World Bank (1997) Five Years after Rio: Innovations in Environmental Policy. Rio+5 Edition: draft for discussion. The World Bank, Washington, D.C.