Towards a valid and viable measurement of SDG target 16.7

My chapter in: SDG16 Data Initiative. Global Report 2020, New York, 32-37

The UN Member States have set ten specific targets for Sustainable Development Goal (SDG) 16 , the promotion of just, peaceful and inclusive societies. Of these targets, target 16.7 aims at ensuring “responsive, inclusive, participatory and representative decision-making at all levels”. Among the 17 SDGs and the 169 targets defined to achieve the Goals, target 16.7 may be viewed as a key target because it focuses on political decision-making, which is a crucial prerequisite for all of the desirable policy outcomes defined in SDG 16 and in the other SDGs. This chapter discusses the official indicators for monitoring target 16.7 and argues that the Global State of Democracy Indices – a set of democracy measures developed by the International Institute for Democracy and Electoral Assistance (International IDEA) – can function as valid proxy indicators. read more

The Quality of Democracy in Central and Eastern Europe

Contribution to “Democracy under Stress“, ed. by. P. Guasti and Z. Mansfeldová, Institute of Sociology, Czech Academy of Sciences, Prague 2018, 31-53.

 

The young democracies in East-Central and Southeast Europe have been particularly susceptible to the wave of populist, anti-establishment and extremist political forces that now challenge liberal democracy across the globe. These challengers claim to represent the opinion of the ordinary people against a political establishment that is portrayed as corrupt, elitist and controlled by foreign interests. Their polarizing and anti-pluralist ideological stances have contributed to a more confrontational political competition. Several countries have also seen “democratic backsliding”,  an erosion of the institutions and mechanisms that constrain and scrutinize the exercise of executive authority. Illiberal policies have targeted opposition parties, parliaments, independent public watchdog institutions, judiciaries, local and regional self-government, mass media, civil society organizations, private business and minority communities. Incumbent elites have justified these policies as measures to strengthen popular democracy and to fulfill the promises of the post-1989 democratic transitions.

The legitimatory references to popular democracy, the incremental nature of institutional change and the embeddedness of governmental actors suggest an analytic approach that places instances of democratic backsliding into a broader theoretical context. Such a framework should specify criteria of democratic quality that allow to assess legitimatory claims and to analyze how the incremental manipulation of interlocking democratic institutions affects democracy as a whole.

This chapter sets out such a conceptual framework based on democratic theory and the scholarly debate about the quality of democracy. The insights from this debate are applied to identify attributes of democratic quality that are then employed to structure the empirical analysis. Key attributes include public accountability, government responsiveness, and policy performance. The chapter maps selected developments in 17 CEE countries with regard to these three attributes.

Following a qualitative and inductive approach, the paper tries to integrate individual causal process observations into larger trends that characterized the situation in the region in 2017. The empirical analysis draws mainly on country reports that have been prepared in early 2017 for a global expert survey, the so-called Transformation Index, a comparison and ranking of democratic and economic developments in 128 countries. Three main empirical findings are presented in this chapter.

Firstly, governing political elites in some countries intentionally weaken the mechanisms and institutions of public accountability.

Secondly, political competition has become more confrontational, resulting in more exclusionary government.

Thirdly, most CEE countries have failed to achieve an economic performance meeting the expectations of citizens who have associated EU accession with rapid convergence to West European levels and tangible prosperity gains. .

These developments have evolved in parallel and are causally linked, but none is fully endogenous. The concluding section suggests an interpretation to account for the reasons that have motivated populist governing parties to weaken public accountability, but the chapter does not aspire to fully disentangle the causal interdependencies linking the three trends. Rather, its aim is more modest, that is, it focuses on providing empirical evidence for the trends and seeks to demonstrate that the proposed conceptualization of democratic quality yields analytic benefits.

The chapter is based on a paper I presented at the Conference “Measuring Democracy: What are We Measuring and How Does CEE Fit in? , Prague 21-22 September 2017

Eastern Partnership Index 2018 Update

On 25 January 2018, the Eastern Partnership Civil Society Forum published the fifth edition of the Eastern Partnership Index. The Index is a set of individual and composite indicators which measure the extent to which the six Eastern European neighbour countries of the European Union have established sustainable democratic institutions and made progress towards closer integration with the EU.

In the methodology suggested by me, “integration” is conceived as a core and multi-dimensional concept that consists of converging norms, growing economic exchange, deeper transnational networks linking up societies, and more frequent contacts between people. This broad notion of integration implies that EU membership or association may be aims, stages or final states of the integration process. However, it is not limited to a measure of harmonisation with EU norms and standards, but also reflects actual societal, economic and political change. The levels of contractual relations between the Eastern Partnership states and the EU are viewed as elements of a much broader process that is, as a whole, not driven or controlled solely by governments and intergovernmental negotiations.

Rather, European integration is seen as a non-hierarchical, networked process where citizens, civic associations and business organisations play important roles. The interplay of these actors has been crucial for the historical development of the EU itself, as it induced and supported national political elites to take legal and institutional steps towards closer integration. Drawing on this experience, the Index is built on the premise that the ties between societies, peoples and economies form dimensions of European integration that are at least as important as the policy agendas of national governments and European Commission officials.

It is further assumed that transnational linkages contribute to the emergence and spread of common European and international norms which, in turn, facilitate closer linkages with the EU. For example, increasing trade is likely to strengthen domestic companies that benefit from foreign investment and are likely to become interested in courts that protect investors’ rights. A judicial system based on fair procedures and professionalism will then contribute to attracting more foreign investors.

An analogous reinforcing dynamic derives from a commitment to international norms and universal values. By incorporating democratic values, the protection of human rights and the rule of law in their constitutions, EaP states have adopted universal norms that have formed the basis of co-operation and integration among West European states since the end of the Second World War. The more these norms are implemented and respected in EaP states, the more co-operation with the EU will ensue because these states and the EU will increasingly recognise each other as partners sharing common norms and underlying values.

Download the report:EaP_Index_2016

Download the dataset

 

Assessing the Fundamentals of EU Accession

A civil society monitoring for Montenegro

Andrija Pejović, Minister of European Affairs (right), and Dragan Koprivica (Center for Democratic Transition) during the presentation of the reports

In 2015, the European Union redesigned its enlargement policy to focus on the rule of law, public administration and civic rights.  These “fundamentals” are required to meet the criteria of membership and constitute the  preconditions for a sustainable modernization of the Western Balkan states. The European Commission has monitored the state of reforms on the basis of consultations with government officials and external observers.

To involve civil society in this assessment and to provide better evidence for public debates, a Montenegrin think tank, the Center for Democratic Transition (CDT), has surveyed 41 experts and analyzed publicly available data. Drawing on the Commission’s new standardized assessment scales, CDT and I developed detailed questionnaires that assess the following areas: functioning of the judiciary; fight against corruption; fight against organized crime; media freedom; public administration reform; human rights. The results of these surveys are now published in two reports.

A key finding of this research has been that there is still a significant gap in implementing the new regulations adopted during the preparation for EU accession. Executive actors retain informal influence on the work of the judiciary. The newly established corruption-prevention agency has not effectively supervised the finances of political parties. Media outlets struggle to survive under growing economic constraints and political pressures. The judiciary has failed to sustain convictions in several cases of organized crime. Discriminatory practices persist due to a lack of both effective anti-discrimination policies and societal awareness.

 

BTI 2018 Kick-Off and Concept Paper

On 12/13 September the core group behind the Transformation Index (BTI) met at Bertelsmann Stiftung’s always impressive Gütersloh headquarters to prepare the new survey wave.

bertelsmann_stiftung
Swans’ view of the Foundation’s headquarters

BTI is a global expert survey on the quality of democracy, market economy and governance, now in its eighth edition (!) scheduled for publication in 2018. Questionnaires will be sent to country experts at the end of October. The submission of country reports will trigger off an elaborate procedure of reviewing, revising, calibrating, editing and lesson-drawing, keeping us busy throughout the next year.

The meeting was a first occasion for Peter Thiery and me to present the draft of a comprehensive paper that situates the concepts guiding the BTI in the scholarly literatures on democratic theory, democratization, policy reform, good governance, economic transformation and aid effectiveness. Our paper also uses unpublished data from the BTI production process to evaluate the validity of the measurement and aggregation techniques underlying the composite indicators in the BTI dataset. Author-reviewer differences are studied across subsequent BTI editions. Various statistical models are constructed to assess the impact of changes among authors, reviewers and coordinators and to compare the effects of different aggregation rules. We plan to complete and publish the paper during the next months.

Policy Analysis and Policy Evaluation

A workshop with Belgrade Open School staff, 16-18 May 2016

To upgrade their research capacity, think tanks need access to methodological and conceptual tools that have recently been developed by researchers in the field of policy analysis and evaluation. My three-day workshop with Belgrade Open School provided an overview on rationalist and institutionalist approaches of policy analysis, focusing on examples from Europeanization studies. Notions of causation and strategies to deal with confounding causes formed the basis of our discussion on evaluation methods. The workshop also included an introduction to regression and factor analysis, two key tools of empirical policy research.

PalataBeogradjanka
Palata Beograđanka, BOŠ premises on 16th floor. Picture source: Lakododoma  Magazine

Founded in 1993, BOS has become one of Serbia’s leading civil society organisations for post-graduate training and public policy advocacy.

BOSAs a non-partisan and non-profit organization, BOS strengthens human resources, improves the work of public institutions and organisations, develops and advocates public policies in order to develop better society based on freedom, knowledge and innovation.

 

Performance Monitoring for Montenegro

CabinetMeeting
Government of Montenegro: Cabinet Meeting

Montenegro belongs to a group of income-poor, small and young Southeast and East European democracies interested in joining the EU. To master the challenges of accession and membership, these states have to create a modern and effective public administration. However, experience has shown that externally imposed accession conditions and regulatory alignment do not sufficiently prepare new EU member states to overcome culturally ingrained practices of corruption and clientelism. Accession-driven administrative reforms yield sustainable effects only if they are complemented by stronger domestic public accountability.

The proposed project seeks to improve this accountability by establishing a system of indicators that enable the public to monitor the performance of central government in Montenegro. The indicators will be designed to map inputs, outputs and outcomes of government activities. They will provide data on financial and human resources employed by ministries, on the management of these resources and on the impacts of public policies. By generating objective, reliable and detailed empirical evidence on government performance, the proposed monitoring system will (1) empower parliament, citizens, civil society organizations and media to better assess and debate the quality of public governance and (2) support political representatives and senior officials in identifying and addressing shortcomings.

The project will be realized in cooperation with a Montenegrin think tank, the Center for Democratic Transition, and the General Secretariat of Montenegro’s Government.

Download: CDT_PolicyBrief_en

Methodology of Social Science Research

Training for the Institute of World Policy, Kiev, 4 September 2015

A full day of teaching with the staff of Ukraine’s leading independent think tank for International Relations, the Institute of World Policy. The main themes were:
1. Research design / causation, theory building;
2. Conceptualization and measurement
3. Qualitative and quantitative methods
4. Evaluating policies

Regression
(c) Martin Brusis

Video clips available here

Quality of Democracy and the “Sustainable Governance Indicators”

Relating the Sustainable Governance Indicators
to the Quality of Democracy

Paper for the Workshop “Measuring Democracy”. IPSA Research Committee on the Quality of Democracy, DVPW-Arbeitskreis Demokratieforschung, Frankfurt/Main, 29 September – 1 October 2013

The Sustainable Governance Indicators (SGI) are a set of 147 detailed items and two aggregated indices that measure the quality of democracy (QoD), policy performance and executive governance in OECD member states. Two editions have been published since 2009 by the Bertelsmann Foundation, an NGO based in Germany. The SGI consist of qualitative expert assessments, country reports and data retrieved from official statistics. As the name suggests, the SGI have been designed to measure the quality of governance, a concept that overlaps with concepts of democracy or QoD, but has its distinct origins, ideational references and discourse communities.

One motive for their creation has been to complement the Foundation’s so-called Transformation Index that is limited to assessing democracy and market economy in the developing world, post-socialist countries and emerging markets. Since the label “Transformation” and the underlying concepts refer to reforms aimed at establishing democracy and market economy, a simple transfer of these concepts to countries considered as “established” democracies and market economies would have raised many problems. It was therefore decided to use the label “Governance” in the SGI project. Apart from the appeal of an internationally more familiar term (and renowned predecessor projects such as the Worldwide Governance Indicators), the label was also chosen to convey a more specific meaning since it refers to the interaction between governments and their political and societal environment. This focus on “executive governance” has been seen as an important added value of the entire project since the governing activities in and by (core) executives constituted a white spot in an increasingly crowded field of cross-nationally comparative indices and datasets.

The present paper does not further explore the SGI’s conceptual background in debates about the quality of governance and government. Rather, it explains the design of the SGI by situating it in the debate on QoD. This debate has been concerned with theoretically grounded concepts able to adequately differentiate among democracies. It has been driven by the global diffusion of democracy that has made differences between individual democracies more visible and has revealed shortcoming of existing measures of democracy which often do not sufficiently distinguish among so-called established democracies.

The paper identifies two conceptual strategies scholars have chosen to evaluate QoD: enhancing the concept of democracy or defining a separate concept of democratic quality. More demanding, “thicker” conceptualizations of democracy must demonstrate that including or excluding additional attributes neither stretches the concept nor introduces voluntarism. Conceptualizations of democratic quality are expected to anchor the underlying idea of quality in the theory of democracy. In the first three sections, these conceptualizations are discussed and integrated into a composite concept that combines the notion of a constitutional democracy with measures of governmental accountability, responsiveness and performance. The paper claims that the SGI, by reflecting this composite concept, provide a valid measure for QoD. In the following sections, the concept is further disaggregated into dimensions, categories and items that enable a measurement based both on indicators from official statistics and expert ratings. The SGI measurement and aggregation procedures are explained and the validity of the measurement is examined.