Geraldo Munck and Jay Verkuilen, “Conceptualizing and measuring democracy: Evaluating alternative indices”, Comparative Political Studies 35 (2002), pp. 5-34.

Main Argument:

The authors create a framework for analysing data sets that operationalise democracy in order to all quantitative measurement or at least more measureable classification.  Specifically, the authors look at current problems with the conceptualisation, measurement and aggregation that have taken away from much of the research on democracy, and they argue that problems of causal inferences should not be prioritised in the research over problems of conceptualisation of abstract terms.


Method: Evaluating 9 large-N data sets that are frequently used in the democracy literature, Munck and Verkuilen use their conceptualisation/measurement/aggregation framework to establish the gaps between the theoretical literature on democracy and the way it is presented quantitatively.


Important Insight: The authors’ differentiation between the problems of minimalist and maximalist definitions and their look at omitted variables common to several data sets highlight the importance of nesting quantitative studies within qualitative frameworks.



== Notes ==


–     Looks at problems in IR research with the selective use of quantitative data

–     To a large extent, problems of causal inference have overshadowed the equally important problems of conceptualization and measurement

–     Authors provide a systematic assessment of the large-N data sets on democracy that are most frequently used in current statistical research

–     They provide a framework of analysis

–     Three challenges in using quantitative data: conceptualization, measurement, and aggregation

–     See table on page 8 for a good explanation of their framework



–     Identification of attributes

–     Maximalist Definitions: tendency to specify the meaning of a concept in a way that includes too many attributes

–     Problem : overburdening of a concept may decrease its usefulness by making it a concept that has no empirical referents

–     Minimalist Definitions

–     Problem: if a concept is so minimalist that all cases automatically become instances, researchers must add attributes to a concept as a way to give it more content

–     conceptualization— the vertical organization of attributes by level of abstraction—all existing indices of democracy carefully distinguish the level of abstraction of their attributes and thus clearly isolate the leaves of their concept trees

–     Problems with inclusiveness, using binary codings



–     Measurement: conceptual attributes identified and logically organized during the prior step with observations

–     The more one seeks to form measures for the purpose of cross-time and cross-space comparisons, the more necessary it becomes to avoid the potential biases associated with single indicators by using multiple indicators

–     Problems with selection of indicators

–     The choice of measurement level should be open to testing, in the sense that the analysts should consider the implications of different assumptions about the level of measurement and use an assessment of these implications in justifying their choices

–     Another standard of assessment pertains to the replicability of measures

–     Critiques existing indices of democracy for not having tackled the challenge of measurement very well



–     To determine whether and how to reverse the process of disaggregation that was carried out during the conceptualization stage

–     The move to a higher level of aggregation may entail a loss of validity, in that information about systematic variation among the cases may be lost

–     The standard practice with regard to the selection of the level of aggregation has been to proceed as though parsimony were the only consideration, fully warranting a decision to push the process of aggregation to the highest level possible so as to reduce the disaggregate data into one single score

–     Rules of aggregation: First, the analyst must make explicit the theory concerning the relationship between attributes. Second, the analyst must ensure that there is a correspondence between this theory and the selected aggregation rule


Outcome: this review shows that no single index offers a satisfactory response to all three challenges of conceptualization, measurement, and aggregation


Critique: The authors indicate that research often errs too much on the side of minimalist definitions of abstract terms such as democracy, however, they do not acknowledge how democracies the reside in the same category may change over time; as some of the fundamental tenants of democracy (i.e. participation and contestation) are broadened in a particular democracy, and these cases become more inclusive, minimalist definitions will fail to encapsulate the movement of the cases even when they are still classified under the same category [i.e. how Canada – always classified as a liberal democracy – shifted to being more participatory after the enfranchisement of women.]