Of Methods and Methodology: 5. Triangulation

methodological pluralism

While it’s necessary, for the sake of illustration, to differentiate between different sociological methodologies, this doesn’t mean positivism and interpretivism simply occupy their own unique social space into which the other cannot enter – an idea reflected in the notion “positivists” would not use qualitative methods for methodological reasons, because such methods “lack reliability”, for example, while “interpretivists” would not use quantitative methods because they “lack validity”.

Rather than see methodologies as being entities whose basic principles are set in stone, it’s more-useful to see them as mental constructs created for theoretical convenience; to help us understand and evaluate, for example, methodological principles such as reliability and validity. In this respect the question of whether we should expect to meet such methodologies in their “pure forms” in the real world of sociological research may be somewhat wide of the mark given that, as Wood and Welch (2010) argue:

“There is now increasing awareness that both quantitative and qualitative styles of research may have a contribution to make to a project, which leads to the idea of mixing methods“.

This idea can be expressed as methodological pluralism, something Payne et al (2004) define as “tolerance of a variety of methods”. It refers, in other words, to the idea of combining research methodologies in ways that allow each to complement the other to improve overall research reliability and validity.

The logic of this argument is that different research methods have different methodological strengths and weaknesses; questionnaires, for example, may produce reliable data, but with low validity (although, once again, this relationship is by no-means set in stone – depending on what is being measured, questionnaires are not methodologically incapable of producing valid data), while the reverse may be true for covert participant observation. 

Rather than approach research methodology from the perspective of a “design problem” therefore – how to test a hypothesis (positivism) or answer a research question (interpretivism) we can approach it from a methodological perspective – how to collect data that has the highest possible levels of reliability and validity, regardless of the actual methods or data types used. In this respect, if methodological pluralism represents the theoretical justification for using mixed methods – because no research method or data type is intrinsically “positivist” or “anti-positivist” – triangulation is the means through which this theory is put into practice.

triangulation

This refers to the various ways a researcher can attempt to improve research reliability and validity and it takes a number of forms:

1. Methodological triangulation refers to what Denzin (1970) calls the use of two or more research methods, the general idea being the researcher can off-set the weaknesses of one method with the strengths of another as a means of improving reliability and validity.

For example, if a general weakness of questionnaires is that the researcher must assume a respondent is “telling the truth” (or that they’re even aware of what they’re saying might not be true), a researcher could off-set this by using an observational method, such as participant observation, to check respondents actually do what they say they do.

In this way a combination of methods can give a more rounded picture of someone’s life and behaviour; a researcher could, for example, observe a respondent’s behaviour using participant observation and also question them about why they did particular things or behaved in one way rather than another.

Alternatively, the researcher could compare the results from two different methods used on the same people (such as a semi-structured interview and a focus group) and if the conclusions drawn are broadly the same this “internal research replication” (using one method to check the data from another method within a piece of research) may be used to confirm the reliability and validity of the data.

Hughes et al’s (1997) examination of “the appeal of designer drinks to young people”, for example, used focus groups and structured interviews, the data from one being used to cross-check and confirm data from the other: each method, for example, showed a strong pattern of age-related differences in attitudes to designer drinks. This approach can be further subdivided in terms of:

within-method triangulation, a technique Bryman (2001) characterises as “the use of varieties of the same method to investigate a research issue”. On a simple level this might involve asking open and closed questions in the same questionnaire.

between-method triangulation, on the other hand, is characterised by what Bryman (2001) calls the use of “contrasting research methods, such as a questionnaire and observation”.

In this respect, Harvey and MacDonald (1993) summarise the use of researcher-focused methodological triangulation as involving any combination of:

two or more researchers using same research technique.

one researcher using two or more research techniques.

two or more researchers using two or more research techniques.

Trochim (2002) argues that all research methods contain “the capacity for error” and for this reason the only sensible thing a researcher can do is combine methods so that one type of error cancels out another. Sociological research, therefore, should of necessity involve collecting different types of data, qualitative and quantitative, primary and secondary – something that serves three main purposes:

To check data reliability and validity.

To allow different researchers using the same method to compare data for similarities and differences.

To verify and confirm the accuracy of different types of data.

Methodological triangulation can therefore, take further forms:

researcher triangulation can be used in studies that rely heavily on a researcher’s interpretations to generate data; if different researchers using the same research method arrive at the same results this helps to confirm data reliability.

alternatively, using researchers from different ethnic, age, gender and class groups can help check for things like observer and interviewer bias – things that lower research reliability and validity.

2. Data triangulation involves gathering information through different sampling strategies – such as collecting data:

at different times,

in different contexts and

from different people.

This idea can be extended to include gathering data from both the individuals involved in a particular situation and the researcher’s own experiences in that situation. Venkatesh (2009), for example, was able to make sense of certain forms of behaviour – such as drug dealing –  and experiences – such as being black and poor – in ways that would not have been possible if he had not been intimately involved in the world he was studying. He gathered data from both those involved – their understanding of what it meant to be black and poor, for example – and from his own experience of living in their world.

3. Theoretical triangulation is based on the idea that just as research methods are inherently error prone, so too are theoretical positions. Different theoretical perspectives, such as Functionalism or Interpretivism, have their strengths and their weaknesses that, again, can be used to the researcher’s advantage. The argument here is that by looking at the social world in terms of both structure and action, for example, we can arrive at the best possible representation and explanation of social behaviour.

To illustrate these ideas we can point to a variety of sociological research that has used methodological triangulation to good effect:

Barker (1984), for example, used overt participant observation, questionnaires and semi-structured interviews in her research on the Unification Church (“Moonies”).

Hey (1997) studied girls’ friendships in two London schools using a combination of participant observation and personal documentation – some of the girls allowed her to read their diaries and she was also given access to the notes girls passed between each other in the classroom.

MacKeogh (2001) studied the “micro-politics of family viewing” in relation to how young people used television and their parents’ attempts to control how and what they watched. Her primary method was overt participant observation, because she wanted to understand the critical awareness young people had about the media they consumed. Her observation notes were complemented by notes made by her respondents as well as semi-structured interviews used to explore some of the issues raised in the observations.

Garforth and Kerr (2010) examined ” Women’s under-representation in science”, using a mix of interviews, focus groups and participant observation.

Methodological pluralism and triangulation are, therefore, frequently employed by sociologists because they improve research reliability and validity; because all research methods have their strengths and weaknesses the strengths of one method can be used to compensate for the shortcomings of another.

triangulation: an holistic strategy

As Parke and Griffiths’ (2002) argue:

“One obvious advantage of non-participant observation is that it relies only on observing behaviour. Since the researcher cannot interact in the behavioural processes, most data collected will be qualitative, interpretative, and to some extent, limited. However, by using other methodological research tools (e.g. structured interviews), suspicions, interpretations and even hypotheses can be confirmed“.

In addition, by gathering and aggregating different types of data and sources (such as respondents and participant observers) the researcher is more-likely to get a complete, fully-rounded (holistic) picture of the behaviour they’re studying.

By using different methods and sampling strategies a researcher can generally improve overall data reliability and validity. More specifically, data collected using higher reliability methods, such as questionnaires, can off-set reliability weaknesses in observational methods, with the reverse being the case for validity. Finlay (1999), for example, compared accounts of the same events given by different respondents in semi-structured interviews and added a further check by comparing “the oral record of those events with the contemporary documentary record in…local newspapers”.

Finally, a researcher’s confidence in things like the accuracy of their data can be increased using triangulation. As Bechhoffer and Paterson (2000) argue: “If we are…able to base part of an explanation on unstructured interview material, on documentary evidence and on the results of a survey, our confidence in our findings is likely to be greatly increased“.

problems: practical and theoretical

While these arguments in favour of methodological pluralism and triangulation are important, it doesn’t mean these techniques are without their practical problems in the sense triangulation adds another layer of time, effort and expense to research, in terms of things like:

the time needed to analyse different data types created from a number of different methods.

the need to employ more researchers – and the cost involved.

the general co-ordination of a much larger research project.

In terms of theoretical problems Bryman (2001) notes triangulation is sometimes seen as a way of getting at “the truth” by throwing a vast array of resources, methods and data at a problem, based on the (naive) idea “there can be a single definitive account of the social world”.

In addition, collecting and comparing different types of data has its complexities, particularly but not exclusively because such data may not always be easily and neatly compared. As Bryman (2001) argues:

 “Triangulation assumes data from different research methods can be unambiguously compared and regarded as equivalent in terms of their capacity to address a research question“.

This assumption may, of course, be incorrect: differences arising between the data from, for example, a structured interview and a focus group may have less to do with the reliability and validity of each method and “more to do with the possibility that the former taps private views as opposed to the more general ones that might be voiced in the more public arena“.

More-specific reliability and validity problems link from the above in the sense that where a researcher gets contradictory data from two different sources it can be difficult to disentangle “truth” from “falsity”; if the researcher receives two opposing accounts of the same thing, which account is true?

And more importantly, how can the researcher tell?

References

Barker, Eileen (1984) “The Making of a Moonie: Brainwashing or Choice?”: Blackwell

Bechhoffer, Frank and Paterson, Lindsay (2000) Principles of Research Design in the Social Sciences: Routledge

Bryman, Alan (2001) “Triangulation”: Loughborough University

Denzin, Norman (1970) “Sociological Methods: A Sourcebook”: Aldine

Finlay ,Andrew (1999) ”Whatever You Say, Say Nothing’: An Ethnographic Encounter in Northern Ireland and its Sequel “:Sociological Research Online, vol. 4, no. 3,

Garforth, Lisa and Kerr, Anne (2010) “Let’s Get Organised: Practicing and Valuing Scientific Work Inside and Outside the Laboratory”: Sociological Research Online:

Hey, Valerie (1997) ” The Company She Keeps: An Ethnography of Girls’ Friendships”: Open University Press

Hughes, Kirsty;  MacKintosh, Anne Marie; Hastings, Gerard; Wheeler, Colin; Watson, Jonathan and  Inglis, James (1997) “Young people, alcohol, and designer drinks: quantitative and qualitative study”: British Medical Journal, 314 : 414

MacKeogh, Carol (2001) “Taking Account of The Macro in The Micro-Politics of Family Viewing – Generational Strategies”: Sociological Research Online, vol. 6, no. 1

Parke, Jonathon and Griffiths, Mark (2002). Slot machine gamblers: why are they so hard to study?”: Journal of Gambling Issues, Issue 6

Payne, Geoff; Williams; Malcolm and Chamberlain, Suzanne (2004) ” Methodological Pluralism in British Sociology”: BSA Publications Ltd / SAGE Publications

Trochim, William, 2002, ‘Positivism and Post-Positivism’: Research Methods Knowledge Base, Cornell University

Venkatesh, Sudhir (2009) ” Gang Leader for a Day”: Penguin

Wood, Michael and Welch, Christine (2010) “Are ‘Qualitative’ and ‘Quantitative’ Useful Terms for Describing Research?”:  Methodological Innovations Online 5(1)