Mixed methods in VET research: Usage and quality
Senior Lecturer (HRM), School of Management and Marketing, FABIE, Central Queensland University, Gladstone QLD
PP: 25 - 39
Mixed methods had been heralded as the third methodological movement with several authorities from across an array of discipline fields contributing to a growing body of literature and theoretical developments. The discipline fields which are showing high levels of acceptance for mixed methods are those related to education, health and nursing, social and behavioural sciences and business/management disciplines. This third methodological movement is now beginning to address issues of quality in the reporting of mixed methods studies. The movement is beginning to question whether researchers utilising mixed methods have gone beyond the one dimensional and relatively rudimentary concept of triangulation to embrace the more complex designs and methodological theory being developed. The aim of the study is to explore the use and quality of mixed methods in vocational education research through a systematic review of a specific sample of vocational education and training (VET) based research. The paper concludes with a call for mixed methods in higher degree research training curricula and a need for established VET researchers to engage with the mixed methods foundational literature and the new and more complex theoretical developments that are emerging.
mixed methods, VET research, research quality, research design, multistrand conversion mixed model, research training
Vocational Education and Training (VET) is a term used internationally to describe education and training arrangements designed to prepare people for work or to improve the knowledge and skills of people already working. VET is one of the three major sectors of education and training in Australia, the other two being the school and higher education sectors. VET covers education and training both before and during employment. People may undertake VET throughout their working lives. VET is provided in colleges and other training institutions, skill centres found in larger companies and in the workplace, in the provision of adult education, community based education and through private training providers.
Vocational Education and Training is supported by the Commonwealth and State governments in Australia primarily through TAFE Institutes (TAFE means Technical [or Training in some States] and Further Education) and TAFE Divisions in dual sector universities. The Australian Vocational Education and Training Research Association (AVETRA) is Australia's only national, independent association of researchers in vocational education and training (VET).
AVETRA was established in 1997 with initial financial assistance from the Australian National Training Authority Research Advisory Council and received support from the National Centre for Vocational Education Research (NCVER). AVETRA is committed to the following principles:
- Furthering the contribution of VET research to the development of Australian VET policy
- Independently reviewing VET research priorities
- Promoting independent and significant research in VET, with due regard for the intellectual properties of researchers
- Raising the profile of VET research in the academic and training communities
- Promoting training in researching methods for those working ion or with the Vet sector
- Providing platforms for the dissemination of research findings (www.avetra.org.au)
The International Journal of Training Research (IJTR) is published twice a year and is the official research journal of AVETRA. The journal publishes articles which advance the knowledge and understanding of vocational education and training in Australia and internationally and focuses on current or recently completed research and reviews of research in VET.
This paper reports findings from a systematic review of conference papers from the annual AVETRA conference held in 2007 and 2008 and journal articles from the International Journal of Training Research (IJTR) from 2003-2008. The paper will briefly outline the rise of mixed methods as a third methodological movement and the importance of quality frameworks in reporting mixed methods studies before detailing the research aims, design, methodology and findings. The research has utilised a multistrand conversion mixed model research design with an overarching research question and separate quantitative and qualitative sub-questions. The systematic review provides a broad based scan of methodological use using the following paper/article categories: conceptual; qualitative; quantitative; and mixed methods. The study then classifies those papers/articles identified as mixed methods in terms of data collection and analysis. This is followed by a qualitative analysis of the mixed methods papers/articles using a set of mixed method quality criteria. The paper concludes by proposing future research in this area and by discussing the implications for building mixed methods research capacity for VET researchers. It is hoped that the paper itself acts as an exemplar for the reporting of a mixed methods study and has aimed to achieve this through following the GRAMMS framework for quality reporting of mixed methods studies developed by O'Cathain, Murphy and Nicholl (2008).
The rise of mixed methods as a third methodological movement
Tashakkori and Teddlie call mixed methods the 'third methodological movement' (2003, p. ix) whilst Johnson and Onwuegbuzie (2004, p. 14) state very clearly that mixed methods research is a 'research paradigm whose time has come'. Mixed methods research is an emerging methodological movement with a growing body of trans-disciplinary literature. Prominent research methodologists/authorities from across discipline fields are emerging and guiding the commentary and the movements' momentum. Creswell and Plano Clark have concluded that 'today, we see cross-cultural international interest, interdisciplinary interest, publication possibilities, and public and private funding opportunities for mixed methods research' (2007, p. 18).
Several definitions of mixed methods exist. For example the Journal of Mixed Methods Research (2006), in its call for papers defines mixed methods as 'research in which the investigator collects, analyses, mixes, and draws inferences from both quantitative and qualitative data in a single study or a program of inquiry'. Johnson and Onwuegbuzie (2004) prefer to define mixed methods research as that in which the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study. Creswell and Plano Clark (2007, p. 5) define mixed methods as follows:
Mixed methods research is a research design with philosophical assumptions as well as methods of inquiry. As a methodology, it involves philosophical assumptions that guide the direction of the collection and analysis of data and the mixture of qualitative and quantitative data in a single study or series of studies. Its central premise is that the use of quantitative and qualitative approaches in combination provides a better understanding of research problems than either approach alone.
The overall purpose and central premise of mixed methods studies is that the use of quantitative and qualitative approaches in combination provides a better understanding of research problems and complex phenomena than either approach alone (Creswell & Plano Clark 2007). Mixed methods research designs are said to add value through providing a more comprehensive and richer understanding of the research problem being investigated. This is because the approach explores the problem through the different lenses and perspectives of qualitative and quantitative research techniques. Greene, Caracelli and Graham (1989) developed five purposes for mixed methods studies: triangulation (to seek convergent results); complimentarity (to explore interconnected and/or distinct aspects of a phenomenon); initiation (to examine similarities, contradictions and new perspectives); expansion (to add breadth and scope to a research study) and; development (use of results from one method to help develop the use of the other method). Bryman (2008) has expanded on these mixed methods purposes further through the compilation of a list of seventeen purposes. In a study of 232 social science journal articles utilising mixed methods Bryman (2008, pp 92-93) found the most popular purposes or rationales for utilising a mixed method approach were 'enhancement' (31%) followed by 'completeness' (13%), 'sampling' (13%) and 'triangulation' (12%).
Mixed methods grew in popularity in specific discipline fields (education, health, nursing and social sciences) and emerged out of the UK and continental Europe before catching the eye of academics and researchers in the USA. Interest in mixed methods in Australian based research to date has come from the social sciences, health and nursing and business/management fields. The interest in mixed methods has seen the recent emergence of several publications including academic journals, chapters within research texts (Nagy Hesse-Biber & Leavy 2008; McMillan & Schumacher 2006) and research texts themselves that are dedicated to mixed methods. The most comprehensive publication of mixed methods to date has been the edited Handbook of Mixed Methods in Social and Behavioural Research (Tashakkori & Teddlie 2003). Another very practical guide to conducting mixed methods is a book by Creswell and Plano Clark (2007) Designing and Conducting Mixed Methods Research. Other texts focused solely on mixed methods include: Greene 2007; Bergman 2008; Teddlie and Tashakkori 2009. Discipline specific mixed methods texts are also being published: in the health and nursing field (Andrews & Halcomb 2009); business and policy research (Cameron & Miller forthcoming) and; psychology (Todd, Nerlich, McKeown & Clark 2004). Academic journals dedicated to mixed methods research include:
- Journal of Mixed Methods Research
- International Journal of Multiple Research Approaches
- The International Journal of Mixed Methods in Applied Business and Policy Research
Over the last several years there has emerged over 40 mixed methods research designs. This can be very confusing for doctoral candidates, early career researchers and even established researchers. In response to this Leech and Onwuegbuzie (2009) developed a three dimensional, integrated typology of mixed methods research designs. The authors saw the need to limit the level of confusion related to the plethora of mixed methods research designs available and to assist researchers in simplifying design choices when first deciding to engage in mixed methods research.
In practical terms those wishing to utilise mixed methods research designs need to be proficient in both qualitative and quantitative research methods. In addition to this they need to be very familiar with a growing body of literature and theoretical discussions on mixed methods. No matter what research method employed, the researcher needs to rigorously defend the methodological choices made. Creswell and Plano Clark have concluded that 'today, we see cross-cultural international interest, interdisciplinary interest, publication possibilities, and public and private funding opportunities for mixed methods research' (2007, p. 18). The aim of this paper is to gauge the use and quality of mixed methods research within the VET based research community, as represented by two AVETRA conferences and the International Journal of Training Research.
The use and quality of mixed methods research across discipline fields ...continues ...
Andrew, S and Halcomb, E 2009, Mixed methods research for nursing and health Sciences, Wiley-Blackwell, London.
AVETRA n.d., Australian Vocational Education & Training Research Association. Retrieved 20 June, 2009, from: http://www.avetra.org.au/
Bazeley, P 2008, 'Mixed methods in management research', in Dictionary of qualitative management research, eds R. Holt & R. Thorpe, Sage, London, pp. 133-136.
Bergman, M 2008, Advances in mixed methods research, Sage, Thousand Oaks, CA.
Bryman, A 2008, 'Why do researchers integrate/combine/mesh/blend/mix/merge/fuse Quantitative and Qualitative research? in Advances in mixed methods research, ed. M Bergman, Sage, Thousand Oaks, CA.
Cameron, R 2008, 'Mixed methods in management Research: Has the phoenix landed?', 22ndt Annual Australian & New Zealand Academy of Management (ANZAM) Conference, Auckland, December 2008.
Cameron, R 2009, 'The use of mixed methods research in VET research', 12th AVETRA Annual Conference, Aligning participants policy and pedagogy: Traction and tension in VET Research, Coogee, 15-17 April 2009.
Cameron, R and Miller, P forthcoming, Mixed methods in applied business and policy research, Pearson, Frenchs Forest NSW.
Creswell, JW and Plano Clark, VL 2007, Designing and conducting mixed methods mesearch, Sage, Thousand Oaks, CA.
Creswell, JW, Tashakkori, A, Jensen, KD and Shapley, K 2003, 'Teaching mixed methods research: Practices, dilemmas, and challenges', in eds A Tashakkori and C Teddlie (2003) Handbook of mixed methods in social & behavioral research, Sage, Thousand Oaks, CA.
Eberle, T and Elliker, F 2005, 'A cartography of qualitative research in Switzerland', Forum: Qualitative Social Research, vol 6, no. 3, Art 24- September.
Greene, J 2007, Mixed methods in social inquiry, Jossey-Bass, San Francisco.
Greene, J and Caracelli, V Eds. 1997, Advances in mixed-method evaluation: The challenges and benefits of integrating diverse paradigms, Jossey-Bass, San Francisco.
Greene, J, Caracelli, V and Graham, WD 1989, 'Toward a conceptual framework for mixed-method evaluation designs', Educational Evaluation and Policy Analysis, vol 11, no. 3, pp255-74.
Guenther, J, Falk, I, Arnott, A, Borgelt, K, Tyrrel, A, Churchill, J, Black, S and Gunn, J 2007, 'VTE in welfare to work: Does it work well', 10th AVETRA Annual conference, The new context for VET, Victoria University, Melbourne, Footscray Park, April 11-13, 2007.
Hanson, W, Creswell, J, Clark, V, Petska, K and Creswell, J 2005, 'Mixed methods research designs in Counselling Psychology', Journal of Counselling Psychology, vol 52, no. 2, pp. 224-235.
Hurmerinta-Peltomaki, L and Nummela, N 2006, 'Mixed methods in international business research: A Value-added perspective', Management International Review vol 46, no. 4, pp. 439-459.
Johnson, RB and Onwuegbuzie, AJ 2004, 'Mixed methods research: A research paradigm whose time has come', Educational Researchers, vol 33, no. 7, pp. 14-26.
Leech , NL and Onwuegbuzie, AJ 2009, 'A typology of mixed methods research designs', Quality & Quantity International Journal of Methodology, vol 43, pp. 265-275.
McMillan, J and Schumacher, S 2006, Research in education: Evidence-based inquiry, 6th edn, Pearson, Boston MA.
Mingers, J 2003, 'The paucity of multimethod research: a review of the information systems literature', Information Systems Journal, vol 13, no. 3, pp. 233-249.
Molina-Azorin, J 2007, 'Mixed methods in strategy research: applications and implications in the resource-based review', Research Methodology in Strategy and International Review, 2007, vol 4, pp. 37-73.
Nagy Hesse-Biber, S and Leavy, P 2008, Handbook of emergent methods, Guilford Press, New York.
Niglas, K 2009, 'How the novice researcher can make sense of mixed methods designs', International Journal of Multiple Research Approaches, vol 3, no.1, pp. 34-46.
Novakovic, N 2008, 'Thinking on the edge: the influence of discussion and statistical data on awarders' perceptions of borderline candidates in an Angoff awarding meeting', International Journal of Research Training, vol 6, no.1, pp74-102.
O'Cathain, A, Murphy, E and Nicholl, J 2008, 'The quality of mixed methods studies in health services research', Journal of Health Services Research and Policy, vol 13, no. 2, pp. 92-98.
Plano Clark, V 2005, Cross-disciplinary analysis of the use of mixed methods in physics education research, counselling psychology, and primary care, Doctoral dissertation, University of Nebraska-Lincoln, Dissertation Abstracts International, 66, 02A.
Rocco, TS, Bliss, LA, Gallagher, S, Perez-Prado, A, Alacaci, C, Dwyer, ES, Fine, JC, and Pappamihiel, NE 2003, 'The paradigmatic and dialectical lenses: two views of mixed methods use in eduaction' in Handbook of Mixed Methods in Social & Behavioral Research, eds A Tashakkori and C Teddlie, Sage, Thousand Oaks, CA.
Rocco, TS, Bliss, LA, Gallagher, S, and Perez-Prado, A 2003, 'Taking the next step: Mixed methods research in organisational systems', Information Technology, Learning and Performance Journal, vol 21, no.1, pp. 19-29.
Tashakkori, A and Teddlie, C 1998, Mixed methodology: Combining qualitative and quantitative approaches, Sage, Thousand Oaks, CA.
Tashakkori, A and Teddlie, C (Eds.) 2003, Handbook of mixed methods in social & behavioral research, Sage, Thousand Oaks, CA.
Teddlie, C, and Tashakkori, A 2009, Foundations of mixed methods, Sage, Thousand Oaks, CA.
Todd, Z, Nerlich, B, McKeown, S, and Clark, D (Eds.) 2004, Mixing Methods in Psychology: The integration of qualitative methods in theory and practice, Psychology Press, New York.