RESEARCH IN THE SOUTH AFRICAN JOURNAL OF ECONOMIC AND MANAGEMENT SCIENCES : AN INVESTIGATION OF TRENDS IN THE LITERATURE

Mixed methods research (MMR), which is touted as a third methodological movement is increasingly becoming a popular approach in several fields as a result of the promise it holds to providing a better and balanced investigation of research problems in context. In spite of that, there is limited knowledge about its pervasiveness in economic and management sciences in South Africa. Based on a content analysis of 332 articles published in The South African Journal of Economic and Management Sciences from 2003 to 2011, the main purpose of this quantitative study is to explore the prevalence of MMR in SAJEMS. Although methodological advances have been made in the field of economic and management sciences as reflected in the articles in SAJEMS, the findings reveal that scholars employ quantitative and qualitative methodologies than MMR. Given the paucity of MMR in the field, this study underscores the potential benefits of embracing methodological pluralism as it offers methodological and theoretical benefits. First, the use of MMR provides the possibility for researchers to obtain a comprehensive picture of a phenomenon under investigation and achieve their research purpose effectively. Secondly, its utilisation may also contribute to theory development and the maturity of the field as reflected in SAJEMS.


Introduction and context
Mixed methods research (MMR) is regarded by some scholars as a third methodological movement (Teddlie & Tashakkori, 2003:ix) that advocates (Teddlie & Tashakkori, 2012: 776) methodological eclecticism which involves the utilisation of quantitative and qualitative approaches within a single study.Although, there is no consensus on various aspects of MMR including, definitions and descriptions of mixed designs (Teddlie & Tashakkori, 2012), many disciplines have embraced MMR (Ngulube, 2013).However, little is known about the extent to which MMR is utilised in the economic and management sciences in South Africa.
The rationale of the case study reported in this article is to provide a lens through which to appreciate the prevalence of MMR in economic and management sciences as it is reflected by the contributors to the South African Journal of Economic and Management Sciences (SAJEMS).The majority of the contributors to SAJEMS are from institutions in South Africa as only 15.5 per cent of the articles in SAJEMS between 2004 and 2010 were from institutions outside the country (Jordaan et al., 2013).
The question, which is the major focus of this article, is: To what extent has research in economic and management sciences reported in SAJEMS embraced the multiple research perspective?To answer the question, we examined the research methods used in Abstract SAJEMS between 2003 and 2011.Thomson Reuters recommends at least five years when deciding which years of publications and citations to use to measure research impact (Pendlebury, 2010).
Assessment of the methodological procedures is increasingly becoming a popular research enterprise.Examining the prevalence rates of methodological approaches within the social sciences is a new line of research that is emerging in MMR (Alise & Teddlie, 2010: 103).In this line of research, this study examined the methodological trends in SAJEMS in order to determine the proportion of articles using MMR and the perceived rationale of using MMR.Many researchers often confuse MMR with triangulation.The misunderstanding stems mainly from the fact that MMR 'implies triangulation' (Howe, 2012:89).There is a need to clarify the confusion so that the reader may appreciate the conceptual differences between MMR and triangulation, and establish a foundation for the classification system we used for the articles that were analysed in this research.

Contextualising mixed methods research
Multiple research strategies are becoming researchers' choices of method due to the fact that methodological pluralism provides better quality data than a single approach (Creswell & Garrett, 2008).That stems from a paradigm shift from the use of mainstream mono methods, or the subscription to the incompatibility thesis, or the perpetuation of the paradigm wars to the utilisation of multiple research perspectives.There is mounting evidence that: '...there is no single omnipotent research method.All methods have advantages as well as limitations.Not to acknowledge these limitations is the central flaw of monomethodology' (Van Peer, Hakemulder & Zyngier, 2012:53).This is not to say that the traditions of disciplinary boundaries should be set aside.Using a single research paradigm and insisting on disciplinary boundaries entrap scholars in a false assumption that knowledge is a certainty in the Wallerstein (2004) sense.Multiple perspectives give scope for interdisciplinarity.By supporting multiple research approaches we are mindful of the fact that they should not be used for the sake of it, but rather for their ability to answer certain kinds of research questions.
Although, the major consideration when deciding on methodology is to ascertain which approach will best answer the research question, many fields are advocating and using mixed methods (De Loo & Lowe, 2011) because of the value and advantages that they offer.In essence, 'qualitative data provide a detailed understanding of a problem while quantitative data provide a more general understanding of a problem' (Creswell & Plano Clark, 2011:8).Despite the growing popularity of mixed methods research (MMR) in other fields its discussion and utilisation in economic and management sciences is relatively new.
Even though integration of methods has a long history in research practice, mixed methods research became prevalent from the 1990s as a result of the desire among researchers to overcome the tensions in the epistemological, ontological, methodological, axiological and doxological differences of quantitative and qualitative research (Tashakkori & Creswell, 2007).That led to the emergence of terms such as multi-method research, nested analysis, multiple stranding, mixing, blending, combining and integrating in dealing with both research perspectives.We deliberately prefer to use the term mixed methods research (MMR) as it has gained currency in the extant literature (Doyle, Brady & Byrne, 2009) and it captures nicely the essence of combining research methodologies better than the other terms.MMR is an: umbrella term applying to almost any situation where more than one methodological approach is used in combination with another, usually, but not essentially, involving a combination of at least some elements drawn from each of qualitative and quantitative approaches to research (Bazeley, 2008:133).
It is noteworthy that Christ (2010) recently discussed combinations of multiple qualitative approaches in the context of MMR.However, the use of multiple qualitative approaches is often discussed using the concept of triangulation as is briefly demonstrated in the next section.We take the definition given by Bazeley (2008) as the background for our study, although the debate on the definition of MMR is still raging in academic circles.Whatever definition is preferred, what is useful is to ensure that mixing is not treated in a superficial manner as explicated in section 2 of this article.It should be both at philosophical and methodological levels rather than only at the research methods phases.
The unprecedented growth of MMR in the last twenty years may be attributed to several factors such as (Ngulube, 2013) • not all the research questions may be answered by MMR as 'the best method for any given study... may be purely QUAL or purely QUAN, rather than mixed' (Tashakkori & Teddlie, 2010a:10); and • there is no consensus on the core characteristics of the field (Teddlie and Tashakkori, 2012).Moreover, some sceptics (Doyle, Brady & Byrne, 2009;Tashakkori & Teddlie, 2010b) argue that MMR is: • inherently more expensive than using either the quantitative-only or the qualitative-only approaches • more time consuming than using either the quantitative-only or the qualitative-only approaches, especially for time bound projects such as master's and doctoral studies, and contracted work • difficult to implement and most MMR studies have a postpositivist bias with a tendency of subordinating the qualitative strand to the quantitative one.Although Tashakkori and Teddlie (2010b:818) went a long way towards addressing these criticisms, we would like to emphasise that we agree with many theorists that the research approach used in any investigation should be determined by the research problem and its epistemological framing.Indeed, scholars such as Saunders, Lewis and Thornhill (2007) argue that the research problem determines the research approach and the methods employed in relation to data needed to answer the questions, where such data is and how such data is to be collected and analysed.This does not in any way weaken the case of MMR.Both words and numbers convey meaning which is needed in fully understanding the world.It is evident that there is benefit in combining the complementary strengths of qualitative and quantitative research methods when conducting a study.MMR has value to various disciplines.The major attraction of MMR is that it: can simultaneously address a diverse range of confirmatory and exploratory questions, and single-approach studies often address only one or the other (Tashakkori & Teddlie, 2010a:9).

Mixed methods research versus classical triangulation
Denzin (2012:80) cautions researchers not to confuse triangulation for mixed methods research.However, our experience in supervising graduate research in the past ten years demonstrates that there is a lot of confusion around 'triangulation' as a concept used in social science methodology and 'triangulation' as a research design in MMR (Creswell, 2008;Greene, Caracelli & Graham, 1989).Indeed, 'the transfer of the notion of triangulation from trigonometry to the realm of mixed methods [appears] to have transformed it into a fuzzy idea with a variety of possible meanings' (Erzberger & Kelle, 2003:461).Further, the term triangulation has been used so much that it has lost its meaning altogether (Sandelowski 2003).Although, classical triangulation may be traced to the 'multi-trait, multi-method matrix' of Campbell and Fiske (1959), its original usage in social science research was linked to validity checking through the use of more than one source, or method of data collection (Hammersley, 2008:23).
Many researchers continue to confuse triangulation with mixed methods design (Creswell, Plano Clark & Garrett, 2008:68).It is not uncommon to find a researcher including some interviews and participatory observation data within a quantitative survey design and construing the research approach to be MMR.Rather, such studies should be termed mixedmode research (see De Leeuw & Hox, 2008) or multi-methods research (Niglas, 2004).MMR goes beyond a mere superficial combination of data from quantitative and qualitative methods.For instance, in methodological triangulation the aim is to check an answer, 'not so as to gain further information in order to produce an answer' (Hammersley, 2008:23).This may involve different sources of qualitative data (e.g. interviews, documents, observation) and not necessarily quantitative and qualitative sources.The concern of methodological triangulation is validation and not development.In that sense it is more of a measurement technique than a research design.
Classical triangulation may also be used as a means of generating divergent interpretations (e.g.reasons why two interviewees given different accounts of the same event) (Hammersley, 2008:26).It may also seek complementary information.In short, triangulation, 'helps us to recognize the limits to what any particular type of data can provide' (Hammersley, 2008:32).On the other hand, the use of triangulation as an MMR design is based on the epistemological assumptions underpinning the choice.In MMR, the triangulation design entails collecting and analysing data concurrently and then comparing the results to bring out a comprehensive picture about the phenomenon under investigation.

Statement of the problem and research questions
The use of MMR by researchers in a discipline indicates their degree of awareness of the advantages that MMR provides in conducting their research.Little is known about the use and prevalence of MMR in management and economic sciences in South Africa.The primary research questions that guided the study were:

Methods and materials
The research questions were investigated by conducting a content analysis of 332 articles that were published in The South African Journal of Economic and Management Sciences (SAJEMS).SAJEMS was a purposively sampled because it is a refereed ISI accredited scientific journal which claims to promote 'interdisciplinary research that breaks down common intellectual silos and prepares a new path for debate on the operation and development of markets in and around Africa' and most of its contributors are from South Africa (Jordaan et al., 2013).Using content analysis as a research strategy, this contribution aims at extending the frontiers of that debate by bringing in a methodological dimension.
Following, Alise and Teddlie (2010) and Venkatesh, Brown and Bala (2013) the study reviewed articles from a referred journal instead of monographs and other vehicles of scholarly communication to analyse research trends in SAJEMS.Although peer-reviewed journals are not the only way of promoting scholarly communication, it is widely agreed that scientific journals publish a significant portion of our scientific knowledge (Bryman, 2008).Creswell and Garrett (2008) have also suggested that journals are one of the indicators that may be used in measuring the extent of the growth of MMR in a discipline.
Content analysis has been variously referred to as archival review (Jordaan et al., 2013) or citation analysis (Molina-Azorin, 2010) or informetrics techniques or bibliometrics (Ngulube, 2013).However, the debate on the appropriateness of these terms for a study of this nature is beyond the scope of this paper.Suffice to say that citations may be used to measure trends and patterns in scholarly communication.The present study builds on others that have used content analysis to establish research and scholarly communication patterns (Harrison & Reilly, 2011;Jordaan et al., 2013).
The time span selected for analysis and the number of articles that were evaluated by these studies were arbitrary choices that may have been influenced by, for instance, the purpose of the analysis, circumstances of the author (for instance, the availability of a reliable database), and the sample that was likely to establish the trends that were of interest to the researcher(s).The 332 articles published by SAJEMS between 2003 and 2011 which were available from an online database during the time of this study were examined.December 31, 2011 was used as the cut-off point for the search because data collection started in January 2012.
We based our analysis on the extant literature and on the 332 articles that we gathered.The use of one journal which was purposively selected excludes others and the opportunity to get a broader picture of the practice of MMR in the discipline.Other approaches would undoubtedly reveal other dimensions to the debate.Content analysis used by our research is without its pitfalls: '[h]ow far the analysis of frequencies…in texts is sufficient for answering substantive research questions has been debated since the early days of research with content analysis' (Flick, 2011: 136).The current research could have benefited from a triangulation of research methods.
Interviews with purposively selected participants, (maybe prolific contributors to SAJEMS), might have provided insights that might have been obscured by our research method.Furthermore, we may have surveyed various members of the editorial board of SAJEMS as method of data collection as employed by Short et al., (2010).That might have revealed a deeper understanding of trends in the utilization of MMR in the contributions to SAJEMS.The limitations of this research should serve as an opportunity for other researchers who may conduct methodological prevalence trends in any field.
To gauge the utilisation of MMR in SAJEMS, firstly, we conducted keyword searches for terms such as 'quantitative' and 'qualitative', 'multi-method', 'mixed method', or 'triangulation' to establish if they appeared in the title, keywords, or abstract.Only two articles were generated by using this search strategy.We then decided to manually search each article in the journal.The fact that SAJEMS is an Open Access journal that is available online made the task much easier.The search for articles was conducted by the first author.Each article was coded in terms of methodology employed by the study and research method used as per the typology of Balsley (1970).As an agreement check the second author also independently coded the articles.
A 100 per cent coding agreement was achieved on the methodologies employed in the studies, but the coding agreement in terms of the research methods was only achieved in 236 articles (89 per cent).Thirty remaining disagreements on the typologies of the methods used in the articles were resolved by involving an independent researcher with expertise in research methodologies and economic and management sciences.That means that the coding scheme was reliable within reasonable limits.Table 1 shows the number and types of articles that appeared in SAJEMS between 2003 and 2011 (inclusive).

Empirical results
This section presents results on the trends in research methods; the prevalence of MMR in SAJEMS; the rationale for mixing; the type of MMR designs used; the sequencing and weighting of the methodologies; and the stage of the study were mixing was conducted.

Trends in research methods in the Journal
The research in SAJEMS may be classified by various methods and techniques that are used.The articles were first categorised as empirical and nonempirical using the categorisation of Bergh, Perry and Hanke (2006).Sixty-six articles that did not report data and included articles on theoretical developments and literature reviews were considered to be nonempirical.Using the typology of Balsley (1970:23-24), we classified the methods and techniques used in 266 empirical articles into one of the six possible categories: (a) case study design, (b) motivation research, (c) survey design, (d) operations research, (e) model building, and (f) simulation.Figure 1 illustrates the results.Though there may have been discrepancies in coding one thing is very clear.Research in SAJEMS was dominated by the survey design followed by case study design.

Figure 1 Research methods in the Journal
The further categorisation of aricles into the three major methodologies, viz., qualitative, quantitative and MMR showed that the quantitaive approach was predominant as detailed in Table 1.Qualitative studies and MMR designs accounted for 9 per cent and 2 per cent respectively.
It is evident that the contributions to SAJEMS were dominated by the normative approach predicated on positivism.We guess that it may be partly due to what Lawson (2008:443) refers to as '...(unthinking) adopting of methods assumed to be successfully utilized in the natural sciences or somehow thought, on an a priori basis, to characterize proper science' by many researchers in the discipline.Leitch, Hill and Harrison (2009) and Jogulu and Pansiri (2011) made similar suggestions when researching on entrepreneurship and management, respectively.Although some of the research techniques in management and economic sciences are unique to the field, they share their 'methodology and techniques with other fields of research activities' (Balsley, 1970:1), and they may also employ mixed methods research.

Prevalence of MMR in SAJEMS
The major challenge of assessing MMR studies is that there is no agreed upon language for discussing mixed methods studies (Bryman 2008:88).Following Guest (2012), we checked if the researchers described the study as 'mixed methods' in the title, abstract and methods section.All 332 articles published in SAJEMS from 2003 to 2011 were examined and four articles were identified that mentioned the use of both qualitative and quantitative methods.This represented a prevalence rate of 2 per cent which was lower the 3 per cent occurrence rate of the study by Barnes (2012), 4 per cent rate of Mendenhall, Beaty and Oddou (1993), 5 per cent rate by Alise and Teddlie (2010) and Venkatesh, Brown and Bala, (2013) and the 7 per cent rate of Leech and Onwuegbuzie (2011).The evidence presented here shows that the under-utilisation of MMR is not exclusively confined to SAJEMS.However, the studies were not described in the title, abstract and methods sections as MMR.
The exception was a study by Steyn and Steyn (2006:326) which mentioned that it combined qualitative and quantitative methodologies in cognizance of the advantages offered by the use of multiple methodologies as compared to mono-methodology approaches.Though the study has a bias towards classical triangulation as described in section 2 of this article, a fact that the authors partially admit, it was nevertheless classified as an MMR study because the authors captured the essence of MMR in the explanation of their methodology.The same coding approach was partly used by Bryman (2008:91).
In the methods section we checked whether or not the timing or time orientation (e.g.sequential) and purpose of integration (e.g.convergence) was explicated.Building on Creswell and Plano Clark (2011) it was also of interest to check whether or not the studies stated a mixed methods question and purpose statement, and specified and justified type of MMR design.We also check if the rationale for combining methods was provided.These indicators of the utilisation of MMR designs are summarised in Table 2 and discussed further in the subsequent sections of this article.

Purpose of using mixed methods research
Creswell ( 2008) cautions that MMR should only be used when it is feasible and 'when such an approach clearly adds value to the study and its findings as compared with...' (Lieber & Weisner, 2010:560) using either qualitative or quantitative approaches.Therefore, researchers should give the rationale for combining qualitative and quantitative methods in their study (Brannen, 2008;Bryman, 2008).Many researchers do not make the purpose for choosing a mixed method design in advance (Bryman, 2006).Only one article stated the purpose of mixing (see Steyn & Steyn, 2006).Although a number of the rationales of using MMR exist, eighteen categories devised by Bryman (2008:91-92) were used to code the justifications for combining quantitative and qualitative methods.Similar categories were used in varying degrees by Collins, Onwuegbuzie and Sutton (2006) and Harrison and Reilly (2011).Sixteen out of eighteen categories of Bryman (2008:91) include: triangulation, offset, completeness, process, different research questions, explanation, unexpected results, instrument development, sampling, credibility, context, illustration, utility, confirm and discover, diversity of views and enhancement, and the last two are other or unclear and not stated.Although the scheme is not parsimonious as the author himself admits (Bryman, 2008:91), its attraction is that it seems to be more simplified and comprehensive than some that are available in the extant literature (c.f.Greene, Caracelli and Graham 1989), for instance.It is probable for different researchers to come up with different purposes of mixing in MMR studies using this typology because there are overlaps between these qualitative categories.
Table 2 shows that one article that was examined used MMR for the purpose of instrument fidelity as it assisted with conceptual and instrument development.In line with the typology of Bryman (2008) qualitative focus group discussions were utilised to construct the questionnaire which was pretested before being validated using the survey design within the quantitative methodology.Two other studies used MMR for the purpose of triangulation.Qualitative and qualitative approaches were used to corroborate findings from each of the research strands.The fourth study used MMR for significance enhancement.A quantitative instrument was used to collect the first set of data.It was followed by in-depth interviews to explain the quantitative findings.However, these MMR rationales and typologies were not sufficiently justified by the authors in all the four studies and they never specifically labelled their research designs as MMR.

The mixed methods design types used within SAJEMS
The determination of the mixed methods design types is 'among the most complex and controversial issues in mixed methodology' (Tashakkori & Teddlie, 2003).Many authors have attempted to develop classification systems for MMR designs in order to develop a language of commonly understanding MMR (Tashakkori & Creswell, 2007).
The four major mixed methods designs articulated by Creswell (2008) informed the analysis of the collected data.They include: explanatory, exploratory, triangulation, and embedded mixed methods designs.In explanatory designs, data collected by quantitative means is explained or expanded using qualitative data.The qualitative and quantitative phases occur concurrently and data is transformed by either quanticizing or qualiticizing in the triangulation design.
In exploratory design researchers explore a topic using qualitative methods and then initiate the quantitative phase on the basis of collected qualitative data.In sequential embedded designs collected qualitative data assists in recruiting participants in intervention while in the concurrent scenario, qualitative data is collected to explore the experiences of the experimental group in relation to the intervention while the quantitative strand focuses on the results of the trial (Creswell, 2008).Exploratory and explanatory designs were used in the articles as illustrated in Table 2.

Stage of the study where the mixing was applied
Qualitative and quantitative perspectives may be mixed in a parallel or concurrent way and sequentially ['build on or extend the other type of data'] (Creswell, Plano Clark & Garrett, 2008:66).Generally, there are three main ways of mixing outlined by Brannen (2008:57): • The dominant quantitative component precedes the qualitative one.• The qualitative component follows on from the quantitative one, but it is subsidiary to the qualitative strand.• The quantitative component comes second, but there is no dominant component.These three ways of mixing were used as analytical lenses for conceptualising the nature of the utilization of MMR by researchers in conjunction with the three-dimensional framework provided by Leech and Onwuegbuzie (2009).The framework deals with the level of mixing (i.e.partially vs fully mixed), time orientation (i.e.concurrent vs. sequential), and emphasis of approaches (for instance, equal status vs. dominant status).
The four articles that were categorized as MMR merely mentioned that both qualitative and quantitative data were collected, but did not report results from the two research strands separately.The status of both data sets was not described except in one article where results from both strands were reported separately.The integration of the research methodologies was partially done in a predominantly sequential manner as illustrated in Table 2.The integration of data was done at the interpretation stage.The weighting was biased towards the quantitative strand as in the first type of mixing described by Brannen (2008:57) with the exception of one article.The tendency of subordinating the qualitative strand to the quantitative one has been highlighted as one of the demerits of MMR designs (Denzin, 2012).

Summary and conclusions
The findings demonstrated that the use of MMR in SAJEMS is limited as only four studies could be classified as MMR.However, the low utilization of MMR by scholars is not peculiar to SAJEMS (c.f.Alise & Teddlie 2010).Articles that employed MMR constituted a mere 2 per cent as compared to 89 per cent for articles that used quantitative approaches.Most of the scholars who used MMR neither described their studies in the abstract and methods sections as MMR nor adequately: • gave the rationale for using it; • specified and justified the type of MMR design they selected; • integrated the results as most of the studies were more biased towards the quantitative strand than the qualitative one; • showed the relative importance of qualitative and quantitative data; and • mixed the qualitative and quantitative findings.Economic and management sciences deal with relativistic, complex and dynamic social constructs that influence a variety of contexts.
MMR provides the possibility to best understand and make assumptions about the complex problems that economic and management science scholars engage with.
The use of mono-methods may limit the ability of scholars to gain a full insight of the research situation under investigation.Moving away from mono-methods, and by building on the strengths and accepting the limitations and weaknesses of mixed methods research, researchers in economic and management sciences as reflected in contributions to SAJEMS may be able to address all the facets of the complex problems they investigate.The use of multiple methods is a kind of insurance policy against drawing limited conclusions from one method alone.Furthermore, the utilisation of MMR may benefit economic and management science scholars who contribute to SAJEMS by giving them space to engage in interdisciplinary research, which is increasingly becomi8ng prevalent in academic landscapes.
Besides its ability to provide a comprehensive picture and a rich insight into a phenomenon, MMR provides a potential for theory building.The discipline of economic and management sciences is rapidly changing because of various factors.Existing theories may not sufficiently provide a framework to understand, explain and predict the new developments in a unique context.Quantitative approaches, which dominated the contributions to SAJEMS, have limitations when it comes to theory development as they are mainly concerned with theory testing and refinement.
MMR may assist scholars to address exploratory context-specific questions they face through qualitative methods and generate a theory that may be tested and confirmed using quantitative methods.Thus, qualitative and quantitative methods may be combined to build, confirm and improve theory.In that light, MMR provides a window of opportunity to develop and generate context specific theories that may contribute to the growth and maturity of economic and management sciences as reflected in articles in SAJEMS.
Our contribution sought to open up more debate than before on methodological matters in context and to encourage what Brannen (2008:55) termed 'thinking outside the box'.However, the methodological issues we have raised in this article are not an exhaustive list of possibilities but are meant to provide a basis for further discussion and to motivate scholars who contribute to SAJEMS to utilise MMR because of the advantages that it offers as described in this article.We do not expect every research study in SAJEMS to use MMR.Lastly, one should be careful to place too much weight on the findings of this single case study, nevertheless it is a good indicator that MMR is not prevalent in the economic and management sciences in South Africa.

Table 2
Summary of MMR attributes for the four relevant articles