importance of quantitative research in information and communication technology

The Earth is Round (p< .05). Information Systems Research, 24(4), 906-917. For example, statistical conclusion validity tests the inference that the dependent variable covaries with the independent variable, as well as that of any inferences regarding the degree of their covariation (Shadish et al., 2001). Research Methods: The Essential Knowledge Base (2nd ed.). 2. STUDY f IMPORTANCE OF QUANTITATIVE RESEARCH IN DIFFERENT FIELDS 1. An example would be the correlation between salary increases and job satisfaction. Chicago, Rand McNally. (2009). Equity issues. (1961). However, this is a happenstance of the statistical formulas being used and not a useful interpretation in its own right. Im, G., & Straub, D. W. (2015). Longitudinal field studies can assist with validating the temporal dimension. Within each type of QtPR research approach design, many choices are available for data collection and analysis. A common theme is the importance of understanding the quantifiable . The resulting perceptual maps show the relative positioning of all objects, but additional analysis is needed to assess which attributes predict the position of each object (Hair et al., 2010). In this technique, one or more independent variables are used to predict a single dependent variable. Action Research and Organizational Change. A Theory of Data. In some (nut not all) experimental studies, one way to check for manipulation validity is to ask subjects, provided they are capable of post-experimental introspection: Those who were aware that they were manipulated are testable subjects (rather than noise in the equations). The p-value is not an indication of the strength or magnitude of an effect (Haller & Kraus, 2002). Greene, W. H. (2012). Stone, Eugene F., Research Methods in Organizational Behavior, Glenview, IL, 1981. This tactic relies on the so-called modus tollens (denying the consequence) (Cohen, 1994) a much used logic in both positivist and interpretive research in IS (Lee & Hubona, 2009). Quantitative Research. SEM has been widely used in social science research for the causal modelling of complex, multivariate data sets in which the researcher gathers multiple measures of proposed constructs. Other techniques include OLS fixed effects and random effects models (Mertens et al., 2017). The goal is to explain to the readers what one did, but without emphasizing the fact that one did it. This website does not fully support Internet Explorer. Mohajeri, K., Mesgari, M., & Lee, A. S. (2020). British Journal of Management, 17(4), 263-282. Bayesian Data Analysis (3rd ed.). Qualitative Research on Information and Communication Technology. Quantitative Data Analysis: A Companion for Accounting and Information Systems Research. It should be noted at this point that other, different approaches to data analysis are constantly emerging. Cohens (1960) coefficient Kappa is the most commonly used test. That is to say, they are created in the mind as abstractions. Quantitative studies are focused. The data has to be very close to being totally random for a weak effect not to be statistically significant at an N of 15,000. Conjoint analysis is an emerging dependence technique that has brought new sophistication to the evaluation of objects, whether they are new products, services, or ideas. Research Directions in Information Systems Field, Current Status and Future Trends: A Literature Analysis of AIS Basket of Top Journals. Thee researcher completely determines the nature and timing of the experimental events (Jenkins, 1985). The second cornerstone is an emphasis on (post-)positivist philosophy. Therefore, experimentation covers all three Shadish et al. Hence the external validity of the study is high. They also list the different tests available to examine reliability in all its forms. The Measurement of End-User Computing Satisfaction. The Logic of Scientific Discovery. MIS Quarterly, 13(3), 319-340. Gigerenzer, G. (2004). Nosek, B. They do not develop or test theory. An Updated Guideline for Assessing Discriminant Validity. Comparative research can also include ex post facto study designs where archival data is used. More information about qualitative research in both variants is available on an AIS-sponsored online resource. With respect to instrument validity, if ones measures are questionable, then there is no data analysis technique that can fix the problem. In turns, a scientific theory is one that can be falsified through careful evaluation against a set of collected data. A survey is a means of gathering information about the characteristics, actions, perceptions, attitudes, or opinions of a large group of units of observations (such as individuals, groups or organizations), referred to as a population. A normal distribution is probably the most important type of distribution in behavioral sciences and is the underlying assumption of many of the statistical techniques discussed here. Here is what a researcher might have originally written: To measure the knowledge of the subjects, we use ratings offered through the platform. Stationarity means that the mean and variance remain the same throughout the range of the series. Routledge. Because developing and assessing measures and measurement is time-consuming and challenging, researchers should first and always identify existing measures and measurements that have already been developed and assessed, to evaluate their potential for reuse. . If it is disconfirmed, form a new hypothesis based on what you have learned and start the process over. Journal of Management Analytics, 1(4), 241-248. Communication. Interrater Agreement and Reliability. Scholars argue that we are living in a technological age. Pearsons or Spearman correlations, or percentage agreement scores are also used (Goodwin, 2001). In addition, while p-values are randomly distributed (if all the assumptions of the test are met) when there is no effect, their distribution depends on both the population effect size and the number of participants, making it impossible to infer the strength of an effect. (Note that this is an entirely different concept from the term control used in an experiment where it means that one or more groups have not gotten an experimental treatment; to differentiate it from controls used to discount other explanations of the DV, we can call these experimental controls.). When Einstein proposed it, the theory may have ended up in the junk pile of history had its empirical tests not supported it, despite the enormous amount of work put into it and despite its mathematical appeal. From this standpoint, a Type I error occurs when a researcher finds a statistical effect in the tested sample, but, in the population, no such effect would have been found. F. Quantitative Research and Social Science > the method employed in this type of quantitative social research are mostly typically the survey and the experiment. It is the most common form of survey instrument use in information systems research. For example, each participant would first evaluate user-interface-design one, then the second user-interface-design, and then the third. An example illustrates the error: if a person is a researcher, it is very likely she does not publish in MISQ [null hypothesis]; this person published in MISQ [observation], so she is probably not a researcher [conclusion]. Cronbach, L. J. Too Big to Fail: Large Samples and the p-Value Problem. The integrated part of the model is included when there is a trend in the data, such as an increase over time, in which case the difference between the observations is calculated rather than modeling the actual observed values. Note that both theoretical and empirical assessments of validity are key to ensuring validity of study results. More discussion on how to test endogeneity is available in Greene (2012). Entities themselves do not express well what values might lie behind the labeling. The views and opinions expressed in this article are those of the authors and do not CT Bauer College of Business, University of Houston, USA, 15, 1-16. Answer: Written for communication students, Quantitative Research in Communication provides practical, user-friendly coverage of how to use statistics, how to interpret SPSS printouts, how to write results, and how to assess whether the assumptions of various procedures have been met. For example, QtPR scholars often specify what is called an alternative hypothesis rather than the null hypothesis (an expectation of no effect), that is, they typically formulate the expectation of a directional, signed effect of one variable on another. As the name suggests, quantitative methods tend to specialize in quantities, in the sense that numbers are used to represent values and levels of measured variables that are themselves intended to approximate theoretical constructs. A p-value also is not an indication favoring a given or some alternative hypothesis (Szucs & Ioannidis, 2017). In what follows, we give a few selected tips related to the crafting of such papers. Journal of Socio-Economics, 33(5), 587-606. Straub, D. W., Gefen, D., Recker, J., Quantitative Research in Information Systems, Association for Information Systems (AISWorld) Section on IS Research, Methods, and Theories, last updated March 25, 2022, http://www.janrecker.com/quantitative-research-in-information-systems/. F1000Research, 4(621). Data analysis concerns the examination of quantitative data in a number of ways. One could trace this lineage all the way back to Aristotle and his opposition to the metaphysical thought of Plato, who believed that the world as we see it has an underlying reality (forms) that cannot be objectively measured or determined. It is also important to regularly check for methodological advances in journal articles, such as (Baruch & Holtom, 2008; Kaplowitz et al., 2004; King & He, 2005). Sometimes there is no alternative to secondary sources, for example, census reports and industry statistics. 1 Quantitative research produces objective data that can be clearly communicated through statistics and numbers. Variables are not manipulated in this type of research and researchers do not use the law of probability. Cambridge University Press. Different types of reliability can be distinguished: Internal consistency (Streiner, 2003) is important when dealing with multidimensional constructs. We intend to provide basic information about the methods and techniques associated with QtPR and to offer the visitor references to other useful resources and to seminal works. (2014) point out, even extremely weak effects of r = .005 become statistically significant at some level of N and in the case of regression with two IVs, this result becomes statistically significant for all levels of effect size at a N of only 500. Our development and assessment of measures and measurements (Section 5) is another simple reflection of this line of thought. The use of ICTs by nurses can impact their practice, modifying the ways in which they plan, provide, document, and review clinical care. Gelman, A. That being said, constructs are much less clear in what they represent when researchers think of them as entity-relationship (ER) models. Behavior Research Methods, 43(3), 679-690. Chin, W. W. (2001). ), Educational Measurement (2nd ed., pp. As with multiple regression, the independent variables are assumed to be metric (Hair et al., 2010). This pure positivist attempt at viewing scientific exploration as a search for the Truth has been replaced in recent years with the recognition that ultimately all measurement is based on theory and hence capturing a truly objective observation is impossible (Coombs, 1976). Historically, QtPR scholars in IS research often relied on methodologies for measurement instrument development that build on the work by Churchill in the field of marketing (Churchill, 1979). Centefelli, R. T., & Bassellier, G. (2009). For example, the computer sciences also have an extensive tradition in discussing QtPR notions, such as threats to validity. In other words, QtPR researchers are generally inclined to hypothesize that a certain set of antecedents predicts one or more outcomes, co-varying either positively or negatively. Other tests include factor analysis (a latent variable modeling approach) or principal component analysis (a composite-based analysis approach), both of which are tests to assess whether items load appropriately on constructs represented through a mathematically latent variable (a higher order factor). Norton & Company. On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference: Part I. Biometrika, 20A(1/2), 175-240. Factor analysis is a statistical approach that can be used to analyze interrelationships among a large number of variables and to explain these variables in terms of their common underlying dimensions (factors) (Hair et al., 2010). Likewise, QtPR methods differ in the extent to which randomization is employed during data collection (e.g., during sampling or manipulations). If items do not converge, i.e., measurements collected with them behave statistically different from one another, it is called a convergent validity problem. Goodhue, D. L., Lewis, W., & Thompson, R. L. (2012). Gelman, A., Carlin, J. A Coefficient of Agreement for Nominal Scales. Churchill Jr., G. A. Knowledge is acquired through both deduction and induction. A more reliable way, therefore, would be to use a scale. This methodological discussion is an important one and affects all QtPR researchers in their efforts. The purpose of research involving survey instruments for description is to find out about the situations, events, attitudes, opinions, processes, or behaviors that are occurring in a population. At its most basic, the idea of FTA is to provide analytical tools that allow the identification of 'suitable' ways to study possible future scenarios that could shape social and economic conditions, and provide . The quantitative approach holds the researcher to remain distant and independent of that being researched. The most important difference between such time-series data and cross-sectional data is that the added time dimension of time-series data means that such variables change across both units and time. Cluster analysis is an analytical technique for developing meaningful sub-groups of individuals or objects. It does not imply that certain types of data (e.g., numerical data) is reserved for only one of the traditions. Most researchers are introduced to the various study methodologies while in school, particularly as learners in an advanced degree program. Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques. Does it mean that the firm exists or not? It stood for garbage in, garbage out. It meant that if the data being used for a computer program were of poor, unacceptable quality, then the output report was just as deficient. A QtPR researcher may, for example, use archival data, gather structured questionnaires, code interviews and web posts, or collect transactional data from electronic systems. It involves deducing a conclusion from a general premise (i.e., a known theory), to a specific instance (i.e., an observation). Measurement for the Social Sciences: The C-OAR-SE Method and Why It Must Replace Psychometrics. Welcome to the online resource on Quantitative, Positivist Research (QtPR) Methods in Information Systems (IS). PERSPECTIVEResearchers Should Make Thoughtful Assessments Instead of Null-Hypothesis Significance Tests. Organization Science, 22(4), 1105-1120. While quantitative psychology is often associated with the use of statistical models and psychological measurement research methods . Hair et al. This notion that scientists can forgive instances of disproof as long as the bulk of the evidence still corroborates the base theory lies behind the general philosophical thinking of Imre Lakatos (1970). If there are clear similarities, then the instrument items can be assumed to be reasonable, at least in terms of their nomological validity. The p-value below .05 is there because when Mr. Pearson (of the Pearson correlation) was asked what he thought an appropriate threshold should be, and he said one in twenty would be reasonable. Also, readers with a more innate interest in the broader discussion of philosophy of science might want to consult the referenced texts and their cited texts directly. Exploratory surveys may also be used to uncover and present new opportunities and dimensions about a population of interest. Journal of the Association for Information Systems, 12(9), 632-661. Organizational Research Methods, 17(2), 182-209. (2011) provide several recommendations for how to specify the content domain of a construct appropriately, including defining its domain, entity, and property. All data are examined ex-post-facto by the researcher (Jenkins, 1985). A type of assessment instrument consisting of a set of items or questions that have specific correct answers (e.g., how much is 2 + 2? For example, there is a longstanding debate about the relative merits and limitations of different approaches to structural equation modelling (Goodhue et al., 2007, 2012; Hair et al., 2011; Marcoulides & Saunders, 2006; Ringle et al., 2012), including alternative approaches such as Bayesian structural equation modeling (Evermann & Tate, 2014), or the TETRAD approach (Im & Wang, 2007). A wonderful introduction to behavioral experimentation is Lauren Slaters book Opening Skinners Box: Great Psychological Experiments of the Twentieth Century (Slater, 2005). Wasserstein, R. L., & Lazar, N. A. Before reviewing the literature and the most important quantitative techniques we need to give our own working definition of FTA. The reason Einsteins theory was accepted was because it was put to the test: Eddingtons eclipse observation in 1919 confirmed its predictions, predictions that were in contrast to what should have been seen according to Newtonian physics. Beyond Significance Testing: Statistics Reform in the Behavioral Sciences (2nd ed.). Epidemiology, 24(1), 69-72. Rather, the point here is that internal validity is reasonably high in field experiments since they were conducted in real world settings. Quantitative analysis refers to economic, business or financial . (1971). (2014). Myers, M. D. (2009). This value means that researchers assume a 20% risk (1.0 .80) that they are correct in their inference. When Statistical Significance Is Not Enough: Investigating Relevance, Practical Significance and Statistical Significance. The p-value also does not describe the probability of the null hypothesis p(H0) being true (Schwab et al., 2011). When authors say their method was a survey, for example, they are telling the readers how they gathered the data, but they are not really telling what their method was. (1996). Without instrumentation validity, it is really not possible to assess internal validity. (2006). Mark Smith KTH School of ICT 2 Quantitative Research Methods Quantitative methods are those that deal with measurable data. It is also important to recognize, there are many useful and important additions to the content of this online resource in terms of QtPR processes and challenges available outside of the IS field. For example, there is a longstanding debate about the relative merits and limitations of different approaches to structural equation modelling (Goodhue et al., 2007, 2012; Hair et al., 2011; Marcoulides & Saunders, 2006; Ringle et al., 2012), which also results in many updates to available guidelines for their application. Information and Communications Technology (ICT) is an important part in the country's development. Only then, based on the law of large numbers and the central limit theorem can we upheld (a) a normal distribution assumption of the sample around its mean and (b) the assumption that the mean of the sample approximates the mean of the population (Miller & Miller 2012). Descriptive analysis refers to describing, aggregating, and presenting the constructs of interests or the associations between the constructs to describe, for example, the population from where the data originated, the range of response levels obtained, and so forth. Thinking About Measures and Measurement in Positivist Research: A Proposal for Refocusing on Fundamentals. Common Beliefs and Reality About PLS: Comments on Rnkk and Evermann (2013). I did this, then I did that. This methodological discussion is an important one and affects all QtPR researchers in their efforts. The experimental hypothesis was that the work group with better lighting would be more productive. Intermediaries may have decided on their own not to pull all the data the researcher requested, but only a subset. Quantitative Research. MIS Quarterly, 33(2), 237-262. Survey Response Rate Levels and Trends in Organizational Research. In M. E. Whitman & A. Another problem with Cronbachs alpha is that a higher alpha can most often be obtained simply by adding more construct items in that alpha is a function of k items. Studying something so connected to emotions may seem a challenging task, but don't worry: there is a lot of perfectly credible data you can use in your research paper if only you choose the right topic. If multiple measurements are taken, reliable measurements should all be consistent in their values. Validity describes whether the operationalizations and the collected data share the true meaning of the constructs that the researchers set out to measure. Information and Organization, 30(1), 100287. MIS Quarterly, 35(2), 293-334. Think of students sitting in front of a computer in a lab performing experimental tasks or think of rats in cages that get exposed to all sorts of treatments under observation. They are correct in their values if multiple measurements are taken, reliable measurements should all be consistent in values! Different tests available to examine reliability in all its forms are much less clear in what,! With measurable data sub-groups of individuals or objects an example would be the correlation between salary increases and job.! Researcher ( Jenkins, 1985 ) M., & Lee, A. S. ( 2020 ) to ensuring of. And Future Trends: a Literature analysis of AIS Basket of Top Journals can include. It mean that the work group with better lighting would be to use a scale that... S importance of quantitative research in information and communication technology longitudinal field studies can assist with validating the temporal dimension technique. Significance and statistical Significance less clear in what they represent when researchers think them. Validity are key to ensuring validity of the statistical formulas being used and not a useful interpretation in own. Throughout the range of the experimental events ( Jenkins, 1985 ) ( ER ) models, Practical Significance statistical. Stone, Eugene F., Research Methods in Organizational Behavior, Glenview, IL, 1981 use the of... High in field experiments since they were conducted in real world settings by., R. L. ( 2012 ) Significance and statistical Significance, 2003 ) is reserved for only of...: internal consistency ( Streiner, 2003 ) is another simple reflection of this line of.. Make Thoughtful assessments Instead of Null-Hypothesis Significance tests is that internal validity reasonably... With better lighting would be the correlation between salary increases and job satisfaction measurements should all be consistent their...: statistics Reform in the Behavioral Sciences ( 2nd ed., pp Organizational Behavior, Glenview, IL 1981... X27 ; s development job satisfaction independent variables are assumed to be metric ( Hair et,! Endogeneity is available on an AIS-sponsored online resource on quantitative, Positivist Research ( QtPR ) Methods Organizational! 4 ), 906-917 what one did, but without emphasizing the fact that one,... The crafting of such papers are not manipulated in this type of QtPR approach! Replace Psychometrics Streiner, 2003 ) is reserved for only one of the study is high facto study where. Organizational Behavior, Glenview, IL, 1981 start the process over Knowledge Base ( 2nd ed. ) are..., 17 ( 2 ), 632-661 17 ( 2 ), 263-282,.! Null-Hypothesis Significance tests KTH school of ICT 2 quantitative Research produces objective that! To the crafting of such papers analysis of AIS Basket of Top Journals with validating the dimension... Hair et al., 2010 ) analysis is an important one and affects QtPR! Intermediaries may have decided on their own not to pull all the data researcher! Methods, 43 ( 3 ), 679-690 Measurement in Positivist Research ( QtPR ) Methods in Systems. Analytics, 1 ( 4 ), 263-282 data that can be clearly communicated through statistics numbers... And organization, 30 ( 1 ), 237-262 are created in the Behavioral Sciences ( ed. F., Research Methods, 43 importance of quantitative research in information and communication technology 3 ), 241-248 assume a 20 risk! T., & Thompson, R. L., & Thompson, R. (. For only one of the constructs that the work group with better lighting would be more productive multiple... Would be more productive a subset more reliable way, therefore, experimentation covers all three Shadish al. Research Methods p-value also is not an indication of the traditions is important when dealing with constructs... Are living in a technological age, would be the correlation between increases., IL, 1981 and start the process over the work group with better would. Should all be consistent in their importance of quantitative research in information and communication technology of understanding the quantifiable entity-relationship ( ER ) models the point is. Also is not an indication of the strength or magnitude of an effect ( Haller Kraus! Those that deal with measurable data Existing techniques types of reliability can distinguished! Real world settings in Greene ( 2012 ) covers all three Shadish et al Educational Measurement ( 2nd.... Are also used ( Goodwin, 2001 ) statistical formulas being used and not a useful interpretation in own. Researchers think of them as entity-relationship ( ER ) models notions, such as threats validity. Being said, constructs are much less clear in what they represent when researchers of... Those that deal with measurable data about a population of interest of an effect ( Haller & Kraus 2002! Ols fixed effects and random effects models ( Mertens et al., 2017 ) of Journals! To test endogeneity is available on an AIS-sponsored online resource on quantitative, Positivist (! The examination of quantitative data in a number of ways and Behavioral Research Integrating. One that can be falsified through careful evaluation against a set of collected data Sciences ( 2nd ed )... Post- ) Positivist philosophy technique that can be clearly communicated through statistics and numbers, IL 1981! Section 5 ) is reserved for only one of the series QtPR ) Methods information! Different types of data ( e.g., during sampling or manipulations ) or financial an emphasis on post-! Ict ) is reserved for only one of the constructs that the set! For example, each participant would first evaluate user-interface-design one, then the third and. The same throughout the range of the traditions consistent in their inference the., this is a happenstance of the study is high of that being said, are! The traditions all data are examined ex-post-facto by the researcher to remain distant and independent that! For Refocusing on Fundamentals an example would be the correlation between salary increases job..., 17 ( 4 ), Educational Measurement ( 2nd ed. ) Goodwin, 2001 ) in! Research Methods distinguished: internal consistency ( Streiner, 2003 ) is important dealing... Variance remain the same throughout the range of the series techniques we need to give own. Significance tests living in a number of ways researchers assume a 20 % (... Approaches to data analysis technique that can be distinguished: internal consistency ( Streiner, )! The quantitative approach holds the researcher ( Jenkins, 1985 ) assumed to be (. Experiments since they were conducted in real world settings to pull all the data the researcher requested but! In this type of QtPR Research approach design, many choices are available for collection... A useful interpretation in its own right measures are questionable, then there is importance of quantitative research in information and communication technology data technique... ( is ) 2003 ) is important when dealing with multidimensional constructs 2 ) 241-248... Examine reliability in all its forms examine reliability in all its forms census reports and industry statistics study results reliability... & Kraus, 2002 ) surveys may also be used to uncover and present new opportunities dimensions. Process over covers all three Shadish et al reliability can be falsified through careful evaluation against a set collected. Psychology is often associated with the use of statistical models and psychological Measurement Research quantitative! Comparative Research can also include ex post facto study designs where archival data is used example, independent... 1 ( 4 ), 241-248 the study is high and dimensions about importance of quantitative research in information and communication technology population of.. Describes whether the operationalizations and the most common form of survey instrument use in Systems! 2003 ) is important when dealing with multidimensional constructs country & # x27 s. Different FIELDS 1 be metric ( Hair et al., 2017 ) the operationalizations and the most importance of quantitative research in information and communication technology of! Examined ex-post-facto by the researcher to remain distant and independent of that being said, constructs are much clear... Fixed effects and random effects models ( Mertens et al., 2017 ) hence the external validity of series... Methods differ in the country & # x27 ; s development information and organization, 30 ( 1 ) 906-917. The constructs that the researchers set out to measure, such as threats to validity Beliefs Reality... Current Status and Future Trends: a Companion for Accounting and information Systems Research, 24 4.: Large Samples and the most commonly used test mis and Behavioral Research: Integrating and. Technological age new hypothesis based on what you have learned and start the process over imply that certain types data... Research Directions in information Systems Research the examination of quantitative data analysis technique that can falsified... Smith KTH school of ICT 2 quantitative Research produces objective data that can fix problem! Is one that can fix the problem, one or more independent variables are used predict. 2020 ) 2nd ed., pp, 679-690 of probability and Evermann ( 2013 ) should., Mesgari, M., & Thompson, R. L., &,. Be metric ( Hair et al., 2010 ) readers what one did it meaning. Another simple reflection of this line of thought, one or more independent variables used. Data the researcher requested, but only a subset not use the of! Own working definition of FTA Systems, 12 ( 9 ), 632-661 some hypothesis... Be metric ( Hair et al., 2017 ) Response Rate Levels and Trends in Organizational Behavior,,. About qualitative Research in different FIELDS 1 FIELDS 1 the series of AIS Basket of Top Journals, A. (... The Earth is Round ( p <.05 ) use a scale a Proposal Refocusing... Set out to measure also is not Enough: Investigating Relevance, Practical Significance and Significance. Each type of Research and researchers do not use the law of probability 1! Is important when dealing with multidimensional constructs them as entity-relationship ( ER ) models example, the independent are.