Validity and Reliability in Research (Research Paper Sample)

ASSIGNMENT 6: PRESENTATION ON VALIDITY AND RELIABILITY
Table of contents
TOC \o "1-4" \h \z \u Table of contents PAGEREF _Toc362274810 \h 1
Tables PAGEREF _Toc362274811 \h 3
Figures PAGEREF _Toc362274812 \h 4
1.0.Introduction PAGEREF _Toc362274813 \h 5
2.0Validity PAGEREF _Toc362274814 \h 5
2.1Types of validity PAGEREF _Toc362274815 \h 6
2.1.1Internal validity PAGEREF _Toc362274816 \h 6
2.1.1.1Threats to internal validity PAGEREF _Toc362274817 \h 6
2.1.2External validity PAGEREF _Toc362274818 \h 7
2.1.2.1Threats to external validity PAGEREF _Toc362274819 \h 8
2.1.3Construct validity PAGEREF _Toc362274820 \h 8
2.1.4Statistical validity PAGEREF _Toc362274821 \h 8
2.2Approaches to the validity of tests and measures PAGEREF _Toc362274822 \h 9
2.3How to improve validity PAGEREF _Toc362274823 \h 9
3.0Reliability PAGEREF _Toc362274824 \h 9
3.1Types of reliability PAGEREF _Toc362274825 \h 11
3.1.1Internal reliability PAGEREF _Toc362274826 \h 11
3.1.2External reliability PAGEREF _Toc362274827 \h 11
3.2Methods of assessing reliability PAGEREF _Toc362274828 \h 11
3.3Factors affecting reliability PAGEREF _Toc362274829 \h 12
4.0Differences between reliability and validity PAGEREF _Toc362274830 \h 13
5.0Conclusion PAGEREF _Toc362274831 \h 15
REFERENCES PAGEREF _Toc362274832 \h 16
Tables
Figure 1: An illustration of the differences between validity and reliability15
Figures
Figure 1: An illustration of the differences between validity and reliability …………..
1 Introduction
Key indicators of the quality of a measuring instrument are the reliability and validity of the measures CITATION Car08 \l 1033 (Carole L. Kimberlin & Winterstein, 2008). Hence, the principles of validity and reliability are fundamental cornerstones of the scientific method. Trustworthiness of findings, conclusions and recommendations in any study largely depends on the validity, and reliability of the methods and instruments used in data collection. A credible researcher minimizes possible errors and bias by maximizing the reliability and validity of data. Reliability and validity are common in both quantitative and qualitative research
2.0Validity
According to Joppe (2000), validity denotes the extent to which a measurement does what it is supposed to do. By asserting validity, the researcher is asserting that the data actually measures or reflects the specific phenomenon claimed. It means that the researcher’s conclusion is true or correct. It also means that the results of a research instrument relates to the initial criteria. When the results and criterion are unrelated then the instrument is invalid, and should not be used in further research. Researchers generally determine validity by asking a series of questions
* How truthful are the research results?
* Is the research truly measuring that which it was intended to measure?
* Does the research instrument allow hitting "the bull’s eye" of the research object?
2.1Types of validity
There are several categories of validity - internal validity; external validity; construct validity and statistical validity.
2.1.1Internal validity
According to CITATION McB10 \l 1033 (McBurney & White, 2010), an experiment is internally valid when the results can be attributed to the manipulation/ independent variable. Research is done to determine cause-and-effect relationships. Questions asked to check internal validity include;
* Can we conclude that changes in the independent variable caused the observed changes in the dependent variable?
* Is the evidence for such a conclusion good or poor?
* If a study shows a high degree of internal validity then we can conclude we have strong evidence of causality?
* If a study has low internal validity, then we must conclude we have little or no evidence of causality?
2.1.1.1Threats to internal validity
* Extraneous variables are variables that may compete with the independent variable in explaining the outcome of a study. Confounding variable is one of the biggest threats in research. It systematically varies or influences both the independent variable and the dependent variable
* History: Experiments that take place over a period of time (difficult to control events (e.g. a coup) – thus can influence the dependent variable
* Maturation: Changes that occur with time. E.g. cognitive, emotional, social, physical, moral) or personal states (e.g., hunger).
* Selection: Anything other than random selection can induce bias.
* Instrumentation: Did any change occur during the study in the way the dependent variable was measured? (Is it a threat to the one group design; not to the two group design? Why?)
* Attrition: - Pertains to some participants dropping out for any reason.
* Statistical regression: Refers to the tendency of extreme scores to move (or regress) toward the mean score on subsequent retesting.
* Compensatory rivalry. When subjects in some treatments receive goods or services believed to be desirable and this becomes known to subjects in other groups, social competition may motivate the latter to attempt to reverse or reduce the anticipated effects of the desirable treatment levels.
2.1.2External validity
Refers to the extent to which the results of a research study are able to be generalized confidently to a group larger than the group that participated in the study CITATION Bra68 \l 1033 (Bracht & Glass, 1968). It is all about generalizability.
External validity is usually split into two distinct types, population validity and ecological validity, and they are both essential elements in judging the strength of an experimental design. For instance, the idea that experimental results obtained in a laboratory setting might be different from those obtained in a natural setting reflects a question about ecological validity(the extent to which an experimental situation mimics a real world situation).
2.1.2.1Threats to external validity
* Interaction effect of testing: Pre-testing interacts with the experimental treatment and causes some affect such that the results will not generalize to an untested population.
* Interaction effects of selection biases and the experimental treatment: An effect of some selection factor of intact groups interacting with the experimental treatment that would not be the case if the groups were randomly selected.
* Reactive effects of experimental arrangements: An effect that is due simply to the fact that subjects know that they are participating in an experiment and experiencing the novelty of it — the Hawthorne effect.
* Multiple-treatment interference: When the same subjects receive two or more treatments as in a repeated measures design, there may be a carryover effect between treatments such the results cannot be generalized to single treatments.
2.1.3Construct validity
This is the extent to which the results support the theory behind the research CITATION McB10 \l 1033 (McBurney & White, 2010). Would another theory predict the same experimental results? Construct validity is similar to internal validity and they share similar threats.
2.1.4Statistical validity
This is the extent to which data are shown to be the result of cause-effect relationships rather than accident. The question on statistical validity is, "was the observed relationship between the independent and dependent variables a true cause-effect relationship, or was the result accidental, and thus caused by pure chance?" CITATION McB10 \p 177 \l 1033 (McBurney & White, 2010, p. 177). Statistical validity is related to internal validity. When statistical tests are used improperly, a lack of validity is reflected.
Some threats to statistical validity include; liberal biases (being overly optimistic regarding the existence of a relationship or exaggerating its strength); conservative biases (being overly pessimistic regarding the absence of a relationship or underestimating its strength) and low power (the probability that the evaluation will result in a Type II error)
2.2Approaches to the validity of tests and measures
* Criterion validity. It assesses whether a test reflects a certain set of abilities
* Content validity. It is the estimate of how much a measure represents every single element of a construct
* Construct validity: This defines how well a test or experiment measures up to its claims.
2.3How to improve validity
* Goals and objectives are clearly defined and operationalized
* Match the assessment measure to the goals and objectives.
* If possible, compare the measure with other measures, or data that may be available.
3.0Reliability
Joppe (2000) defines reliability as the extent to which results from a measurement are: consistent over time; an accurate representation of the total population under study and that can be reproduced under a similar methodology.
In quantitative research, the extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability. If the "results of a study can be reproduced under a similar methodology”, then the research instrument is considered to be reliable CITATION Gol03 \l 1033 (Golafshani, 2003)
Accordingly, the key elements of reliability include:
* Replicability /repeatability of results
* degree to which a measurement, given repeatedly, remains the same
* Stability of a measurement over time
* Similarity of measurements within a given time period
* Internal consistency with which questionnaire[test] items are answered or individual’s scores remain relatively the same through test-retest method at two different times CITATION Gol03 \l 1033 (Golafshani, 2...