{"id":76278,"date":"2019-05-15T11:07:27","date_gmt":"2019-05-15T09:07:27","guid":{"rendered":"https:\/\/www.scribbr.nl\/?p=76278"},"modified":"2023-06-22T13:31:03","modified_gmt":"2023-06-22T11:31:03","slug":"internal-vs-external-validity","status":"publish","type":"post","link":"https:\/\/www.scribbr.com\/methodology\/internal-vs-external-validity\/","title":{"rendered":"Internal vs. External Validity | Understanding Differences & Threats"},"content":{"rendered":"

Internal and external validity<\/strong> are two ways of testing cause-and-effect relationships.<\/p>\n

Internal validity<\/strong><\/a> refers to the degree of confidence that the causal relationship being tested is trustworthy and not influenced by other factors or variables<\/a>.External validity<\/strong><\/a> refers to the extent to which results from a study can be applied (generalized<\/a>) to other situations, groups, or events.<\/figure>\n

The validity of a study is largely determined by the experimental design<\/a>. To ensure the validity of the tools or tests you use, you also have to consider measurement validity<\/a>.<\/p>\n

<\/p>\n

Trade-off between internal and external validity<\/h2>\n

Better internal validity often comes at the expense of external validity (and vice versa<\/a>). The type of study<\/a> you choose reflects the priorities of your research.<\/p>\n

Trade-off example<\/strong>
\nA causal relationship can be tested in an artificial lab setting or in the real world. A lab setting ensures higher internal validity because external influences can be minimized. However, the external validity diminishes because a lab environment is different than the outside world (that does have external influencing factors).<\/figure>\n

A solution to this trade-off is to conduct the research first in a controlled<\/a> (artificial) environment to establish the existence of a causal<\/a> relationship, followed by a field experiment to analyze if the results hold in the real world.<\/p>\n

Threats to internal validity<\/h2>\n

There are eight factors that can threaten the internal validity<\/a> of your research. They are explained below using the following example:<\/p>\n

Research example<\/strong>
\nThe management of a popular jeans company wants to know if flexible working hours will improve job satisfaction among employees. They set up an experiment with two groups: 1) control group<\/a> of employees with fixed working hours 2) experiment group with employees with flexible working hours.\u00a0The experiment will run for six months. All employees fill in a survey<\/a> measuring their job satisfaction before the experiment (pre-test) and after the experiment (post-test).<\/figure>\n\n\n\n\n\n\n\n\n\n\n\n\n
Threats to internal validity<\/caption>\n
Threat<\/th>\nExplanation<\/th>\nExample<\/th>\n<\/tr>\n<\/thead>\n
History<\/th>\nUnanticipated events change the conditions of the study and influence the outcome.<\/td>\nA new (better) manager starts during the study, which improves job satisfaction.<\/td>\n<\/tr>\n
Maturation<\/th>\nThe passage of time influences the dependent variable (job satisfaction).<\/td>\nDuring the six-month experiment, employees become more experienced and better at their jobs. Therefore, job satisfaction may improve.<\/td>\n<\/tr>\n
Testing<\/th>\nThe pre-test (used to establish a baseline) affects the results of the post-test.<\/td>\nEmployees feel the need to be consistent in their answers in the pre-test and post-test.<\/td>\n<\/tr>\n
Participant selection<\/th>\nParticipants in the control and experimental group differ substantially and can thus not be compared.<\/td>\nInstead of a randomly assigning employees to one of two groups, employees can volunteer to participate in an experiment to improve job satisfaction. The experimental group now consists of more engaged (more satisfied) employees to begin with, but can lead to self-selection bias<\/a>.<\/td>\n<\/tr>\n
Attrition<\/a><\/th>\nOver the course of a (longer) study, participants may drop out.\u00a0If the drop out is caused by the experimental treatment (as opposed to coincidence) it can threaten internal validity and cause attrition bias<\/a>.<\/td>\nReally dissatisfied employees quit their job during the study. The average job satisfaction will now improve, not because the \u201ctreatment\u201d worked, but because the dissatisfied employees are not included in the post-test.<\/td>\n<\/tr>\n
Regression to the mean<\/a><\/th>\nExtreme scores tend to be closer to the average on a second measurement.<\/td>\nEmployees who score extremely low in the first job satisfaction survey<\/a> probably show greater gain in job satisfaction than employees who scored average.<\/td>\n<\/tr>\n
Instrumentation<\/th>\nThere is a change in how the dependent variable<\/a> is measured during the study.<\/td>\nThe questionnaire<\/a> in the post test contains extra questions compared to the one used for the pre-test. This leads to information bias<\/a>.<\/td>\n<\/tr>\n
Social interaction<\/th>\nInteraction between participants from different groups influences the outcome.<\/td>\nThe group of employees with fixed working hours are resentful of the group with flexible working hours, and their job satisfaction decreases as a result.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Threats to external validity<\/h2>\n

There are three main factors that might threaten the external validity of our study example.<\/p>\n\n\n\n\n\n\n\n
Threats to external validity<\/caption>\n
Threat<\/th>\nExplanation<\/th>\nExample<\/th>\n<\/tr>\n<\/thead>\n
Testing<\/th>\nParticipation in the pre-test influences the reaction to the treatment.<\/td>\nThe questionnaire<\/a> about job satisfaction used in the pre-test triggers employees to start thinking more consciously about their job satisfaction, leading to demand characteristics<\/a>.<\/td>\n<\/tr>\n
Sampling bias<\/a><\/th>\nParticipants of the study differ substantially from the population<\/a>.<\/td>\nEmployees participating in the experiment are significantly younger than employees in other departments, so the results can\u2019t be generalized<\/a>.<\/td>\n<\/tr>\n
Hawthorne effect<\/a><\/th>\nParticipants change their behavior because they know they are being studied.<\/td>\nThe employees make an extra effort in their jobs and feel greater job satisfaction because they know they are participating in an experiment.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

There are various other threats to external validity<\/a> that can apply to different kinds of experiments.<\/p>\n

Other interesting articles<\/h2>\n

If you want to know more about statistics<\/a>, methodology<\/a>, or research bias<\/a>, make sure to check out some of our other articles with explanations and examples.<\/p>\n

\n
\n
\n

<\/em>Statistics<\/strong><\/p>\n