What type of validity is not prioritized when testing association or frequency claims but is when testing casual claims?

Presentation on theme: "CHAPTER 3 Three Claims, Four Validities: Interrogation Tools for Consumes of Research PART I."— Presentation transcript:

1 CHAPTER 3 Three Claims, Four Validities: Interrogation Tools for Consumes of Research
PART I

2 Chapter Overview Variables Three claims
Interrogating the three claims using the four big validities Prioritizing validities This chapter sets up the framework for the textbook, so it is a very important chapter to understand. The lecture should start with a discussion on what variables are. Then, you should continue with a discussion about the three types of claims (frequency, association, and causal) and the four validities (construct, external, statistical, and internal), and how the validities are prioritized differently depending on what kind of claim is being made.

3 Variables Variable versus constant
Measured variable and manipulated variable From conceptual variable to operational definition Variable: A variable is something that changes or varies, so it needs to have at least two levels or values (but it can have more). Here are examples of variables from made-up headlines: “80% of college undergraduates have sexted.” Sexting is the variable, and there are two levels: those who have sent sexually explicit photos electronically, and those who have not. “People who attend church regularly lie more often than others.” There are two variables here: church attendance (attend church at least once a month or do not attend church at least once a month) and lying (some numerical score on a lying inventory). Constant: A constant does not vary. In other words, it stays the same. Here is an examples of a constant from a made-up headline: “30% of women have been sexually harassed in the workplace.” Gender is a constant. Next, we will look at different types of variables: Measured variables Manipulated variables Conceptual variables Operationalized variables

4 Measured and Manipulated Variables
A measured variable is observed and recorded. A manipulated variable is controlled. Some variables can only be measured—not manipulated. Some variables can be either manipulated or measured Researchers either measure or manipulate variables in a study. Measured variable: Levels are observed and recorded by the researcher. Examples of a measured variable include height, IQ, gender, and hair color. Psychologists also measure more abstract variables such as depression and stress. Manipulated variable: The researcher controls a variable, usually by assigning participants to different levels of that variable. As an example of a manipulated variable, a researcher might assign some people to take a test in a room with many other people and assign others to take the test alone. Some variables can only be measured—not manipulated: You cannot assign participants to be a particular age, as age is a naturally occurring variable. Also, sometimes it is unethical to manipulate variables. For example, in a study on the long-term effects of elementary education, you could not ethically assign children to “high-quality school” and “low-quality school” conditions. Some variables can be either manipulated or measured: For example, if childhood extracurricular activities were the variable of interest, you could measure whether children already take music or drama lessons, or you could manipulate this variable by assigning some children to take music lessons and others to take drama lessons.

5 From Conceptual Variable to Operational Definition
Variable name, conceptual variable Operational definition, one possibility Levels of this variable Is the variable measured or manipulated in this context Car ownership Researchers asked people to circle "I own a car" or "I do not" on their questionnaire. 2 levels: own a car or not Measured Expressing gratitude to a romantic partner Researchers asked people in relationships the extent to which they agree with items such as "I tell my partner often that she or he is the best" 7 levels, from 1, strongly disagree, to 7, strongly agree Type of story told about a scientist Researchers assigned participants to read stories about Einstein and Curie, which related either their work struggles or their achievements 2 levels: a story about a scientist's struggles and a story about a scientist's achievements Manipulated What time children eat dinner Using a daily food diary, researchers had children write down what time they ate dinner each evening. Researchers divided children into two groups: those who ate dinner between 2 P. M. and 8 P. M., and those who ate after 8 P. M. Table 3.1: Describing Variables Conceptual variables, construct, and conceptual definitions are abstract, theoretical concepts such as “infant temperament” and “anxiety.” I think of them as being “up in the clouds,” but we need to bring them down to the ground from the lofty theoretical level so that we may measure them. Operational definitions, operational variables, and to operationalize: In order to test their hypotheses with empirical data, researchers need to develop operational definitions, or operational variables. To operationalize is to turn a conceptual definition into a measured or manipulated variable. Example: spending time socializing is a conceptual variable, and how often a person spends an evening alone, socializes with friends, and sees relatives in a typical week are operational variables.

6 Operationalizing “School Achievement”
Figure 3.2: Operationalizing “School Achievement” A conceptual variable can be operationalized in different ways. The conceptual level is “school achievement.” Operational definitions can include self-report questionnaires, checking records, and obtaining teachers’ observations.

7 Three Claims Frequency claims Association claims Causal claims
Claim type Sample headline Frequency Claims 4 in 10 teens admit to texting while driving, 42 percent of Europeans never exercise, middle school kids see 2 to 4 alcohol ads a day Association claims Single people eat fewer vegetables, angry twitter communities linked to heart deaths, girls more likely to be compulsive texters, suffering a concussion could triple the risk of suicide Casual Claims Music lessons enhance I Q, babysitting may prime brain for parenting, family meals curb eating disorders, why sleep deprivation makes you crabby Frequency claims Association claims Causal claims Not all based on research Table 3.2: Examples of Each Type of Claim Claim: An argument someone is trying to make. Psychological scientists use data to test and refine theories and claims. Frequency claim: one variable Association claim: two variables that are related Causal claim: two variables, one of which causes the other

8 Frequency Claims A frequency claim describes a particular rate or degree of a single variable. Frequency claims involve only one measured variable. Examples: Two Out of Five Americans Say They Worry Every Day Just 15% of Americans Smoke 72% of the World Smiled Yesterday 4 in 10 Teens Admit to Texting While Driving

9 Association Claims An association claim argues that one level of a variable is likely to be associated with a particular level of another variable. Association claims involve at least two measured variables. Variables that are associated are said to correlate. Association claim: One level of a variable is likely to be associated with a particular level of another variable. Examples: People with Higher Incomes Spend Less Time Socializing Romantic Partners Who Express Gratitude Are Three Times More Likely to Stay Together People Who Multitask the Most Are the Worst at It A Late Dinner Is Not Linked to Childhood Obesity, Study Shows Correlate: Variables covary, meaning they are related; as one variable changes, the other tends to change, too. Three basic types of associations: Positive Negative Zero

10 Positive Association Figure 3.4A: Positive Association
Positive association (also known as positive correlation): High scores on one variable are associated with high scores on another variable or low scores on one variable are associated with low scores on another variable. Example (see the figure): Romantic partners who express gratitude are more likely to stay together. Scatterplot: One way to represent an association is to plot one variable on the x-axis and the other variable on the y-axis The slope of a positive association goes up as you move from left to right.

11 Negative Association Figure 3.4B: Negative Association
The figure depicts a scatterplot showing negative association (also known as inverse association or negative correlation): High scores on one variable are associated with low scores on another variable or low scores on one variable associated with high scores on another variable. Example: “People who multitask the most are the worst at it.” Describe the negative slope, which decreases as you move from left to right across the scatterplot.

12 Zero Association Figure 3.4C: Zero Association
The figure depicts a scatterplot showing zero association (also known as zero correlation): There is no association between variables. Example: “A late dinner is not linked to childhood obesity, study shows.” Describe the cloud of points that results.

13 Making Predictions Based on Associations
Some association claims are useful because they help us make predictions. The stronger the association between the two variables, the more accurate the prediction will be. Both positive and negative associations can help us make predictions, but zero associations cannot. Some association claims are useful because they help us make predictions. Prediction is using the association to make our estimates more accurate; it does not necessarily mean we can know what will happen in the future. The stronger the association between the two variables, the more accurate the prediction will be. And the weaker the relationship, the less accurate the prediction will be. Both positive and negative associations can help us make predictions, but zero associations cannot.

14 Verbs for Association and Causal Claims
Association Claim Verbs Casual Claim Verbs Is linked to Causes Promotes Is at higher risk for Affects Reduces Is associated with May curb Prevents Is correlated with Exacerbates Distracts Prefers Changes Fights Are more or less likely to May lead to Worsens May predict Makes Increases Is tied to Sometimes makes Trims Goes with Hurts Adds Table 3.3: Verb Phrases That Distinguish Association and Causal Claims Causal claim: One of the variables is responsible for changing the other; one measured variable and one manipulated variable. Like an association claim, the two variables covary. Example: Music lessons enhance IQ. Verbs for casual claims: affects leads to reduces Verbs for association claims: is associated with is related to is linked to A causal claim that contains tentative language (could, may, seem, suggest) is still a causal claim. Example: Music lessons may enhance IQ.

15 Not All Claims Are Based on Research
Not all claims we read about in the popular press are based on research. Some claims are based on experience, intuition, or authority. As discussed in Chapter 2, there are other ways of knowing besides research (e.g., experience, intuition, or authority). Not all claims that we see, hear, or read about in the popular media are based on research. Examples: 12-Year-Old’s Insight on Autism and Vaccines Goes Viral Living in the Shadow of Huntington’s Disease Baby Born Without Skull in the Back of His Head Defies Odds

16 Interrogating the Three Claims Using the Four Big Validities
Interrogating frequency claims Interrogating association claims Interrogating causal claims Valid: reasonable, accurate, and justifiable; validity is the appropriateness of a conclusion or decision.

17 The Four Big Validities
Type of Validity Description Construct validity How well the variables in a study are measured or manipulated. The extent to which the operational variables in a study are a good approximation of the conceptual variables. External validity The extent to which the results of a study generalize to some larger population, for example, whether the results from this sample of children apply to all U. S. schoolchildren, as well as to other times or situations, for examples, whether the results are based on this type of music apply to other types of music. Statistical validity The extent to which the data support the conclusions. Among many other questions, it is important to ask about the strength of an association and its statistical significance, the probability that the results could have been obtained by chance if there really is no relationship. Internal validity In a relationship between one variable, A, and another, B, the extent to which A, rather than some other variable, C, is responsible for changes in B. Table 3.4: The Four Big Validities Introduce the four big validities in general before describing how they are interrogated for the three different types of claims.

18 Interrogating Frequency Claims
Construct validity External validity, or generalizability Statistical validity When interrogating frequency claims, we will want to focus on construct validity and external validity. Statistical validity may be relevant as well. Construct validity: How well a conceptual variable is operationalized. Example: 80% of college students have been depressed during the last year. How was depression operationally defined? Were college students simply asked if they’d ever been depressed during the last year, or was the frequency claim based on a score on a depression inventory or records from psychological services on campus? External validity (also known as generalizability): How well the results of a study represent the people or contexts besides those in the study itself. Example: 72% of the world smiled yesterday. Which people did they survey, and how did they choose their participants? Did they include only people in major urban areas? Did they ask only college students from each country? Or did they attempt to randomly select people from every region of the world? Statistical validity (also known as statistical conclusion validity): The extent to which the study’s conclusion are reasonable and accurate. The questions asked will vary depending on the claim. When interrogating frequency claims, percentages are usually accompanied by a margin of error (a statistic based on sample size which indicates where the true value in the population probably lies. Example: In the report about how many teenagers text while driving, the Centers for Disease Control’s finding of a 41% value was accompanied by this note: “The margin of error is +/–2.6 percentage points” (CDC, n.d.). The margin of error helps describe how well our sample estimates the true percentage. Specifically, the range of 38.4–43.6% is highly likely to contain the true percentage of teens who text while driving.

19 Interrogating Association Claims
Construct validity External validity Statistical validity A reminder: Association claims involve the relationship between two measured variables, so we must interrogate construct, external, and statistical validities. Construct validity: similar to construct validity for a frequency claim except that you now have two conceptual variables that need to be operationalized appropriately. Example: People who multitask are the worst at it. How were the frequency and ability to multitask measured? Ability to multitask could be measured by self-report (How good are you at multitasking?) or by an observational measure in which participants are given a task that requires them to multitask. Frequency of multitasking could also be self-report (keep track of how many times you multitask) or by observing participants. External validity: Does the association claim generalize to other populations, contexts, times, or places? Example: For the association between expressing gratitude and relationship length, you would ask whether the results from this study’s participants, 194 California college students currently in a romantic relationship, would generalize to other people and settings. Would the same results be obtained if all of the participants were midwest­ern couples 45 or older? You can evaluate generalizability to other contexts by asking, for example, whether the link between gratitude and relationship length also exists in friendships. Statistical validity: When referring to an association claim, statistical validity is the extent to which the statistical conclusions are accurate and reasonable. There are two aspects of statistical validity to consider: strength and statistical significance and two kinds of mistakes. (See next slide for details.)

20 Statistical Validity of Association Claims
Strength and significance Avoiding two mistaken conclusions Type I error Type II error Strength: How strong is the association? Some associations are strong and others are weak. Remember that the stronger the association, the more accurate our predictions will be. Statistical significance: If an association is statistically significant, then the result is probably not due to chance based on that sample. If an association isn’t statistically significant, then the result probably is due to chance. Avoiding two mistaken conclusions: When interrogating the statistical validity of an association claim, there are two kinds of mistakes that can be made. Type I error: false positive; a study might mistakenly conclude that there is an association between two variables in their sample when there actually is no association in the population. You want to increase the chances that you will find an association only when there really is an association. Type II error: a “miss”; a study might mistakenly conclude from a sample that there is no association between two variables when there actually is an association in the population. You want to minimize, or at least reduce, the chances of missing associations that are really there.

21 Table 3.5: Interrogating the Three Types of Claims Using the Four Big Validities
Table 3.5: Interrogating the Threes Types of Claim Using the Four Big Validities This table is a summary of how we interrogate the three claims (frequency, association, and causal) using the four big validities (construct, statistical, internal, and external).

22 Interrogating Causal Claims
Three Criteria for Causation Covariance Temporal precedence Internal validity Criterion Definition Covariance The study's results show that as A changes, B changes; for example, high levels of A go with high levels of B, and low levels of A go with low levels of B. Temporal precedence The study's method ensures that A comes first in time, before B. Internal validity The study's method ensures that there are no plausible alternative explanations for the change in B; A is the only thing that changed. Table 3.6: Three Criteria for Establishing Causation Between Variable A and Variable B Because causal claims state that one variable causes another variable, they use directional verbs like leads to, affects, and influences. There are three criteria for causation: Covariance: This simply means that the two variables are related. Association claims fulfill this criterion. Temporal precedence: one variable comes before the other variable in time. Because the research is manipulating one variable and then measuring the other variable, she knows the manipulated variable comes before the outcome variable, which is measured after the manipulation. Internal validity (also known as the third-variable criterion): a study should be able to eliminate alternative explanation. In other words, Variable A is the only thing that changed.

23 Experiments Can Support Causal Claims
Independent variable Dependent variable Random assignment Figure 3.5: Interrogating a Causal Claim In order to support a causal claim, a researcher needs to conduct an experiment in which one variable is manipulated and the other is measured. Independent variable: manipulated or variable (cause) in an experiment (e.g., type of lessons; four levels: keyboard, voice, drama, no lessons). Dependent variable: measured variable (effect) in an experiment (e.g., IQ) Random assignment: A method of assigning participants to levels of the independent variable such that each group is as similar as possible; flipping a coin or rolling a die; increases internal validity by controlling for potential alternative explanations. Example: We know that it was the lessons that caused the differences in IQ and not something else because IQ was similar in the four groups due to random assignment.

24 When Causal Claims Are a Mistake
Does eating meals as a family really curb eating disorders? Does social media pressure cause teen anxiety? Figure 3.7: Support for a Causal Claim? Sometimes an article in the popular press may sound like it’s a causal claim when really it’s not. Here are two examples: Do family meals really curb eating disorders? Covariance: Yes, there is an association between family meals and eating disorders. Temporal precedence: Did the family meals increase before the eating disorders decreased? It’s not clear. Internal validity: No. We can’t rule out third variable explanations without an experiment. Other possible explanations: (1) Perhaps girls from single-­parent families are less likely to eat with their families and are vulnerable to eating disorders, whereas girls who live with both parents are not. (2) Maybe high-achieving girls are too busy to eat with their families and are also more susceptible to eating-­disordered behavior. Does Social Media Pressure Cause Teen Anxiety? The relationship sounds causal, but is it? Covariance: Yes. The results showed that teens who felt more pressure to respond immediately to social media were also more anxious. Temporal precedence: No. this was a correlational study, in which both variables were measured at the same time Internal validity: No. It was not an experiment so it did not rule out possible alternative explanations.

25 Other Validities to Interrogate in Causal Claims
Construct validity External validity Statistical validity We have been emphasizing internal validity because it is only important when interrogating causal claims. However, we still need to consider the three other types of validity: Construct validity: Always important regardless of what kind of claim you’re making; need to consider the construct validity of both the manipulated variable and the measured variable External validity: Do the results generalize to other people or contexts? This probably isn’t paramount since internal validity tends to be emphasized in experiments, and internal and external validity are kind of like two sides of a coin. As internal validity increases, external validity decreases and vice versa. Statistical validity: How strong is the relationship between the independent variable and the dependent variable? Is the difference between groups statistically significant?

26 Prioritizing Validities
Which of the four validities is the most important? It depends on what kind of claim the researcher is making and the researcher’s priorities. It’s impossible to find a study that satisfies all four validities at once. This means that researchers must decide on their priorities according to the goals of the study. Example: Internal validity is typically a top priority when making causal claims but not when making frequency or association claims.

27 Conclusion Chapter 3 Research Methods in Psychology Third Edition
This concludes the Lecture Slides for Chapter 3 Research Methods in Psychology Third Edition by Beth Morling For more resources to accompany this text, see wwnorton.com/instructors and everydayresearchmethods.com.

What validity is important for frequency claims?

Which validity is most important for causal claims?

What is construct validity examine in association claim?

What type of validity is a priority when interrogating an experiment?