Test #3 Research Methods Flashcards

1. Covariance
1. study’s results can establish this
2. if independent variables did not vary, or if there is no manipulating of the independent variable, then a study could not establish covariance
3. there is a need for comparison groups
2. Temporal Precedence
1. The cause variable precedes the effect variable
2. Experimenters manipulate the causal (independent) variable to ensure it comes first
3. Internal Validity
1. to be internally valid, a study must ensure that the causal variable and not other factors are responsible for the change in the outcome variable

1. independent group design
1. experimental design in which different groups of participants are exposed to different levels of the independent variable for each participant to experience only one level of the IV
2. separate groups of participants are placed into different levels of the independent variable
3. includes both the posttest-only design and the pretest/posttest design
4. disadvantage
1. groups can have extraneous differences (switch the disadvantage of within-groups
2. more participants and cost more
2. within-groups design
1. each participant is presented with all levels of the IV
2. acted on their own controls
3. advantages -
1. ensures the participants in the two groups will be equivalent
2. enables researchers to make more precise estimates of the difference between conditions
3. requires fewer participants overall
4. disadvantages
1. repeated measure designs have the potential for order effects, which can threaten the internal validity
2. design may not be possible or practical
3. when people see all levels of the IV and then change how they normally act

• experimental design in which different groups of participants are exposed to different levels of the independent variable for each participant to experience only one level of the IV
• separate groups of participants are placed into different levels of the independent variable
• includes both the posttest-only design and the pretest/posttest design
• disadvantage
1. groups can have extraneous differences (switch the disadvantage of within-groups
2. more participants and cost more

• each participant is presented with all levels of the IV
• acted on their own controls
• advantages -
• ensures the participants in the two groups will be equivalent
• enables researchers to make more precise estimates of the difference between conditions
• requires fewer participants overall
• disadvantages
• repeated measure designs have the potential for order effects, which can threaten the internal validity
• design may not be possible or practical
• when people see all levels of the IV and then change how they normally act

• design confounds
• a threat to internal validity in an experiment in which a second variable happens to vary systematically along the IV
• an alternative explanation for the results
• experimenter’s mistake in designing the independent variable, carelessness
• ex. See if people write better on pink or green paper and schedule the test at 8 am and one at 1 pm
• selection effects
• occurs in an independent design when the kinds of participants at one level of the independent variable are systematically different from those at the other level
• can be avoided → with random assignment
• assignng participants at random to different levels of the independent variable controls for all sorts of potential selection effects
• order effects

• manipulated variable
• A variable in an experiment that a researcher controls
• ex. assigning participants to it’s different levels (values)
• usually the independent variable
• measured variable
• a variable in a study whose levels (values) are observed and recorded
• usually the dependent variable
• experimental condition
• a study in which at least one variable is manipulated and another is measured

• a group in an experiment whose levels on the IV differ from those of the treatment group in some intended and meaningful way
• can see if there is a difference or a contrast

• assigning participants to different experimental groups
• important b/c it controls for all sorts of potential selection effects and a way of desystematizating the types of participants who end up in each level of the IV

• pretest
• participants are randomly assigned to at least 2 groups and are tested on the key dependent variable twice - once before and once after exposure to the IV
• advantages
• can be used when they want to make sure random assignment made groups equal, enabling researchers to track people’s change in performance over time (not seeing change over time)
• similar to within-group
• looking for changed scores
• ex. seeing one group change 1% or 10%
• disadvantages
• can be problematic
• informing what the participants are doing in the test, can change the effect of the study
• test with the Solomon test design

participants respond to a dependent variable more than once, after exposure to each level of the independent variable

• reason to combine independent variables in a factorial design so you can study how they interact
• can measure whether the results are consistent with the theory

• a difference in difference
• results from a factorial design in which the difference in the levels of one IV changes depending on the level of the other IV
• the effect of one independent variable depends on the level of the other independent variable

• main effect: the overall effect of one independent variable on the dependent variable, averaging over the labels of the other independent variables
• ex. there is not a main effect because the no difference in the marginal means OR There are main effects because there is a difference in the marginal mean *different enough, can be a small number, but be reasonable *
• provide a reasonable explanation
• interactions
• There is no interaction between the 2 variables because the lines are parallel
• There is an interaction between the 2 variables because the lines are not parallel

• the overall effect of one independent variable on the dependent variable, averaging over the labels of the other independent variables
• ex. there is not a main effect because the no difference in the marginal means OR There are main effects because there is a difference in the marginal mean *different enough, can be a small number, but be reasonable *
• provide a reasonable explanation

1. There is no interaction between the 2 variables because the lines are parallel
2. There is an interaction between the 2 variables because the lines are not parallel

• To test limits means to test for moderators
• moderator: variable that changes the relationship between the other two variables
• One can test theories by seeing how variables interact and combine them in a factorial design. You then measure whether or not the results are consistent with this.

• independent-groups factorial design
• both IVs are studied as independent groups
• can be used as a posttest-only design and pretest/posttest design
• within-groups factorial design
• each person is presented with all levels of the independent variable
• randomly assigned
• mixed factorial design
• one independent variable is manipulated as independent groups and the other is manipulated as within groups
• ex. 2 groups of people eat carbs & lift weights + aerobic, another group would eat protein & lift weights + aerobic
• one part is between groups and another part is within-group

both IVs are studied as independent groups

can be used as a posttest-only design and pretest/posttest design

each person is presented with all levels of the independent variable

randomly assigned

• one independent variable is manipulated as independent groups and the other is manipulated as within groups
• ex. 2 groups of people eat carbs & lift weights + aerobic, another group would eat protein & lift weights + aerobic
  one part is between groups and another part is within-group

• get info about, add more information about the independent variable
• explain understanding of the relationship
• ex. to find food consumption on stress, you can choose specifically what type of food (carbs & protein)

differ from true experiments because the researchers do not have full experimental control

• Small-N designs focus on one or a few people in a study, gather information from just a few cases
• Small-N designs
1. each participant is treated separately in; small-N designs are almost always repeated measures designs
2. data for each individual are presented
3. careful designs enable us to compare each individual during treatment periods and control periods

• stable baseline: a study in which a practitioner or researcher observes behavior for a baseline period before beginning a treatment or other intervention
• make sure behavior is stable
• result of small N experiments -
• multiple baselines: researchers stagger their introduction of an intervention across a variety of individuals, times, or situations to rule out alternative explanations
• result of small N experiments -
• reversal baseline: Researchers observe a problem behavior both with and without treatment, but take away the treatment for a while (the reversal period) *taking away treatment can be caused by one of the threats * to see whether the problem behavior returns (reverses)
• result of small N experiments - enables a practitioner to evaluate the effectiveness of a treatment, it may be considered harmful and unethical to withdraw an effective treatment

• a study in which a practitioner or researcher observes behavior for a baseline period before beginning a treatment or other intervention
• make sure behavior is stable
• result of small N experiments -

• researchers stagger their introduction of an intervention across a variety of individuals, times, or situations to rule out alternative explanations
• result of small N experiments -

• Researchers observe a problem behavior both with and without treatment, but take away the treatment for a while (the reversal period) *taking away treatment can be caused by one of the threats * to see whether the problem behavior returns (reverses)
• result of small N experiments - enables a practitioner to evaluate the effectiveness of a treatment, it may be considered harmful and unethical to withdraw an effective treatment

• interrupted time-series design: a quasi-experiment in which participants are measured repeatedly on a dependent variable before, during, and after the “interruption” caused by some event
• we use them to see if the independent variable changes

• important to be replicable b/c you will be able to interrogate statistical validity
• gives the study credibility and important part of the scientific process
• Direct replication
• Researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data
• use this when you want to confirm what you already know about the study
• Conceptual Replication
• researchers explore the same research question but use different procedures
• replication - plus extension
• reserachers replicate their original experimemt and add variables to test additional questions

• Researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data
• use this when you want to confirm what you already know about the study

researchers explore the same research question but use different procedures

researchers replicate their original experiment and add variables to test additional questions

• what does the literature say about the topic?
• meta-analysis
• a way of mathematically averaging the effect sizes of all the studies that have tested the same variables to see what conclusions that whole body of evidence supports
• one would do one b/c → to see the overall effect size, can detect new patterns in literature, and test new questions

• theory testing
• not need much external validity in comparison to internal validity
• need to generalize when testing theory
• researcher’s intent for a study, testing association claims or causal claims to investigate support for a theory
• generalization modes
• intent of researchers to generalize the findings from the samples and procedures in their study to other populations or contexts
• more concerned with external validity b/c they need to take in samples with appropriate diversity of gender, age, ethnicity, etc.

• not need much external validity in comparison to internal validity
• need to generalize when testing theory
• researcher’s intent for a study, testing association claims or causal claims to investigate support for a theory

• intent of researchers to generalize the finding from the samples and procedures in their study to other populations or contexts
• more concern with external validity b/c they need to take in samples with appropriate diversity of gender, age, ethnicity, etc.

• theory-testing mode
• association claims
• casual claims
• Generalization mode
• frequency claims (always): representation samples are important for supporting frequency claims
• association (sometimes):
• casual (sometimes):