Alan S. Gerber, Donald P. Green, and Edward H. Kaplan, “The illusion of learning from observational research”, in Ian Shapiro, Rogers Smith, and Tarek Massoud, Problems and Methods in the Study of Politics. (2004), pp. 251-73

Main Argument: The principal difference between experimental and observational research is the use of randomization procedures. The risk of bias is typically much greater for observational research and it is difficult to detect the direction and magnitude of biases in observational research. Therefore we must be careful in the application of the inferences derived from observational research in the furthering of theory.
Key Definitions:
Illusion of Observational Learning Theorem [GGK]: In the absence of prior knowledge about bias of OR, one accords it zero weight [see 257]
== Notes: ==
*  The aim of experimental research is to examine the effects of random variation in one or more independent variables
*  In observational studies, the data generation process by which the independent variables arise is unknown to the researcher
*  The risk of bias is typically much greater for observational research
*  The weight accorded to new evidence depends upon what methodological inquiry reveals about the biases associated with an estimation procedure as well as what theory asserts about the biases [252]
Assumptions around Observational Research:
*  DV is distributed normally with mean μ and variance σ2M.
*  central limit theorem leads you to believe that your estimator, Xe, will be normally distributed
*  we assume that priors about DV and IV are independent [multicollinearity]
*  the observational study produces an estimate (xo) that may be biased in the event that b is not equal to 0
In sum, our model of the research process assumes:
(1) normal and independently distributed priors about the true effect and the bias of observational research and
(2) normal and independently distributed sampling distributions for the estimates
Justifications for experimentation:[263]
(1) uncertainty about the biases associated with observational studies
(2) ample resources
(3) inexpensive access to experimental subjects
(4) features of experimental design that limit disturbance variability
What about Bias in Experiments?
*  The external validity of an experiment hinges on four factors:
•     Influenced of treatment on experimental grp vs the population
•     Treatment in the experiment corresponds to the treatment in the population of interest
•     Response measure used in the experiment corresponds to the variable of interest in the population
•     How the effect estimates were derived statistically
The Value of Methodological Inquiry
*  Difficult to detect the direction and magnitude of biases in observational research
*  One potential advantage of experimentation is that it imposes discipline on data analysis by forcing researchers to specify their research design and statistical model in advance [267]
*  Biased samples can be useful so long as one possesses a strong analytic understanding of how they are likely to be biased
When is Observational Learning Not Illusory?
*  Experimentation requires no control group when range of alternative explanations is so small
*  it may be better to study fewer observations, if those obs are chosen ways that minimize bias [268]