o Return to: 'SPRING SYSTEMS HOME PAGE'
03-13-98
The issue in this comment area is not really experimental design per se, but the more weighty subject of how we know that X causes Y, i.e. what are we willing to accept as evidence for CAUSE and EFFECT. More often than not we are willing to rely on nothing more than anecdotal associative evidence that X causes Y. Simply because X and Y occur together (once in the anecdotal mode, several times in the statistical mode) is taken as evidence that X causes Y.
At best, this kind of information is barely adequate to formulate a hypothesis. If there is correlation or more loosely based association, there may or may not be cause and effect. We have to test for this experimentally.
The minimal requirement for a decent EXPERIMENTAL DESIGN is that there exist matched TEST and CONTROL groups, and that they are measured on a PRE/POST basis on the DEPENDENT VARIABLE. If the time interval between the PRE and POST measurements is minimal, a CONTROL group is less critical and PRE/POST measures on just the TEST group may be adequate (see CONCEPT MAPPING).
What this also implies when we examine evidence, is that we should demand to know (at a minimum), what kind of CHANGE (rather than absolute level) the INTERVENING (TEST) VARIABLE is associated with, and that nothing else can be reasonably assumed to account for this CHANGE (i.e. the PRE/POST reading in the CONTROL group should be ZERO).
The NULL HYPOTHESIS is that the introduction of the INTERVENING (TEST) variable has no effect, and the CHANGE is therefore ZERO. To disprove this hypothesis, the experiment must yield a PRE/POST CHANGE for the TEST group that is both statistically greater than ZERO, and also larger than the CHANGE observed in the CONTROL group. This CHANGE must be sufficiently large given the sample sizes of the TEST and CONTROL groups, to minimize the probability that it could have occured by chance alone. We typically accept a 5% (or smaller) probability that the result could have occured by chance.
In other words, EXPERIMENTAL evidence rather than mere association should be the criterion for judging that a CAUSE and EFFECT relationship is plausible. Anything else (including expert opinion/testimony) should be viewed with skepticism unless it's backed up by EXPERIMENTAL evidence.
Something this dumb would put most lawyers out of business if judges, juries, and the general public used it as a criterion for evidence. In any event, it would certainly go a long way toward curtailing Junk Science.
05-16-07
Experimental design can be implemented in two ways. First of all, it can be treated as a special case of MULTIPLE REGRESSION, with ZERO/ONE dummy variable codes set up as independent variables associated with a dependent variable. Here's a link to some MULTIPLE REGRESSION info:
(SEE: MULTIPLE REGRESSION)
Another way is to use module #14 ANOVA in the Spring-Stat(c) statistical system.
(SEE: SPRING-STAT(c) MODULES)
o Return to: 'SPRING SYSTEMS HOME PAGE'