Significance and p-values

How do I interpret my results?

There's a measurement called the "p-value" which a statistical test returns. The p-value indicates how likely it is that you would have seen the difference between your experimental and control conditions if the null hypothesis were actually true.

The smaller the p-value, the more likely it is that the difference between your experimental condition and your control condition is the result of a "real" difference, and not random chance.

In the behavioral sciences, there's a tradition that the value at which one can publish is p <= .05 -- so less than a 5% chance that you would have gotten this data if the difference were not really there.

next: pilot experiments

Experimental Design for Wearable Computing -- Pilot Experiments

Pilot Experiments

What do I do if my experiment didn't work?
  • Show some pictures with a subliminal cue, and some without, and count how many are correctly recognized.

Out of all the pictures you chose to show, it may be that some are so inherently memorable that people recognize them regardless of the condition in which they appear. These would be considered "outlier stimuli", and may cloud the significance of a genuine effect.

There's a tradition of running "pilot" experiments, which test out one's design as well as one's hypotheses. If you think you really do have an effect, but your experiment failed to demonstrate it conclusively -- call what you've done a "pilot", and revisit your design to see if you can create a better experiment.

Hint: if your p-value is between .05 and .1, you may be able to demonstrate significance with a better-designed experiment.

the end of the lecture, and on with the workshop!