Analysis of Small Sample Size

This study was done specifically to research the results of a certain test when the sample size of the research was very small (De Winter, 2013, p. 1). A larger sample size may allow for a greater margin of error in the results (and also allow for a greater range of results, thereby increasing the researcher’s ability to extrapolate their findings into an effect size). On the other hand, a smaller sample size of N<5 might be helpful to someone who has limited resources and therefore cannot afford to commit too many of them to the statistical research design (De Winter, 2013, p. 1).

A further disadvantage to a small sample size is of course that no testing of the results can be done. Usually, when an experiment is done, a similar test is undertaken with a larger sample to test the results of the original experiment, but since the original sample taken was so small, such an approach is not possible (De Winter, 2013, p. 2). There was, however, also no indication that t-test results were materially damaged by such a small sample size (de Winter, 2013, p. 6)

The experiments to be described in this paper are all different in terms of the sample types and of course the subjects of their experiments. In the third experiment, for example, the researcher was testing the mean reaction time of five different people (which is where the t-test comes in); but in the case of the second experiment, the researcher had not originally intended for his experiment to only cover four subjects. Unfortunately, a hardware failure forced him to continue with that sample size (De Winter, 2013, p. 1).

The study specifically refers to small-n theories in the first few pages of this article, referring to the number of samples in each case (three, four and five respectively (De Winter, 2013, p. 1)). The article does not refer to any of the researchers acting as their own control, which is a requirement for a single-subject experiment, so the assumption can be made that none of the experiments fall into that category. Similarly, all three experiments are continually referred to as using the N<5 theory as a basis for their experiments, with no mention of the phrase case study; it seems safe to assume that they are small-n experiments rather than case studies of any kind.

Reference

De Winter, J.C.F. (2013) Using the student’s t-test with extremely small sample sizes. Practical Assessment, Research and Evaluation, 18(10), 1-12. Retrieved from https://owl.english.purdue.edu/owl/resource/560/10/