The results of the paired t-test are reported above. P-values are frequently used to assess statistical significance and can carry significant weight. Generally, for research in LIS, a p-value less than 0.05 is significant. 0.05, 0.01, and 0.001 are frequently used thresholds.
In 2016, the American Statistical Association (ASA) released a statement on p-values and statistical significance. They informally defined a p-value as “the probability under a specified statistical model that a statistical summary of the data (e.g., the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.” They note both that “p-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone” and that “scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold,” which is possibly to say that these thresholds are arbitrary.
In the same statement, the ASA notes that p-values don’t measure the size of an effect. As such, in my reporting of the results, I’ve included the effect size. For t-tests, the effect size is measured using Cohen’s d. Based on Cohen’s recommendations in his 1988 textbook, 0.5 is considered medium. The effect size measures the strength of the relationship between the variables.
Additionally, the 95% confidence intervals for the mean difference and Cohen’s d are reported. The confidence interval takes into account the size of the sample. There’s a 95% chance that the true value falls within the confidence interval. For example, there is a 95% chance that the true mean is between 0.16 and 0.25. For smaller samples, the confidence interval would be larger.
I included the code for the t-test in R below just to show how simple it is to run in R. Cohen’s d is also simple to calculate using the effectsize package from easystats. Click the arrow to unfold the code.
