Shorter Data Analysis Example

Descriptive Statistics

• Descriptive statistics summarize the data set (representing the study sample), identifying some key features such as central tendency and variability or spread.

• Skewness and kurtosis are numerical tests of the normality of data. The closer the skewness and kurtosis are to zero the closer the approximation of normality.

• Figure 1 presents a box plot as another test for normality. A violin plot or raincloud would be more useful visuals but are more difficult to read and interpret.

Figure 1: Distribution of Job Control Data.

Hypothesis Testing

• Hypothesis testing is a method of statistics inference used to decide if the sample data provide enough evidence or support to draw conclusions about the population. For this study, I used t-tests and ANOVA to compare means.

T-test and ANOVA

• Student’s t-test is a statistical test used to compare the means of two groups.

• A paired t-test compares means of two measurements taken from the same individual. For example, paired t-tests are frequently used for comparing pre- and post-test scores. In this case, the instruction and general job control scores are from the same group of people.

• Analysis of Variance (ANOVA) compares the means of multiple groups.

Reporting t-test Results

The paired t-test testing the difference between job control general and job control instruction suggests that the effect is positive, statistically significant, and medium (difference = 0.20, 95% CI [0.16, 0.25], t(244) = 8.29, p < .001; Cohen’s d = 0.53, 95% CI [0.40, 0.66]). The effect size is labeled following Cohen’s (1988) recommendations.

• P-values are frequently used to assess statistical significance and can carry significant weight. Generally, for research in LIS, a p-value less than 0.05 is significant. 0.05, 0.01, and 0.001 are frequently used thresholds.

P-values

• “the probability under a specified statistical model that a statistical summary of the data (e.g., the sample mean difference between two compared groups) would be equal to or more extreme than its observed value”

• “p-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone”

• “scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold”

Other Means of ‘Proof’

• p-values don’t measure the size of an effect. The effect size measures the strength of the relationship between the variables.

• Effect size can be measured and is measured differently depending on the statistical test. For t-tests, Cohen’s d can be used to measure the effect size. Various guidelines can help to interpret effect size.

• The confidence interval takes into account the size of the sample. There’s a 95% chance that the true value falls within the confidence interval.

• The 95% confidence intervals for the mean difference and Cohen’s d are reported. For example, there is a 95% chance that the true mean is between 0.16 and 0.25. For smaller samples, the confidence interval would be larger.

Just a Tiny Code Chunk

I included the code for the t-test in R below just to show how simple it is to run in R. Cohen’s d is also simple to calculate using the effectsize package from easystats.

t.test(d$jobcontrol.general.score.1989, 
       d$jobcontrol.instruction.score.1989, 
       paired = TRUE) 
## 
##  Paired t-test
## 
## data:  d$jobcontrol.general.score.1989 and d$jobcontrol.instruction.score.1989
## t = 8.2908, df = 244, p-value = 7.593e-15
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##  0.1552993 0.2520865
## sample estimates:
## mean difference 
##       0.2036929

cohens_d(t.test(d$jobcontrol.general.score.1989, 
                d$jobcontrol.instruction.score.1989, 
                paired = TRUE))
## Cohen's d |       95% CI
## ------------------------
## 0.53      | [0.40, 0.66]

Correlation & Linear Regression

• Pearson’s product-moment correlation is a measure of the strength of the linear correlation between two sets of data. The correlation coefficient (r) is a value between -1 and 1. Similarly to ANOVA and t-tests, Pearson’s product moment correlation is a parametric test.

• Linear regression is a method of modeling the relationship between variables. The coefficient of determination (r-squared) is a measurement of the proportion of variance in the dependent variable that can be explained by the independent variable. The value of the coefficient is always between 0 and 1.

Interpreting Correlation & Linear Regression

• Correlation measures the linear relationship between the variable and the linear regression allows us to predict the impact of the independent variable on the dependent variable.

• Linear regression may be described as explanatory. However, as with correlation, causation is not implied or inferred.

• Example: job control in instruction predicts a moderate proportion of variance in work-related burnout. Hence, we can predict that academic instruction librarians with low job control in instruction will experience greater work-related burnout in comparison to colleagues with greater job control.

The Pearson’s product-moment correlations between job control (instruction) and the three dimensions of burnout are as follows and as visualized in Figure 2:

• personal burnout: negative, statistically significant, and medium (r = -0.29, 95% CI [-0.40, -0.17], t(243) = -4.76, p < .001)

• work-related burnout: negative, statistically significant, and large (r = -0.36, 95% CI [-0.47, -0.25], t(243) = -6.10, p < .001)

• client-related burnout: negative, statistically significant, and large (r = -0.33, 95% CI [-0.44, -0.21], t(243) = -5.41, p < .001).

Figure 2: Correlation Matrix.

Linear models (estimated using OLS) were fitted to predict TWRBS (Figure 3), TPBS (Figure 4), and TCRBS (Figure 5) with job control when completing instructional responsibilities:

• TWRBS ~ Job Control (Instruction): The model explains a statistically significant and moderate proportion of variance (R2 = 0.13, F(1, 243) = 37.22, p < .001, adj. R2 = 0.13). The model's intercept, corresponding to Job Control (Instruction) = 0, is at 86.96 (95% CI [74.77, 99.15], t(243) = 14.05, p < .001). Within this model, the effect of Job Control (Instruction) is statistically significant and negative (beta = -11.85, 95% CI [-15.67, -8.02], t(243) = -6.10, p < .001; Std. beta = -0.36, 95% CI [-0.48, -0.25]). The model is shown in Figure 3 along with the linear model to predict TWRBS with Job Control (General).

Figure 3: Correlation between Work-related Burnout and Job Control.

• TPBS ~ Job Control (Instruction): The model explains a statistically significant and weak proportion of variance (R2 = 0.09, F(1, 243) = 22.61, p < .001, adj. R2 = 0.08). The model's intercept, corresponding to Job Control (Instruction) = 0, is at 85.16 (95% CI [73.27, 97.04], t(243) = 14.11, p < .001). Within this model, the effect of Job Control (Instruction) is statistically significant and negative (beta = -9.00, 95% CI [-12.73, -5.27], t(243) = -4.76, p < .001; Std. beta = -0.29, 95% CI [-0.41, -0.17]). The model is shown in Figure 4 along with the linear model to predict TPBS with Job Control (General).

Figure 4: Correlation between Personal Burnout and Job Control.

• TCRBS ~ Job Control (Instruction): The model explains a statistically significant and weak proportion of variance (R2 = 0.11, F(1, 243) = 29.23, p < .001, adj. R2 = 0.10). The model's intercept, corresponding to Job Control (Instruction) = 0, is at 63.41 (95% CI [50.52, 76.30], t(243) = 9.69, p < .001). Within this model, the effect of Job Control (Instruction) is statistically significant and negative (beta = -11.10, 95% CI [-15.15, -7.06], t(243) = -5.41, p < .001; Std. beta = -0.33, 95% CI [-0.45, -0.21]). The model is shown in Figure 5 along with the linear model to predict TCRBS with Job Control (General).

Figure 5: Correlation between Client-Related Burnout and Job Control.

Standardized parameters were obtained by fitting the models on a standardized version of the dataset. 95% Confidence Intervals (CIs) and p-values were computed using a Wald t-distribution approximation.

While the R2 for all of the above models using job control for instruction are lower than found in Johnson (2023) using job control in general, which suggests that job control for instruction explains a weaker proportion of variance, the data still suggest some impact of job control on burnout. Since burnout is a multi-faceted phenomenon, it makes sense that job control wouldn't be a sole predictor of burnout.

Analysis of variance can compare means across multiple groups. In this case, ANOVA tests were used for all of the demographic and job characteristic data collected in the study as the goal is to compare the means across multiple groups (e.g., among people of different genders, different income levels, etc.).

While one-way ANOVA can test the difference between multiple groups, Tukey’s HSD can be used for post-hoc or follow up analysis test for statistically significant differences between pairs of group means.

The ANOVA testing the effect of time since degree on job control for instruction suggests that the main effect is statistically significant and small (F(4, 238) = 2.81, p = 0.026; Eta2 = 0.05, 95% CI [2.91e-03, 1.00]). Post-hoc analysis using Tukey's HSD test revealed a significant difference between participants with 16 or more years since receiving their degree and participants with 1 to 5 years since receiving their degree (p < 0.05) as demonstrated in Table 4.

As demonstrated in Figure 6 and Table 3, job control for instruction generally increases over time, though the mean job control is highest for individuals who received their degrees less than a year ago, though the number of participants in that category is particularly low.

Figure 6: Average job control for instruction duties by intervals of numbers of years since library school.

The ANOVA testing the effect of whether or not teacher training was received on job contrtol for instruction suggests that the main effect is statistically significant and small (F(2, 242) = 3.38, p = 0.036; Eta2 = 0.03, 95% CI [9.70e-04, 1.00]). For this test, teacher training received was flattened to yes or no. Respondents had three yes options: "Yes, in library school and on the job," "Yes, only in library school," and "Yes, only on the job." The difference between these was not statistically significant, but the difference was statistically significant when comparing yes to no. Post-hoc analysis using Tukey's HSD test revealed a significant difference between respondents who received no teacher training and respondents who provided free text responses coded as "Other” as demonstrated in Table 6.

As demonstrated in Table 5 and Figure 7, mean job control for instruction is higher for those who have received training for library instruction than those who haven't; however, job control for instruction is particularly high for those who responded "Other." Responses for other generally referenced teacher training that was not specific to libraries or not provided through a library program.

Figure 7: Average job control for instruction duties by whether or not participant received training.