1. Loading, setting up

We use variables for cognitive (made from “Important to you” and “Important to your future goals” variables), behavioral (“How well were you concentrating” and “How hard were you working”) and affective (“Did you enjoy” and “Was the main activity interesting”) variables.

2. Identifying the number of MEPs.

In this section, we identity the numbers based on the r-squared values and the cross-validation Fleiss’ Kappa. The R-squared values are calculated from the k-means step of the two-step cluster analysis. The cross-validation is carried out 100 times each for two through nine profile solutions

r_squared_vals <- plot_r_squared(df, 
                                 behavioral_engagement,
                                 cognitive_engagement,
                                 affective_engagement, 
                                 challenge,
                                 good_at,
                                 to_center = TRUE,
                                 to_scale = TRUE,
                                 r_squared_table = F)

r_squared_vals

x <- cross_validate(df,
                    behavioral_engagement,
                    cognitive_engagement,
                    affective_engagement,
                    n_profiles = "iterate",
                    k = 30,
                    to_center = TRUE,
                    to_scale = TRUE)

x

The R-squared lot shows a flattening of the change in the R-squared value from four to five profiles, and from six to seven profiles, suggesting either a four or six profile solution.

The cross-validation results show that Fleiss’ Kappa is highest for the three profile solution, followed by the four, and then six profile solution. This same pattern is observed for the percentage agreement.

These results suggest investigating both a four and six cluster solution.

Creating profiles

p4 <- create_profiles(df,
                      behavioral_engagement,
                      cognitive_engagement,
                      affective_engagement,
                      n_profiles = 4,
                      to_center = TRUE,
                      to_scale = TRUE)

plot(p4)

four_p <- p4$.data

p6 <- create_profiles(df,
                      behavioral_engagement,
                      cognitive_engagement,
                      affective_engagement,
                      n_profiles = 6,
                      to_center = TRUE,
                      to_scale = TRUE)

plot(p6)

six_p <- p6$.data

Modeling

Using four-profile assignments to predict change in STEM interest

## Linear mixed model
##  Family: gaussian (identity)
## Formula: overall_post_interest ~ overall_pre_interest + race + gender + prof_1 + prof_2 + prof_3 + (1 | program_ID)
## 
##   ICC (program_ID): 0.000000
    overall_post_interest
    B std. Error p
Fixed Parts
(Intercept)   0.65 0.34 .056
overall_pre_interest   0.56 0.07 <.001
race (Black)   -0.06 0.22 .798
race (Hispanic)   0.03 0.21 .900
race (Multiracial)   -1.52 0.71 .031
race (White)   0.45 0.30 .134
gender (M)   0.14 0.12 .256
prof_1   0.99 0.24 <.001
prof_2   1.74 0.43 <.001
prof_3   0.32 0.26 .217
Random Parts
Nprogram_ID   9
ICCprogram_ID   0.000
Observations   140
R2 / Ω02   .501 / .501

Plotting partial pooling

ggplot(to_m, aes(x = overall_pre_interest))

Using six-profile assignments to predict change in STEM interest

## Linear mixed model
##  Family: gaussian (identity)
## Formula: overall_post_interest ~ overall_pre_interest + race + gender + prof_1 + prof_2 + prof_4 + prof_5 + prof_6 + (1 | program_ID)
## 
##   ICC (program_ID): 0.018982
    overall_post_interest
    B std. Error p
Fixed Parts
(Intercept)   0.87 0.41 .031
overall_pre_interest   0.54 0.07 <.001
race (Black)   -0.08 0.22 .727
race (Hispanic)   -0.01 0.21 .972
race (Multiracial)   -1.73 0.71 .015
race (White)   0.37 0.30 .218
gender (M)   0.12 0.12 .310
prof_1   0.43 0.54 .426
prof_2   1.78 0.44 <.001
prof_4   -0.18 0.38 .641
prof_5   0.76 0.33 .021
prof_6   0.51 0.34 .142
Random Parts
Nprogram_ID   9
ICCprogram_ID   0.019
Observations   140
R2 / Ω02   .531 / .531

Takeaways

Models with the four and six profiles seem to tell a similar story in terms of their effect on interest - both the “Full” profile and that with higher affective and behavioral engagement seem to be associated with change in interest,

On the other hand, the profile with slightly higher cognitive engagement in both has a positive coefficient, but does not significaly differ from 0 in its relationship with change in interest.

In both, multiracial students are associated with negative changes in interest, whereas none of the other demographic characteristics are.