5 Analyses (Construct & Criterion Validation)

5.1 Post-pilot administration modifications

Because the bifactor analyses resulted in some equivocation regarding final scale definitions, we entered into construct validation with some flexibility regarding final scale definitions. Twenty items were retained for construct validation, with the intent of further reducing the instrument to our desired 18 items upon psychometric evaluation of records obtained from this second round of data collection. Furthermore, four pilot items were lightly revised for grammatical considerations:

Pilot administration:

  • Item 1 (Pilot): I'm able to concentrate on my work without distractions
  • Item 19 (Pilot): I feel motivated to go beyond what is asked of me
  • Item 25 (Pilot): I plan my future with this company

Construct validation administration:

  • Item 1 (Revised): I am able to concentrate on my work without getting distracted
  • Item 19 (Revised): I feel motivated to go beyond what is asked of me at work
  • Item 25 (Revised): I plan to stay with this company as my career advances

Slight modification was also made to the response scale - the middle-most pilot administration options of “Slightly Disagree” and “Slightly Agree” had their semantic modifiers changed to “Somewhat”. The response scale offered in the construct validation was therefore:

  • Strongly Disagree
  • Disagree
  • Somewhat Disagree
  • Somewhat Agree
  • Agree
  • Strongly Agree

5.2 Procedure

Qualtrics panels were solicited, with the only eligibility requirement being working adults (either part- or full-time). The engagement items were presented in one of two orderings, prioritizing either the attitudinal or substantive structure:

5.2.1

Attitudinal Prioritization Substantive Prioritization
Affective - Absorption 0.66
Affective - Vigor 0.71
Affective - Dedication 0.75
Behavioral - Absorption 0.56
Behavioral - Vigor 0.7
Behavioral - Dedication 0.64
Cognitive - Absorption 0.59
Cognitive - Vigor 0.62
Cognitive - Dedication 0.83

5.2.2

5.2.3 Construct validation

For convergent validity indices, we administered the 17-item Utrecht Work Engagement Scale [UWES; Schaufeli et al. (2002); Schaufeli & Bakker (2010)] as well as the Saks (2006) 12-item scale which aggregates to two scales: job and organizational engagement (see also Saks, 2019). 5

The English version of the UWES may actually be a translation (difficult to say for sure, however, as the test manual describes an original Dutch sample although the manual is written in English), and some items have odd grammar (for example, “I am proud on the work that I do”).

Two short scales from the Oregon Avocational Interest Scales [ORAIS; Goldberg (2010)] were retained for discriminant validitation - the 5-item “Pets” and 5-item “Game-Playing” scales were administered.

  1. We had intended to use the Gallup “Q12” for construct validation (Harter et al., 2013; Thackray, 2005), but Gallup was not willing to share item- or person-level data.↩︎