Discussion
Each of the disciplines that contribute to STEM learning - science, technology and computer science, engineering, and mathematics - involve work with data. In this study, engagement was used as a lens to understand the experience of youth working with data during summer STEM programs. In particular, five aspects of work with data, a) asking questions, b) observing phenomena, c) constructing measures and generating data, d) data modeling, and e) interpreting and communicating findings, were occurred regularly in the programs. There were some examples of ambitious activities centered on working with real-world data as well as some that highlight substantial heterogeneity in how work with data was enacted.
I identified six profiles of engagement using LPA. These profiles represented different configurations of how youth were working hard, learning, enjoying themselves, and feeling challenged and competent at the time they were signaled as part of the ESM approach. Relations of the five aspects of work with data and youth characteristics (pre-program interest in STEM and youths’ gender and status in terms of being a member of under-represented groups in STEM) were, overall, not strongly related with the profiles of engagement, though some key findings were identified. Generating and modeling data were both related to the most potentially beneficial profile (full engagement), one characterized by high levels of all five of the engagement variables.
This study suggests that work with data and contemporary engagement theory as interpreted in this study can serve as a frame to understand what youth do in summer STEM programs. These findings also show the value of an innovative method, ESM, and an analytic approach designed to identify engagement holistically, LPA, that together to provide some access to youths’ experience in-the-moment of the activities they were involved in during the program. Data, and how youth and students in K-12 settings can themselves work with data, is an important, yet perhaps under-emphasized part of STEM learning. In the remainder of this section, I discuss key findings with respect to a) work with data, b) youths’ engagement, and c) what relates to youths’ engagement. Also, some limitations and recommendations for future research as well as implications for practice are identified and described.
Key findings related to work with data in summer STEM programs
Results showed that work with data was common in the summer STEM programs. There was variability in which aspects of work with data was present: Making observations, in some form, occurred during 24% of the program’s time, for example, while generating data and communicating findings both occurred more frequently, during 43% of the instructional episodes. These findings, broadly, suggest that work with data occurred enough that we might expect to see differences in youths’ engagement. They align with what may be expected given past research: Such programs are designed to engage youth in the practices, including and as I argued earlier especially those relating to work with data, of STEM domains (Dabney et al., 2012; Elam et al., 2012). Even still, these are the first results of this kind (in terms of the proportion of the time spent in the programs). Using video-recording data and a sampling strategy that can provide insight into the amount of overall time spent was an important component of achieving these findings. While there are no other results of this particular kind, a related, an area of related work concerns other studies that have used the PQA measure. Some research reports call for greater use of measures (such as the PQA) in the study (and evaluation) of summer and outside-of-school STEM programs (e.g., Yohalem et al., 2005). As one example of such a study that used the PQA (but one that is not focused on STEM), Smith et al. (2012) reported findings from a continuous improvement intervention, finding that the intervention positively impacted the quality of instruction in the programs.
In addition to work with data being common, I found it was highly varied in how it was enacted. In the course of the four-week summer STEM programs, youth engaged in what can be described as ambitious, specific, and potentially highly engaging ways of being involved in work with data. For example, when generating data, many times (in 47% of the episodes that involved this aspect) youth recorded their observations; when modeling data, youth were involved (in 72% of the episodes) in the use of statistical and mathematical models of real-world phenomena. When interpreting and communicating findings, youth regularly (during 48% of episodes) had opportunities to share (with other youth in the program) what they found or created as a result of their earlier investigations or work.
What occurred during the rest of the program’s time was also notable. When youths’ questions, for example, were not focused on predicting or hypothesizing about what they were exploring, the type of question was more general, or was instructor-led, rather than driven by youth. These instructor-driven forms of work with data were not aligned with recent reform efforts (i.e., National Research Council, 2012; NGSS Lead States, 2013; National Governors Association Center for Best Practices, Council of Chief State School Officers, 2010), but could be expected given past research pointing out variability in what evidence and data mean, especially in science education settings (McNeill and Berland, 2017; Lehrer & Schauble, 2015). Also of note was the frequency of these three aspects of work with data overall: They occurred much more frequently than the two (making observations and data modeling) for which a larger proportion of their enactment was more in-line with policy and curricular standards. The type of activities that may be the most demanding for youth was still common (and may spark youths’ engagement) but was not quite as common as the overall frequencies presented for the quantitative would suggest.
Past research does point out a heterogeneity in how work with data was enacted in education similar to that found in this study. For example, Hancock et al. highlight the use of “data to solve real problems and to ask authentic questions” ( p. 337). Research on generating data emphasized an aspect not very much the focus of the present research, namely, structuring data into spreadsheets (Konold, Finzer, & Kreetong, 2017; Lehrer & Kim, 2009). This suggests a reason why youth were able to ask questions and ideas for how they might do so more: Make activities in summer STEM program youth-centered, rather than instructor-centered. Research on the data modeling aspect of work with data highlights the use of statistical models much more than the physical models which were sometimes found to be a way in which youth engaged in data modeling (Petrosino, Lehrer, & Schauble, 2003; Lesh, Middleton, Caylor, & Gupta, 2008; Lee, Angotti, & Tarr, 2010). Nevertheless, many of the ways youth engaged in data modeling aligned with this past research, particularly when the goal of the activity was to model variability. This past research that encouraging youth to consider summaries of data, such as the mean and standard deviation, may be a promising way for them to engage more deeply in data modeling (Lehrer, Kim, & Schauble, 2007; Lehrer & Schauble, 2004). In this way, some (but not all) of the aspects work with data aligned with past research; when they align, they are encouraging, and when they do not, I present some ideas for how to involve youth in more engaging aspects of work with data.
Key findings related to engagement
Six profiles of engagement were found using a rigorous model selection approach. It is important to note that LPA is an exploratory approach: The number and nature of the profiles identified were found through a rigorous and systematic approach, but this is not a guarantee that the same number and make-up of profiles would emerge in other samples and other contexts: These profiles should be considered as initial evidence, and not as proof that these are the six profiles of engagement that will exist in all settings. The number of profiles found is broadly similar to that found in past research. Six is the same number of profiles of engagement identified in recent, past research and the similar number provides further information about the nature of engagement in educational contexts: Schmidt et al. (2018) found six profiles of engagement. Their profiles were constructed on the basis of the indicators (cognitive, behavioral, and affective) of engagement, and not perceptions of challenge and competence.
As Schmidt et al.‘s study is the only other to examine engagement profiles, another point of comparison is other outcomes that are different from but related to engagement, such as youths’ (and students’) achievement goals (see Wormington and Linnenbrink-Garcia, 2017, for a review in educational settings). Wormington and Linnenbrink-Garcia (2017) report that, usually, a smaller number, with only two of the 22 studies reviewed finding six profiles of goals. This suggests, on the basis of this study and Schmidt et al.’s (2018) study, that there may be a greater variety of types of engagement exhibited than, for example, types of achievement goals. However, in addition to the different construct, the engagement profiles were constructed on the basis of data collected via ESM, while the achievement goal profiles were constructed on the basis of self-report surveys and not via ESM. Knowing the number of profiles found when profiles are used to explore various constructs is potentially helpful, additional information about how engagement was experienced by youth. In addition, the greater number of groups may suggest that engagement, explored through ESM, may demonstrate more variability in terms of how its constituent parts are experienced together and at once. Exploring whether this greater variability was due to the method of data collection (ESM or self-report), the construct (engagement or achievement goals), or some other reason.
In terms of comparing the make-up of the specific profiles to other, past research, little work has examined profiles of engagement. Schmidt et al. (2018) did examine profiles of engagement, which were constructed from indicators cognitive, behavioral, and affective engagement (but not perceptions of challenge and competence, as in this study). Schmidt et al. (2018) found six profiles, some of which partially overlap with those found in the present study. In particular, on the basis of the items shared between the studies, a Universally low, All moderate, and Full profile were found in both studies. However, as these profiles are characterized by the (uniform) level across all of the variables, this is only limited evidence for the presence of these profiles in the larger population of youth engaged in science and STEM-related learning activities.
The six profiles lend insight into how youth engage during summer STEM programs. In particular, the Only behavioral, Only affective, and Engaged and competent but not challenged profiles were found in the present study, but not in Schmidt et al.’s (2018) study. Youth were highly engaged (as may be anticipated given the goals and design of such programs), but perceive a misalignment between their (high) competence and how (not very) challenged they were. According to past theory (e.g., Csikszentmihalyi, 1997) and some research (e.g., Shernoff et al., 2016), such a profile would be unexpected, as high levels of engagement are expected to be associated with high levels of both challenge and competence. In this study, a profile characterized by high competence but (very) low challenge was associated with very high engagement. This profile, Engaged and competent but not challenged, then, seems to suggest a type of engagement that may be unique and common to summer STEM programs. Perhaps such a profile may be expected given the lower stakes (compared to formal educational settings) of summer STEM programs (and other informal learning environments) and the degree of competence that youth–many of whom have chosen to attend the particular program (Beymer et al., 2018)–perceive during them.
In addition to suggesting a profile of engagement that is distinct to summer STEM program, this profile and the other two not found in past research have some implications for youth activity leaders. In particular, they suggest that lower challenge may not, as would be anticipated given theory and past research, be associated with lower engagement. Because of this, it may be recommended that activities that are not challenging but have other possible benefits to youth (i.e., benefits from activities designed to support youths’ social skills), can be integrated into programs, along with other, more challenging activities that are also highly engaging to youth.
These profiles have some implications for the study of engagement. They also have some implications for the analysis of multidimensional data on engagement. First, they suggest that perceptions of challenge and competence be considered in future research. This is because some of the profiles were distinguished on their basis. This approach also may be more parsimonious than including perceptions of challenge and competence as separate predictors (i.e., Shernoff et al., 2003). In addition to these empirical reasons, past research on engagement (i.e., Csikszentmihalyi, 1990) and on the profile approach (Bergman & Magnusson, 1997) suggest that they are theoretically inseparable from engagement, another reason for modeling them as they were modeled in the present study. These implications, then, are specific to the study of engagement but also highlight some of the potential of the profile approach, as well.
Key findings related to work with data and youth characteristics and their relations to engagement
In line with what the sources of variability would suggest, relations between work with data were minimal, though some small, statistically significant relations were identified. The question of whether and how work with data relates to engagement has not been the focus of past research on work with data. This past research has focused more on very specific cognitive outcomes, designs (often from design-based research) for work with data, and the challenges teachers and learners may anticipate when they are involved with particular aspects of work with data, particularly data modeling (and accounting for variability in data). Given the absence of research from an engagement perspective, these are new findings that suggest, in this context, that work with data may not be strongly related to engagement in educational contexts.
Why might these relations be so minimal? First, and foremost, the little variability at the instructional episode level was noteworthy because it means that few relations between variables at the instructional episode were expected. In particular, there were small ICCs at the instructional episode level for all six profiles. This suggests that there was very little systematic variability at the particular level that a variable for work with data was at. Additionally, the ICC values found in this study were smaller than those found in the one other past study that employed the same analytic approach (Strati et al., 2017). The relative absence of variability at the instructional episode level may be due to the summer STEM setting: Perhaps youth are less likely to engage differently from instructional episode to instructional episode (compared to in K-12 educational settings) because there is less variability in what took place across the episodes or because youth perceive there to be lower stakes for the programs’ activities and therefore do not perceive the changes in the instructional episode as a salient factor in terms of their engagement. This consideration is described in greater detail in the limitations section.
There are other possible reasons, though, too, for the minimal relations. One may be that work with data is not, as carried out in these summer STEM programs- very engaging, even accounting for the small amount of variability at the instructional episode level. The comparison between the five individual aspects of work with data and not working with data as well as the comparison of instructional episodes that involved any of the aspects of work with data and those that contained none showed minimal relations. This suggests that work with data is not more engaging than other activities carried out in summer STEM programs.
Another noteworthy possibility is that the novel analytic approach or the measures used also had impacts; but, again, the small variability at the instructional episode level is likely a greater factor than these, and a review of the correlations between the aspects of work with data and the variables used to create the profiles showed minimal relations. This and the previous potential explanation are explored even further in the next section, on limitations to the present study and recommendations for future research. Taken together, it seems that the major reason for limited relations between work with data and youth engagement is that youth simply did not engage very differently from instructional episode to instructional episode.
Even so, there were some noteworthy findings that could be anticipated on the basis of the importance of the two aspects of work with data that were found to relate positively to youths’ engagement. In particular, both generating and modeling data were found to be positively (and statistically significantly) related to the Full profile, suggesting that when youth were involved in these practices, then they were more likely to be highly engaged. In particular, given the makeup of this profile, this suggests that when youth were involved in these aspects of work with data, they were more likely to report high levels of cognitive, behavioral, and affective engagement, and high perceptions of competence and challenge. Generating and modeling data may have such relations because they were particularly important aspects of work with data. As Lehrer and Schauble (2006) explain, inscriptions serve commitments: Choosing to record an observation or an idea as data involves the process of identifying something that is worth recording and then recording the parts that are of interest. Thus, generating data may be fully engaging to youth because it is, generally, demanding and important with respect to work with data. Modeling, too, is an important practice. It has been described as the central scientific and engineering practice (Lehrer & Schauble, 2015; Weisberg, 2012), and its relations with full engagement provides some actionable evidence for its importance in the context of summer STEM programs. Modeling may be especially engaging to youth because such work positions learners as the creators of new information, in addition to using models created by others to learn about authoritative sources of information. This is one of the affordances of modeling in teaching and learning contexts (Berland et al., 2016; Schwarz et al., 2009). Moreover, when learners create new knowledge through activities such as modeling, they can begin to shape the process of constructing new knowledge in a domain, a challenge in science education contexts (Miller et al., 2016) and likely in other STEM content areas, we well.
The null findings for the relations of asking questions, making observations, and interpreting and communicating findings were noteworthy, too. They suggest that their effects were not large enough to be detected They may not be able to be detected for a number of reasons: simply because they were very small given all of the other factors that impact youths’ engagement, because the aspects of work with data were enacted in a myriad of ways which may be more or less engaging, and because they simply were not as engaging. Nevertheless, future research may seek to understand why these aspects did not relate to engagement.
As there is no research on how work with data relates to youths’ engagement, the findings associated with this research question provide some initial evidence for how some aspects of work with data relate to youths’ engagement. These findings suggest that these activities may not be more engaging per se. Instead, it may be the way that youth engage in them that matters, in alignment with past research (Berland et al., 2017). While the findings for this question were somewhat minimal, there are key findings from both the important relationships that were found to be statistically significant (between generating data and data modeling and Full engagement) and from those that were not. Other samples, other enactments of work with data, and, possibly, other analytic approaches can build on this work to further substantiate what is known about how work with data engages youth and other learners.
Why were there such limited findings in terms of relations between youths’ characteristics and youths’ engagement? There was a lot of variability in the profiles of engagement at the youth level, but there were not many relations in terms of youths’ gender, URM status, or pre-program interest. Given past theory and research have suggested that learners’ gender, URM status, and individual or pre-program interest can predict engagement (Bystydzienski, Eisenhart, & Bruning, 2015; Hidi & Renninger, 2006; Shernoff & Schmidt, 2008). Despite these surprising findings, youth with higher pre-program interest were found to be more likely to be Engaged and competent but not challenged. This suggests that youth with a higher interest in STEM were inclined to be highly engaged and good at what they were doing, but were not challenged by the activities they experience. This finding is in line with past research suggesting a relationship (direct or as a moderator) between youth characteristics (including interest) and their engagement (Shernoff et al., 2003; Shernoff et al., 2016; Strati et al., 2017). More specifically, this finding suggests that for youth who were particularly interested (and those who choose to attend) summer STEM programs, what they were involved in may not challenge them very highly. This finding has implications for past research that shows youth who choose to attend summer STEM programs were more engaged (but that does not speak to their degree of challenge; Beymer, Rosenberg, Schmidt, & Naftzger, 2018).
While the findings for this research question, like those for the relations between the aspects of work with data and youths’ engagement, they provide some information about how these characteristics relate to youths’ engagement. Knowing that youth who were more interested before the beginning of the summer STEM programs were more likely to be working hard, learning something new, and enjoying what they were learning, and perceive themselves to be good at what they were doing but not challenged, is novel. Moreover, the null findings suggest that other characteristics, including those measured but not included for this analysis (such as youths’ pre-program perceptions of their competence) as well as those not measured at all, may be considered in follow-up studies and future research. While the programs that were involved in the study have many affordances for work with data and for being highly engaging for youth, they have some limitations, too, particularly with respect to support work with data. Importantly, these were not programs explicitly designed to support work with data; while such contexts are being developed, they are not yet widespread. Moreover, youth may perceive the programs to have lower stakes in terms of their future. This may mean that the individual activities that youth engage in were less connected to their engagement: Youth instead engage in typical (to each youth) ways, rather than in ways that were much more sensitive to changes in their context. Another possible explanation for these limited findings may be that youth were not very challenged or were not very supported. A profile with low challenge but high competence and cognitive, behavioral, and affective engagement was found, suggesting that youth may be engaged and good at what they were doing, but were not challenged: Greater challenge may be found to be associated with more full engagement, for example.
Limitations to the present study and recommendations for research
To summarize the previous sections, work with data was frequent but varied in how it was enacted and profiles of engagement representing different and interpretable configurations of five engagement-related variables were found, but work with data and youths’ characteristics were not found to be very strongly related to any of the profiles. Some limitations to the study that may provide insight into why such minimal relations to the profiles were found and into other findings are detailed in this section.
First, the programs participating in this study were not designed especially to support youth in work with data. Instead, the programs were designed around best practices for summer STEM programs to support youth to engage in a wide variety of STEM-related practices–and in other activities, such as those intended to build a sense of camaraderie among the youth in the programs. In this study, aspects of work with data were identified and were found to be common, but some of the heterogeneity in the nature of working with data may be due to this reason: Planning and instruction for the programs did not aim to foster rich work with data any more than the other activities (STEM and otherwise) that made up their programming. In addition to the varied ways in which youth worked with data, some of the relations of the variables for the five aspects of work with data to youths’ engagement may be due to the ways that the variables for work with data indicated, in fact, many different ways of working with data. Some of these aspects of working with data, particularly those that were highly-specific with respect to how the data was involved and to how focused and sustained the work with data-related activity was, may be more engaging to youth than the others, such as those that were more general, instructor-focused, or brief. These two types of working with data were considered the same in the variables used to predict youths’ engagement. Future research can aim to understand youths’ engagement in outside-of-school data science programs and K-12 units, for example, that are focused more on work with data to understand better how work with data engages youth. Nevertheless, this study does provide insight into how work with data took place during model (i.e., designed around best practices for such programs) summer STEM programs and how such work relates to youths’ engagement.
In a related point, it is important to point out that while outside-of-school STEM programs have affordances, they also have some distinct features as well as some limitations. One feature is the substantial, but still limited period of time, which was around four weeks. Another feature concerns the nature and quality of the teaching and learning that is afforded. The contexts (including in field settings) in which youth were engaged could spark their engagement and could support work with data better than some K-12 classrooms. They also have limitations, such as the chance that youth considered their time in them to be fun and to be social, rather than educational, in nature. Of course, this is not unreasonable or unexpected on the part of youth, but it may mean that the ways that youth engaged in the programs as documented in this study could be unique to outside-of-school STEM programs. In particular, engagement as reflected in the engaged and competent but not challenged profile may be unique to the experiences of youth in summer STEM programs: It may not be common in K-12 classrooms. This limitation is in addition to and in the context of those documented in earlier parts of this section, particularly, that the limited variability at the instructional episode level may also be due to the lower stakes that learners in these contexts may perceive.
Learning environments that deliberately support work with data over an extended period may demonstrate different patterns of engagement. One key reason why this may be is the importance of work with data being part of a cycle (and how this cycle often did not take place in these outside-of-school STEM programs). Nevertheless, in addition to illustrating the nature and frequency of work with data, the open-ended, qualitative coding carried out for research question #1 also provided a lens into how work with data was (or was not) sequenced. There were instances of youth activity leaders linking earlier to later activities. For instance, the mathematics-focused programs, such as the Adventures in Mathematics program, the youth activity leaders, recognizing that youth had difficulty solving equations, used duct tape–and building on an earlier activity in which youth considered what constituted a rate–asked youth to count how many “hops” it would take someone to move from one end of a line of duct tape to the other. The youth activity leader than asked youth to consider how far they could move in one hop and to consider how they could find out many hops it would take, using a mathematical equation. In this activity, youth were supported in their attempts to approach mathematics problem-solving by linking data modeling to an earlier activity that involved generating data about the number of hops.
Other instructional episodes evidenced fewer connections between earlier and later activities and also the opportunity for more sustained involvement in work with data. For example, during some instructional episodes, youth-generated data, but they did not use the data they generated in subsequent activities. In the engineering-focused programs (Uptown Architecture, Crazy Machines, and Dorchester House particularly, youth often generated data that resulted from their engineering designs (and communicated and interpreted their findings,) but did not model this data as a regular part of their activities. In one particular example, in the Ecosphere program, youth collected water samples in the field. They then brought these samples to the classroom and tested the water, involving youth in both collecting and, to a degree, generating data (by noting the pH levels of the water). However, later in the day, youth created a small-scale model (with inclined trays of dirt, rocks, and plants) of an ecosystem, in which they added food coloring to determine the impacts of chemicals and acid rain. Youth then interpreted and discussed these findings, but did not connect the discussion to the water samples youth collected and tested earlier. While these specimens were collected to serve as data for future activity, there was no generating data observed during the episode. In other instances, youth were involved in observing phenomena but were not ever asked to use those data in subsequent activities. How this sequencing of work with data may impact youths’ engagement was not considered in this study, though past research suggests that this factor may make work with data more (or less) engaging and impactful to learners. As McNeill and Berland (2017) argue, it is not just engaging in these practices by rote, but about integrating them, as they overlap and interconnect. They argue that a view of work with data focused on “making sense of” data generated from real-world phenomena, as well as sustained engagement in work with data involving the revision of earlier, intermediate ideas, are important considerations regarding the enactment of work with data.
In addition to limitations related to the focus of the programs and how work with data was enacted as part of a cycle, there were also some general measurement-related limitations. Work with data can be difficult to measure because, as the qualitative analysis revealed, there were a variety of ways in which youth can be involved in work with data. McNeill and Berland (2017) describe a similar type of disagreement across science education settings: While a limitation, the coding frame did represent agreement across a range of studies across STEM contexts for the aspects of work with data. In terms of the alignment of the measure with the conceptual framework for work with data, the dimensions of the STEM-PQA measure aligned closely with the aspects of work with data. However, there were some divergences that may have had an impact upon some of the findings. For example, for the interpreting and communicating findings code, the STEM-PQA codes for Analyze (“Staff support youth in analyzing data to draw conclusions”) and Use symbols or models (“Staff support youth in conveying STEM concepts through symbols, models, or other nonverbal language”) were used. In the case of the latter STEM-PQA code, conveying STEM concepts through symbols, models, or other nonverbal language could have reflected instructional episodes in which youth used, for example, mathematical equations or formulas, but did not do so as part of modeling data of a phenomena in the world: They could have simply been using an equation outside of the context of any particular phenomena. Future research may consider the usefulness of coding for this aspect of work with data (and this aspect of science curricular standards in particular; see NGSS Lead States, 2013).
As another example of this limitation related to how work with data was measured, generating data was an aspect of work with data that the open-ended qualitative analysis revealed to be less associated with less systematic groups of practices, or themes, than the other aspects. The STEM-PQA codes corresponding to this aspect of work with data were Collect data or measure (“Staff support youth in collecting data or measuring”) and Highlight precision and accuracy (“Staff highlight value of precision and accuracy in measuring, observing, recording, or calculating”). Particularly in the case of the latter code, the emphasis on precision and accuracy may have been outside of activities focused on recording data or creating coding frames. Future research may consider a coding frame that is (more) focused on generating data, though considerations of precision and accuracy are key aspects of doing so, and so perhaps separating the act of generating data from considerations that are important to keep in mind while doing it may be a promising direction for future research. While these divergences in measures were not large, they suggest that the coding frame for work with data is a limitation of the present study.
It is possible that the somewhat minimal findings are, in part, a result of the analytic approach. A similar mixed effects modeling approach has only been used in one other study (Strati et al., 2017), and that approach did not use profiles (as in this study) as the outcome. In this study, little variability at the instructional episode level was found, and so minimal relations between factors at this (instructional episode) level and the profiles of engagement was expected. Might profiles, but not the variables used to create them, be less variable at the instructional episode level? One way to consider such an alternate explanation is to use the data used in this study as part of correlational analyses, other analyses that use the variables used to create profiles of engagement but do not use the profiles themselves. An analysis in this spirit was reflected in the correlations including the aspects of work with data (presented in Table 4.2). These indicated very modest relations with engagement, indicating that work with data and the individual variables used to create the profiles are not related. Because of this, it is not surprising that the (more complex) mixed effects models used to explore the relations between work with data and engagement showed minimal relations. Related to pursuing a different approach to the data analysis, other outcomes from working with data may also show different (and more strongly positive or negative) relations. Such outcomes may be at the instructional episode level, like engagement, or may be longer-term, like youths’ future goals and plans after the conclusion of programs.
Implications for Practice
A few implications for practice can be drawn from this study, though these are somewhat restricted given the minimal findings. First, generating data and modeling data, in particular, may be beneficial in terms of engaging youth. Youth activity leaders (in summer STEM and other STEM enrichment contexts) and teachers (in formal learning environments) can best include the beneficial practices of generating and modeling data not in isolation, but rather through involving youth and learners in complete cycles of an investigation. This aligns with both foundational and contemporary research on work with data in education (Berland et al., 2018; McNeill & Berland, 2017; Hancock et al., 1992; Lee & Wilkerson, 2018).
Another implication concerns how work with data was enacted. As found in this study, work with data (and even specific aspects of work with data, such as asking questions) does not involve activities that are enacted in a universal way. An instructor instead of youth interpreting and communicating findings, for example, or learners asking general, conceptual questions about work with data, as another, are different from youth working to interpret findings and figuring out how to ask a question that can be answered with data, respectively. This heterogeneity suggests to those involved in planning and enacting engaging activities that involve data to consider who works with data carefully, how they do so, and how much time and sustained focus is required for such activities to be carried out. This implication aligns with recent curricular reform efforts, some of which suggest that involving work in STEM-related practices is most effective when it involves learner-driven (but instructor-supported) iterative processes of identifying a question or problem, marshaling sources of data that can be used to figure out what is happening, and developing model-based explanations that are shared with the learning community (National Governors Association, 2013; National Research Council, 2012; NGSS Lead States, 2013). While just two implications, youth activity leaders and teachers and those designing data-rich activities and evaluating the impacts of instruction based on such activities can use the findings from this study as a starting point to consider how engaging in work with data may also prepare learners to think of, understand, and take action based on data in education and in other areas of their lives.