Over the past five years, EF+Math has shown how interdisciplinary teams can co-create innovative, equity-centered math learning experiences that support students’ executive function skills, affirm their identities, and improve outcomes. We’ve built products and tools, generated scientific knowledge, and cultivated a growing community that believes deeply in student brilliance. As EF+Math is in its final program year, we are reflecting on our journey so far and what we’ve learned about what truly impacts student learning.
We have continued to advocate for a more expansive notion of valuable evidence of impact—one that views students holistically as developing individuals, recognizes learning as part of human development, and acknowledges the importance of shifting achievement outcomes. We use this expanded definition to drive how we measure the impact of each curricular product in the EF+Math Program portfolio and inform continued improvement through iterative, inclusive R&D efforts.
What Does It Look Like to Have Better Math Learning Experiences for Students?
The EF+Math Program believes that all students are brilliant and have the capacity to succeed in mathematics. Our goal is for all students, particularly Black and Latino students, to have rigorous and equitable mathematics learning experiences throughout their educational journey. As a program, we hypothesize that we can achieve this goal by integrating EF skill development into math learning in ways that support conceptual understanding, complex problem solving, and equitable classroom experiences. Furthermore, these types of equitable learning experiences can enhance math learning outcomes, narrow gaps in math learning, and foster students’ self-perception, enabling them to engage in higher levels of mathematical activity.
As our work progresses, we’ve gotten clearer about what it means to create these types of learning experiences. Below, we outline the three things that are essential to claim that math learning experiences are better. It is critical to see improvement across all three of these aspects to transform math learning experiences:
- Improved Math Learning Outcomes: Our products aim to improve mathematics learning outcomes, including those measured through standardized tests and class grades, together with other more localized, proximal assessments. We know that it is not enough to provide students with resources that support learning (such as access to high-quality curriculum) unless these resources actually lead to tangible results.
- Improved Math Perceptions: Student perceptions of themselves as math learners have long-term impacts on their ability to persevere and achieve in mathematics. We believe that the ability for a curriculum product to improve students’ sense of belonging and the extent to which they see themselves as mathematicians is an essential component of evidence that must be demonstrated for that product to be seen as successful.
Increased Engagement in Mathematical Activity: We know that mathematics learning is not just the demonstration of content understanding, but also the development of skills (or practices) that students use when doing mathematics. Enhancing students’ ability to engage in mathematical activity is a key component of learning that enables them to tackle more complex mathematical content throughout their learning trajectory.
Together, these three types of outcomes constitute what EF+Math targets as “better math learning experiences for children.” Our teams developed approaches that align with this definition by taking a holistic approach to target the different areas of impact at the intersection of math, EF skills, and equity. This work involves efforts to shift the paradigm where traditionally executive function interventions are not embedded within students’ math learning experiences, do not prioritize affirming students’ identities and brilliance, and cannot easily be utilized by educators in real classrooms. To shift this paradigm, we should measure how products achieve these three areas of improvement when evaluating their impact in real-life classrooms.
In early phases of our work (Phases 1-3), the R&D project teams worked closely with teachers and students to co-develop prototypes. They conducted small pilot studies, along with testing the usability and feasibility of the developed curricula. After three years of iteration, we advanced three R&D project teams that had shown promising initial evidence of impact.
In Phase 4, the MathicSTEAM team delved deeper into the iterative development of their platform, conducting essential back-end technology updates and usability and feasibility testing in preparation for a Phase 5 study. For the Fraction Ball and CueThinkEF+ teams, the EF+Math Program commissioned independent, mid-scale evaluation studies through a partnership with American Institutes for Research to test the most updated versions of their learning approaches and gather more evidence of their impact on student learning experiences. The study of CueThinkEF+ trialed an innovative study design (a Sequential, Multiple, Randomized, Assigned Trial (SMART design), which tested how students who experienced different combinations of features in the CueThinkEF+ platform performed in relation to each other.
Insights Across Our Impacts on Student Learning
Over the last several months, this blog series profiled each of EF+Math’s three advanced R&D teams. Each blog provided more detail on each of their approaches and situated their work as part of the collective EF+Math Program portfolio and core hypothesis:
You can explore each of the team’s profiles here:
- Unlocking the Power of Math Through Inclusive Design and Executive Function Training: The MathicSTEAM Story
- Empowering Learners, Transforming Math Problem-Solving: Inside The CueThinkEF+ & Pennesota Project
- Bringing Joy to Fraction Learning: How Fraction Ball is Transforming Math Education
Evidence of Improved Math Learning Outcomes
As the blogs detail, each of the three advanced R&D teams targets specific mathematics content, integrates EF skill development in math learning, and seeks overall improvement in standardized outcomes such as state assessments and class grades. In Phases 1-3, teams primarily measured student learning growth on proximal assessments aligned with the mathematical concepts and skills at the focus of their interventions. Fraction Ball saw increases in student performance on measures of rational number understanding (e.g., Begolli et al., 2024). CueThinkEF+ focused on the development of students’ problem-solving skills by leveraging certain EF skills which are used to tackle grade-level problems with open-ended solutions, and saw growth in students’ ability to solve such problems correctly (Rhodes, et al., in press).
MathicSTEAM found that students who played games on their platform improved in both their executive function skill performance and math fact fluency accuracy (Feng et al., 2022).
In Phase 4, the MathicSTEAM team revised their game designs to more effectively sequence which games students would experience based on their past game performance, providing targeted and scaffolded opportunities for students to work on the content with which they were struggling. The evaluation studies of Fraction Ball and CueThinkEF+ gathered evidence of impact on student math learning outcomes. The two teams demonstrated a mix of positive impacts, with Fraction Ball showing improvements on various rational number knowledge assessments, and CueThinkEF+ showing small, but non-significant, effects on state achievement scores. Furthermore, the mid-scale evaluations provided new types of evidence and analyses, such as correlational analyses between impact outcomes and “dosage” metrics, which capture how much students used the products. Such analyses provide valuable insights to the teams on potential areas for iteration in their products that may impact learning outcomes, and highlight the importance of cyclic R&D processes.
Evidence of Improved Math Perceptions Outcomes
Our three advanced R&D teams intentionally designed their products to ensure they foster an increased sense of belonging and mathematical identity for students, with early studies showing promising results. In Phases 1-3, team efforts were focused on co-designing these product features and doing initial testing. Teams built in representation of diverse characters engaged in real-world mathematics, designed prompts and reflection questions for students to consider their own learning pathways, and emphasized positive emotions associated with play, joy, and creativity. In addition, some teams conducted more formalized assessments of students’ math perceptions and affect, seeing positive, promising impacts on student learning (e.g. Guo, et al., 2024; Grose, et al., 2023).
We more systematically investigated these trends as part of our mid-scale evaluation studies through the development and use of measures of students’ math-related sense of belonging, perceptions, identity, self-efficacy, and other related factors. Quantitative analyses showed small, positive improvements on multiple constructs, and provided systematic evidence that student characteristics, such as grade level and free or reduced lunch status (a proxy for socio-economic status), significantly moderate students’ perceptions and beliefs in math learning contexts. In addition to these findings, the development of these measures demonstrates a novel contribution to increasing the field’s ability to assess how well they are supporting or improving students’ positive learning experiences. The MathicSTEAM team also engaged in rounds of co-design with teachers and students to add features to their product, such as mini-lessons that provide the opportunity for classes to explicitly discuss how the platform’s games connect to and support the development of their agency and identity as math learners.
Evidence of Increases in Student Engagement in Authentic and Rigorous Mathematical Activity
Finally, the Fraction Ball, CueThinkEF+, and MathicSTEAM teams are prioritizing the ways learning shows up in not only student performance on assessments, but also their engagement in mathematical activity within the classroom. We know that learning is an active process, and our teams have created learning approaches that change the ways students think and do mathematics in classrooms every day (e.g. Alvarez-Vargas, et al., 2024). In early phases, teams leveraged co-design sessions with students to create their initial product features that invite rich mathematical thinking, such as CueThinkEF+’s “Thinklets” or the MathFluency+ games. They continued to iterate and revise their lessons and games to ensure that students could access the mathematics and that there were multiple pathways for engaging in learning through choice, goal setting, and collaboration.
The mid-scale evaluation studies conducted in Phase 4 provided a rich corpus of evidence on how students participate in mathematical activity, from their own perspectives as well as those of their teachers. Teachers described how students increased their ability to communicate their thinking using mathematical language, as well as to effectively plan strategies for tackling problems. Students reported that their understanding of mathematical concepts improved as a result of engaging in the games and platform stages of Fraction Ball and CueThinkEF+, respectively. Overall, the studies contributed new forms of evidence through focus groups, logs, and surveys, providing more visibility into how students engage in transformative mathematics within these products.
Building a Portfolio of Evidence through Research, Development, & Evaluation: The Value of Mid-Scale Evaluation
In addition to gathering additional evidence of student learning impacts, conducting mid-scale evaluation studies in Phase 4 of our program generated valuable insights around study design methods and the role of evaluation in research and development programs. Our mid-scale evaluations provided an opportunity to investigate implementation conditions and participant perspectives regarding the latest iteration of the learning approaches, while also systematically examining the impact of these approaches on student learning. We:
- leveraged unique approaches to the design of research studies for each product to ensure our studies were responsive to both product iteration and real-world classroom contexts, including a Sequential, Multiple Assignment Randomized Trials design not commonly used in education (Williams, et al., in press);
- gathered more insight into all three focal elements of impact, especially in contributing evidence of change in math perception outcomes and student engagement, from an independent team;
- deepened our understanding of the mechanisms within each product that are driving learning through preliminary analyses, setting the stage for our future analysis; and
- expanded our inclusive research and development processes and practices to address evaluation goals, while finding ways to intentionally engage educators and students in co-design and co-research at a larger scale than previous phases.
Conducting mid-scale evaluation studies at this juncture of the EF+Math Program trajectory was essential to develop these learnings. We were only able to conduct these studies because of the processes and evidence developed through Phases 1 – 3. The learnings from Phase 4 set the foundation for even larger studies of impact conducted in Phase 5. For more detailed results from our evaluation studies conducted during the 2023-2024 academic year, find our study reports at the EF+Math Resources page, and stay tuned for forthcoming results from our large-scale evaluations conducted in 2025.
Looking Forward
To truly transform math education and provide the types of math learning experiences that all students deserve, we must simultaneously attend to all three elements of our expanded definition of impactful math learning experiences: ones that result in improved math learning outcomes, improved math perceptions, and increased engagement in mathematical activity. Our evaluation studies, combined with continued iterative and inclusive R&D efforts in Phase 4, demonstrate the evolution of our evidence base in each of these areas, providing invaluable insights and guiding iterative refinement toward greater effectiveness and equity for the students we aim to serve. We continue to learn about how the unique ways our teams have designed their math learning approaches are having an impact on students, and we continue to learn how the integration of EF skill development in math learning is positively affecting students.
If we want every student to see themselves as capable mathematicians, we must redefine what success looks like, ensuring that every child has access not only to rigorous math learning but also to experiences that affirm their identity and unlock their potential through strengthened executive function skills. This is EF+Math’s commitment: to create a portfolio of approaches that combine high-quality math learning with opportunities for EF skill-building and mathematical identity development, recognizing that these core capacities are the foundation for mathematical agency and long-term success.
