Assessment For Good, a R&D program with the Advanced Education Research and Development Fund (AERDF), creates rigorous assessments that provide a comprehensive picture into skills that power learning. While most assessments focus on subject mastery, educators are often missing information about foundational skills to understand the next best step to support the learning process. Even if thoughtful educators want to understand student learning, existing assessments are limited in terms of portraying personalized and contextualized views of learning beyond whether or not a state standard was reached. AFG is focused on how to more effectively measure over 30 skills that power learning, such as perspective taking, initiation, or metacognition, which are key parts of the learning equation, but are often missed in the information we provide to educators for instructional decision-making.
At AFG, inclusive research and development (R&D) is at the core of how we build. At this year’s AI show at ASU/GSV, we shared a case study highlighting how AFG has scaled its inclusive R&D approach through a strategic and intentional integration of AI, while keeping community voice at the center. Our goal was to honor educators and learners as the expert—ensuring a continuous community loop throughout the process to center subject matter expertise and community insights as part of our R&D process.
Our assessment development process starts and ends with the learners we serve.
From the start, every assessment item or question has been shaped by real feedback from learners. From 2022 until very recently, nearly 1000 assessment items were co-created with learners to ensure they reflect learners’ contexts and reflect the assets they bring. This feedback loop ensures our assessments reflect not just academic standards, but learners’ lived experiences.
In 2024, we built a capability that explored the use of generative AI as a tool to inform assessment development. Instead of thinking about this as a way to replace the human-in-the-loop, we thought about how to leverage an inclusive process that expanded how learner, educator, and research expertise contribute to assessment development, while reducing the resource burden of measure development, which is felt across the assessment industry.
What is different about this process?
We wanted to include the learner and educator voice at the beginning, middle, and end. Our training set of items was rooted in AFG’s scientific framework and that was carried through to subject matter expert review, in which AI generated assessment items were accepted, rejected, or revised for inclusion. Overall, the AI generated items had an acceptance rate of 82% for alignment to our scientific frameworks. Lastly, we tested our AI generated items alongside our human-generated items with students and found no differences in item performance. In six months, we’d been able to increase our bank of assessment items by 167%, far faster than if we had continued to generate them all by hand. This technical capability, born out of our inclusive R&D efforts, can now be deployed within AFG’s other assessment products to personalize assessments at scale or licensed to other assessment creators.
Our AI strategy is to support and scale human-centered processes, not replace them.
By anchoring AI tools in trusted knowledge bases, including content informed by learners and educators, we can streamline content creation while maintaining high ecological validity—the alignment between what’s being assessed and the real world learners inhabit. We believe that more assessment development processes should be broadened to include a greater variety of experts in the process, including learners, caregivers, and educators; the key to better R&D is to continue to center their voice.
Inclusive R&D isn’t about completing a checklist—it’s a smarter approach to development. By honoring human insight and strategically using AI to support, not replace, community-driven expertise, we’re redefining what it means to scale assessment innovation.
