Efficacy Portfolio
Short Answer is committed to measuring and sharing our product’s efficacy. The portfolio below reflects our efforts to continually understand how, why, and if our product is improving learning. We have chosen a portfolio approach because we believe it best captures Short Answer’s efficacy as our product continues to grow and develop. We are interested in understanding who our product works for, in what contexts, and why. The resources below chronologically detail our efforts to understand these questions. You can click on the icon to the left of each portfolio item. Taken together, we believe this collection qualifies Short Answer as containing ESSA Tier 4 Evidence of efficacy.
1. Theory of Change
Short Answer grew from our cofounder’s masters capstone work in the Learning Design and Technology (LDT) program at The Stanford Graduate School of Education during the 2021-2022 school year. It was in LDT that Short Answer first developed our theory of change. Our theory of change outlines how and why the specific interventions used within our platform are expected to lead to desired learning outcomes in an educational setting, including the research supporting those interventions.
2. Initial Learner Study
In the late summer and fall of the 2022-2023 school year, Short Answer conducted an initial survey and pilot with 20 teachers across the United States. This learner study included interviews with teachers and surveys of teachers and students. Amongst other findings, 81.9% of students surveyed indicated that Short Answer had a positive impact on their understanding of content.
3. 2023-2024 Educator Experience Survey
This case study was our effort to understand how users experienced Short Answer over the course of the ‘23-’24 school year in the United States. A mixed-methods approach combined user interviews, net promoter score (NPS) surveys, and an end-of-year educator experience survey to assess Short Answer’s effectiveness. Key findings revealed that Short Answer fosters critical thinking and engagement and has had a positive reception from educators, evidenced by an average NPS of 82. Interface issues and time constraints were also noted. The study concludes with recommendations for further development of Short Answer.
