An Inflection Point for K12 Writing
K12 writing instruction is at an important inflection point. New AI tools like ChatGPT have turned long-held practices upside down. Teachers, especially those teaching ELA classes, are caught between wanting to teach their students to use these new tools effectively while also being required to measure what students can write independently of AI.
In these early days of our newly AI-infused education system, many have turned to AI detectors as a barrier to AI’s influence on student writing processes. Recent research shows this may be a faulty approach. AI detectors simply don’t work….and may never.
Evidence Against AI Detectors
1. OpenAI, the creators of ChatGPT, recently admitted that AI detectors do not work in their new guide for educators. In their words, “While some have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.”
2. Peer-reviewed research from the computer science department at the University of Maryland found that all current AI detection methods used by leading plagiarism detection software are vulnerable to simple paraphrasing. The researchers used a paraphraser tool to rephrase text that had previously been generated through AI tools like ChatGPT, sentence by sentence. The results? AI Plagiarism detection accuracy dropped by as much as 75%. The researchers were also able to write human text that tricked AI detection tools into thinking the text had been produced by AI. Most interesting to consider as we weigh what this means for the future of education, the authors also argue for a theoretical impossibility of AI detection. Any detector that could distinguish between AI-generated text and human-generated text, they argue, would also necessarily be able to be used to generate AI text that could evade detection by the same detector. This result holds even when the AI text is generated with a watermark or when the detector is trained on a large dataset of human-generated text. Similar results were found in a study out of the the University of Applied Sciences in Berlin.
3. Researchers at Stanford recently demonstrated GPT detector bias against non-native English speakers. Vanderbilt University also recently advised against using AI detectors because of their high likelihood of false positives and the outsize harm this holds for non-native English speakers. These types of negatives are already evidenced by the myriad of viral stories of students who have been falsely accused of using AI (here and here).
4. Increasingly, AI detection companies are being less than truthful, misrepresenting some of the studies listed above as actually supporting their software.
Writing Instruction Moving Forward
It is our opinion that schools planning to use AI detectors are signing up for an endless game of whack-a-mole that will waste time and resources and, more importantly, run a high risk of negatively impacting the educational experience of students. We believe more holistic curriculum and instruction approaches are needed to address the new challenges and opportunities presented by AI writing tools like ChatGPT. For us, this looks like:
1. Bringing at least 50% of writing instruction and assessment into the classroom. In-class writing allows for much more control over how much influence AI has in the writing process (in the same way that in-class math assessment allows for better control over calculator usage). It’s why we added copy/paste controls for teachers to Short Answer and it will inform much of our product roadmap moving forward. It is our goal to support teachers who are working to bring writing instruction and assessment back into the classroom in age of AI.
2. Building in a reflective element to any writing assessment conducted outside of the classroom. Not only does this aid in personal growth and encourage critical thinking, but it also requires students to delve into personal feelings, experiences, and perceptions that are difficult for AI to replicate easily. It is also difficult to plagiarize because a personal reflection is so inherently individualized. If students are using AI tools, it will also empower them to reflect on how much of an influence AI had in their thinking and writing processes.
3. Discussing the “why” with students to develop motivation for writing. As John Hattie, Dylan Wiliam, and Arron Hamilton’s working paper on AI discusses, one of the most profound early impacts that AI will have in our classrooms is to decrease motivation. This is especially true as it pertains to writing. “Why should I learn how to write when AI can do it for me, better, and faster?” will be a question teachers will have to answer in the coming years. The simple answer here is that writing is almost inseparable from thinking. It teaches us to organize and structure our thoughts in a way that improves our ability to process and retain information. It develops metacognitive and self-reflection skills. It builds and reflects identity. It is foundational to literacy. All of these are self-evident to us as teachers. They may not be to students raised in a world where AI can write anything for them in seconds. With that in mind, it will be imperative that the “why” of writing instruction be embedded across the curricula.
4. Teaching students how to use AI effectively. The other 50% of writing instruction and assessment that is happening outside of the classroom will need to include explicit instruction on how to use AI effectively: Writing effective prompts, reorganizing and paraphrasing the generic writing produced by AI, inserting personal voice into AI-generated writing, and many more skills yet unknown.
5. Setting clear classroom management policies around AI. Moving forward, all assignments (but especially writing-focused assignments) should be labeled in advance as to whether or not AI can be used to complete them. This simple “green, yellow, red” framework from teacher Adrian Cotterell is a great place to start. For long-form writing instruction, requiring that students construct the paper in one Google Doc with the revision history turned on and a tool like Draftback enabled will also help. It certainly won’t stop students from using AI-generated writing structure and content on another screen, and then paraphrasing that content. But if paired with, say, the personal reflection element of point 2 above, it will still go a long way in promoting the deep thinking and thought organization that makes writing so powerful for learning.
At Short Answer, we plan to support all 5 of the initiatives above through professional development services and our digital writing tool. You can read more about case studies of that professional development work here and read more about why we feel our digital writing tool is well suited to develop K12 writers in the age of AI. We’d love to support your work, so please reach out.
