Assessing student writing with ai
We are currently exploring a variety of options for assessing student writing using ai tools. We have explored AI Grader and found that whilst the feedback was well targeted and constructive, the scores (marked against IPEEL criteria) were considerably higher than teachers were awarding. We have also asked CoPilot to assess against the same marking grids and the results were generally within one mark of the teacher assessment. We are a specialist setting with small classes so volume of work isn't an issue when it comes to marking but moderation, compared with national is always on our minds. What are others using? Any thoughts or suggestions welcomed.
17 replies
My own leaning would be to use AI in a few limited ways. Copilot as a rubric-aligned feedback assistant, as a moderation partner to compare with teacher judgement, and as a way to identify class-wide trends in writing strengths and gaps. I’d be much more hesitant about using it to generate the final score without a strong local moderation process around it.One practical approach might be to...
How odd, here's the direct link: https://substack.com/@rangaedu/p-159183311
Prior to using MagicSchool.AI with teacher candidates at college level, our IT department arranged for a security expert review of the policies of the site. This is a long post, but it is informative about the security of the site: MagicSchool has established strict policies to protect student data privacy, specifically aiming to provide a safe environment for K-12 settings. Key facts about...
Thanks. That responsible use of AI guide would be especially useful for any teacher involved in long-term student research and writing projects.
Thanks for your comments, our purpose for using AI is wider moderation, we are a small specialist school so have very small samples- we are not looking to replace the human in the marking but support our judgements on a wider basis.
Our division has Magic School, which teachers can use as an assistant for assessing student writing, however, teachers are still expected to add their own input and adjust the AI review, if needed.Our state uses AI for standardized writing prompts (one AI, one person per essay). Thus, we want students to see how AI focuses on grading, as well as what humans focus on when grading.
I err on the side of caution with AI grading due to a lack of consistency. See my substack article here.What I have been more impressed with is the comparative judgement approach used by No More Marking (Daisy Christodoulou) which has shown that in this use case the AI is often more accurate than humans, and the biggest glaring errors are more often made by the human, not AI.
Really interesting discussion folks.In higher education, using AI to assist with grading or to grade is not permitted because the student's work is their own and and as holders of that intellectual property, having their work then be used to further train an LLM without their express consent is very problematic.I would be interested to find out if this is topic of conversation in K-12?Thanks...
Hi - have you tried Brisk? On the free version you can do plenty such as 'Glow and Grow' rubric criteria and next steps - you just need to add the extension to your Google Chrome browserhttps://www.briskteaching.com/give-feedback
I am an ELA teacher and I have used Khanmigo Writing Coach for composing student drafts and have action research under peer review regarding it that I would be willing to share with you. One thing that I would highly recommend considering: what is the purpose for using an AI grader for student writing, and does it truly help your students? I fear this could be a slippery slope when it comes to...