I have been thinking a lot lately about AI-assisted writing and reading as well as what human-AI collaborations mean as we teach first-year writing in diverse institutional contexts. I always approach AI conversations with a note of epistemic humility: I am not as an AI expert, but I am a writing professor and a specialist in languaging, multimodal content creation, and rhetorical practices. My professional passion centers on teaching students how to articulate their thoughts clearly and compellingly. It is precisely this grounding in thoughtful crafting and critical rhetorical engagement that led me to be deeply intrigued by recent findings on the transformative potential of AI as an instructional assistant.
When I say recent – I mean it. Published this month by Pratyusha Maiti and Ashok Goel of Georgia Tech’s Design Intelligence Lab in the Proceedings for the International Conference on Intelligent User Interfaces, the paper I am so intrigued with explores the impact of conversational AI, specifically Georgia Tech’s AI instructional partner named Jill Watson. The Tech research team integrated Jill Watson into classroom Learning Management Systems (LMS’s), including into those of first-year writing courses at Wiregrass and Southern Regional technical colleges. Over three semesters and across different educational settings, the researchers collected more than 5,500 student-generated questions posed to Jill Watson. These questions were analyzed using a fine-tuned BERT model based on Bloom's Taxonomy to categorize their complexity.
Their findings were striking:
- Students consistently posed a significantly higher proportion of higher-order questions (such as analysis, evaluation, and synthesis) compared to traditional classroom norms.
- Sustained interactions with Jill Watson resulted in increased cognitive complexity of student inquiries, indicating growth in students' confidence and initiative in engaging with course material.
- The effectiveness of Jill Watson was notably evident in English Composition and Rhetoric courses, where assignments included writing tasks, journal reflections, and topic research—activities intrinsically suited to eliciting higher-order questioning.
Three particularly compelling insights emerged for me. I hope they resonate with you:
- Higher-Level Engagement:
"These findings demonstrate that Jill empowers learners to engage in critical questioning, thereby enhancing their educational experience by promoting depth, relevance, and application of course concepts" (Maiti & Goel, 2025, p. 314).
- Facilitation of Critical Thinking:
"Conversational AI partners like Jill have the potential to foster critical thinking by encouraging students to engage with more complex concepts through active, self-driven knowledge construction" (Maiti & Goel, 2025, p. 321).
- Personalized Learning Trajectories:
"Students who interacted with the AI tool more frequently exhibited a trend of increasing higher-order questions over time. This trend was more pronounced among individual students than at the overall classroom level" (Maiti & Goel, 2025, p. 321).
The effectiveness of Jill Watson in fostering critical inquiry aligns closely with my pedagogical objectives. As someone interested in how intentional and mindful interactions with AI might enhance students’ metacognition, I find that the principles of the Rhetorical Prompting Method (RPM)—a method encouraging purposeful, iterative, and strategic human-AI collaboration—can potentially complement the strengths demonstrated by Jill Watson. RPM helps writers thoughtfully craft inputs and revise outputs by considering their intent, context, and desired outcomes, which might similarly foster critical and reflective engagement as well as metacognition. Stay tuned for a Kennesaw State case study results on that last one. We are analyzing some interesting data.
RPM encourages learners to:
- Reflect deeply on their writing objectives.
- Strategically shape their inputs and output revisions.
- Read and synthesize outputs.
- Critically evaluate and refine AI outputs.
- Be the human-at-the-helm of AI collaborations.
Exploring methods like the rhetorical prompting alongside AI-driven learning environments could amplify the benefits observed with Jill Watson, guiding students towards sophisticated critical engagement that supports writing and learning processes.
Together, these complementary approaches have the potential to make AI interactions more intentional, reflective, and impactful—transforming conversations with AI into genuine opportunities for intellectual growth and skill development.
I'm curious—are there other methods or pedagogical models you've found effective in first-year writing that might pair well with tools like Jill Watson? I'd love to hear your thoughts and experiences.
As always, I encourage you to take my work and models. My AI work is an OER for all of us.
References
Maiti, Pratyusha, and Ashok Goel. "Can an AI Partner Empower Learners to Ask Critical Questions?" Proceedings of the 30th International Conference on Intelligent User Interfaces, 2025, pp. 314-324, doi:10.1145/3708359.3712134.
