TBR AI EXCHANGE

AI Learning Collaborative

Comparing AI and Human Writing: A Micro-Activity Identifying Limitations in AI‑Generated Abstracts

Submission Date

Submitter’s Name/Email

Institution/School

Department/Discipline

Activity Purpose (assessment, data collection, classroom management, etc.)

AI Tool(s)

Activity Details

Context: For a session at Summer Institute at Southwest TN Community College, the Course Design Support Coordinator and Director of Assessment collaborated with TBR’s AI Learning Collab Team to facilitate an impactful session on redesigning an assessment applying the Student AI Literacy Competency Framework. An intentional selection of course assessments served as the redesign component for participants. The assessment asked “students” to critically evaluate AI‑generated academic writing. Since many students often accept AI outputs at face value, without recognizing limitations such as incomplete information, vague language, or hallucinated content, this activity was designed to address this gap by guiding students to compare an AI‑generated abstract with an instructor‑provided model abstract for the same research article. Application: As a low-stakes, scaffolding micro-activity, AI was used to generate a sample abstract based on a scholarly source. A common practice of students is to ask AI to write an abstract for the article, providing the source link or name of the article. The model that was generated included common issues such as inaccuracies in the methods sections, vague information, and generalized claims. Students (aka our workshop participants) engaged in a short micro-activity where they used a checklist to evaluate the AI‑generated abstract as it compared to the the model abstract, identified inaccuracies, missing elements, and vague language, and reflected on strengths, weaknesses, and necessary revisions in their discussions. Outcomes (from the Student AI Literacy Framework): Describe common limitations of AI systems, such as incomplete information and hallucinated content Compare tasks completed by humans with those supported by AI systems Evaluate AI generated content with instructor provided sample: The activity allowed for transferable academic skills, commonly asked of students enrolled in courses across varied disciplines, of summarizing/analyzing scholarly articles and evaluating credibility,