Recommendation #1: Employ diverse, multi-layered assessment approaches to evaluate student learning
Global Context
In November 2022, OpenAI’s large language model (LLM) ChatGPT has gained global recognition, as its public release came just in time for final examinations at universities and colleges in many parts of the world. One of the biggest concerns then, and now, is how LLMs are trained to simulate human expressions based on probability models. For students who waited until the night before the final paper was due, LLM might have provided a welcoming last resort to generate an essay in the blink of an eye. For instructors, this meant take-home final papers can no longer be the sole method of assessment to measure how much students have learned over the course of a term. In response, Harvard University and many others in higher education have prepared policies on generative AI tools in the classroom, requesting faculty to define what type of usage is appropriate for a given course.
SUTD Examples
At SUTD, course assignments vary across pillars and clusters, ranging from logging of progress in studio tasks, to printing 3D models to iteratively test the validity of ideas, to conducting user interviews with industry partners. In courses like 3.007 “Design Thinking and Innovation” (DTI), Sumbul Khan (SMT) and Bradley Camburn (EPD) use a range of assessment approaches during the term, to balance students’ gaining critical familiarity with existing generative AI tools—such as an AI-powered chatbot designed by instructors and researchers to aid their learning—and their ultimate acquisition of necessary concepts, methodologies, and communication skills independent of AI. The assessment framework includes a range of assessment checkpoints, e.g., gallery-style presentations (akin to viva) and in-class assignments. The final assessment includes multi-modal artefacts such as videos, posters, CAD models, oral presentations, and functional, physical prototypes that require human orchestration for synthesis.
Image 1: A screenshot image of the AI grader used in DTI & iDeA (3.007) for journey mapping.