Recommendation #1: Employ diverse, multi-layered assessment approaches to evaluate student learning
Global Context
In November 2022, OpenAI’s large language model (LLM) ChatGPT has gained global recognition, as its public release came just in time for final examinations at universities and colleges in many parts of the world.1 One of the biggest concerns then, and now, is how LLMs are trained to simulate human expressions based on probability models. For students who waited until the night before the final paper was due, LLM might have provided a welcoming last resort to generate an essay in the blink of an eye. For instructors, this meant take-home final papers can no longer be the sole method of assessment to measure how much students have learned over the course of a term. In response, Harvard University and many others in higher education have prepared policies on generative AI tools in the classroom, requesting faculty to define what type of usage is appropriate for a given course.2
A screenshot image of the AI grader used in DTI & iDeA (3.007) for journey mapping.
SUTD Examples
At SUTD, course assignments vary across pillars and clusters, ranging from logging of progress in studio tasks, to printing 3D models to iteratively test the validity of ideas, to conducting user interviews with industry partners. In courses like 3.007 “Design Thinking and Innovation,” Sumbul Khan (SMT) and Bradley Camburn (EPD) use a range of assessment approaches to balance two goals: building students’ critical familiarity with existing generative AI tools—such as an AI-powered chatbot designed by researchers to aid learning—and ensuring students ultimately acquire the necessary concepts, methodologies, and communication skills independent of AI. To achieve these goals, their assessment framework includes multiple checkpoints such as gallery-style presentations (akin to vivas) and in-class assignments. The final assessment requires students to produce multi-modal artefacts, including videos, posters, CAD models, oral presentations, and functional physical prototypes. The use of diverse approaches allows them to ensure students have engaged with the content in various ways, and that human orchestration and synthesis remain essential to successful completion of the project.