To Be Announced
1:00pm CDT, Center for Human-Computer Interaction + Design (Frances Searle Building 1-122)
Speaker: Prof. Elena Glassman, Harvard University
To Be Announced
1:00pm CDT, Center for Human-Computer Interaction + Design (Frances Searle Building 1-122)
Speaker: Prof. Sarah Fox, Carnegie Mellon University
Redefining the Image Captioning Challenge: From Truthful to Useful
1:00pm CDT, Center for Human-Computer Interaction + Design (Frances Searle Building 1-122)
Visual media increasingly dominates modern (online) communication, but this heavily visual landscape creates substantial barriers for those who cannot access it. To facilitate non-visual accessibility, we need to translate visual information into linguistic descriptions. What appears straightforward on the surface is in fact a fundamental communicative challenge: when moving between modalities, what exact information should be selected and conveyed? While modern vision–language models achieve ostensibly “superhuman” benchmark results on image-to-text tasks, they frequently underperform in real communicative settings. I argue that this gap reflects a deeper problem of information selection: effective image descriptions require not just stating what is literally true about an image, but deciding what should be said given a specific communicative goal and audience. Drawing on cognitive experiments with sighted participants and people who are blind or have low vision, I show that communicative goals strongly shape which visual information people choose to express, and that current AI tasks and evaluation methods fail to capture this nuance. I conclude by discussing recent work in which we start to quantify informativity in image descriptions, and how information selection varies across cultures, highlighting a broader research agenda for understanding and modeling cross-modal communication.
Elisa Kreiss’ work combines tools from natural language processing, psycholinguistics, and human-computer interaction to advance our understanding of how communicative context shapes language use. This research has direct applications to image accessibility – the challenge of (automatically) generating image descriptions for blind and low vision users.
Digital Culture Shock: Who Creates Technology and Why This Matters
1:30pm CDT, Center for Human-Computer Interaction + Design (Frances Searle Building 1-122)
Robots that encroach on your personal space, baffling emojis, a chatbot that gives you an answer that seems terribly rude—does any of this sound familiar? If so, you may know what it feels like to experience a clash of cultures in technology. In this talk, I will present key insights from my book Digital Culture Shock: Who Creates Technology and Why This Matters (Princeton, 2025), showing how culture—shared values, norms, and behaviors—influences both the design of technology and its use. Drawing on a set of common assumptions that do not translate across cultures, the talk will outline what is at stake and how we can resist generalizing our own cultural peccadillos in technology design.
Katharina Reinecke is a Professor and Associate Director for Research and Communication in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, Seattle. She has been speaking and writing about intelligent user interfaces that seamlessly adapt to people’s cultural and demographic backgrounds for nearly two decades and has pioneered ways for reliable, large-scale data collection from participants around the world with her virtual lab, the LabintheWild. Katharina received a PhD in Computer Science from the University of Zurich and was a postdoctoral fellow at Harvard University. Prior to coming to the University of Washington, she was an Assistant Professor in the School of Information at the University of Michigan.