Industry Affiliates

Faculty Panel Discussion

Moderator: Magda Balazinska

Panelists: Taylor Kessler-Faulkner, Steve Seitz, Ben Shapiro, and James Weichert

Keynote Speaker: Stephanie Wang

Abstract: While AI agents have revolutionized software engineering, it remains very challenging to build state-of-the-art AI systems. This is in part because of the long tail of workloads in both training and inference, spanning many use cases, models, and hardware. Such diversity demands optimizations tailored to the workload. In this talk, I’ll discuss some of our ongoing work on harnessing coding agents to autonomously build reliable and production-quality AI systems.

Biography: Stephanie Wang joined the faculty of the Paul G. Allen School of Computer Science & Engineering at the University of Washington in fall 2024. Her research is primarily in distributed systems. She is especially interested in how we can build intermediate abstractions that make it easier to build high-performance, fault-tolerant, and domain-specific systems. Along those lines, she is interested in domains where large-scale application development is still pretty hard, such as machine learning and data processing. She’s also interested in programming languages as the interface that can improve distributed systems.

Previously, Wang was a Ph.D. student in the RISELab at UC Berkeley, where she was advised by Ion Stoica. She is also a co-creator and committer for the open-source project, Ray, which has been used to train ChatGPT, serve high-performance LLMs, and break the CloudSort 100TB record. She subsequently continued to develop the Ray ecosystem as a software engineer at Anyscale, working primarily on Ray Data, a system for distributed data preprocessing for ML.

1:30-1:35: Introduction and Overview, Speaker TBD

1:35-1:55: Symbolic Relations for Network Change Verification, Brody Franks

Abstract: Relational network verification is a practical way to validate network changes without requiring exhaustive specifications, but existing tools compare a pair of concrete network states (pre- and post-change). They cannot answer what operators often care about: across the space of all possible network environments, which includes external route advertisements and failures, which behaviors could be affected by a change, and how?

To answer this question, we introduce Symbolic Relation Algebra (SRA), a new language and engine for network change verification. SRA can: (1) verify both the control plane and the data plane, in one general formalism, (2) customize the data representation at a low level of granularity, while retaining the same frontend specification, (3) support specialized data structures that hybridize concrete and symbolic verification and are efficient on real-world workloads.

SRA models network behaviors as binary relations whose fields may range over large domains. Input specifications are lowered to query plans over field-decomposed decision diagrams, propagating intermediate constraints lazily during specification checking. We demonstrate the effectiveness of our tool by applying SRA to representative network-change scenarios, examining both its expressiveness and performance. We show that SRA checks symbolic relational specifications 25-541× faster than a general-purpose solver.

1:55-2:15: TBD

Abstract: Coming soon

2:15-2:35: TBD

Abstract: TBD

1:30-1:35: Introduction and Overview, Jennifer Mankoff

View the Grand Challenge that relates to this topic

1:35-1:55: ArtInsight: AI-Powered Artwork Engagement for Mixed Visual-Ability Families, Arnavi Chheda-Kothary

Abstract: Coming soon

1:55-2:15: TaskAudit: Detecting Functiona11ity Errors in Mobile Apps via Agentic Task Execution, Mingyuan Zhong

Abstract: Coming soon

1:30-1:35: Introduction and Overview, Katharina Reinike

View the Grand Challenge that relates to this topic

1:35-1:50: AI safety is a multi-agent problem, Carrie Juan

Abstract: When we train AI, we assume the classic MDP paradigm, where the AI is training against a static environment. But the real world is not static. It’s full of humans who change and adapt as a result of using AI. As AI is deployed into the world, it alters both the people who use it (through social sycophancy and cognitive decline) and even our collective culture. One example is the artificial hivemind: language models increasingly produce the same homogeneous outputs. Another is the way LLMs distort our written language. In this talk, I will argue that AI safety is a multi-agent problem, and I will present MODA, a multi-agent approach to diverse generation. MODA treats each mode as a lightweight agent within a single shared policy, and it trains these agents to produce responses that are both high quality and diverse. This addresses a critical issue for building transparent and globally beneficial AI.

1:50-2:05: Open AI Infrastructure to Accelerate Science, Jacob Morrison

Abstract: Coming soon

2:05-2:20: African Data Ethics: A Discursive Framework for Black Decolonial AI Ethics, Teanna Barrett

Abstract: The shift towards pluralism in global data ethics acknowledges the importance of including perspectives from the Global Majority to develop responsible data science practices that mitigate systemic harms in the current data science ecosystem. Sub-Saharan African (SSA) practitioners, in particular, are disseminating progressive data ethics principles and best practices for identifying and navigating anti-blackness and data colonialism. To center SSA voices in the global data ethics discourse, we present a framework for African data ethics informed by the thematic analysis of an interdisciplinary corpus of 47 documents. Our framework features six major principles: 1) Challenge Power Asymmetries, 2) Assert Data Self-Determination, 3) Invest in Local Data Institutions & Infrastructures, 4) Utilize Communalist Practices, 5) Center Communities on the Margins, and 6) Uphold Common Good. We compare our framework to seven particularist data ethics frameworks to find similar conceptual coverage but diverging interpretations of shared values. Finally, we discuss how African data ethics demonstrates the operational value of data ethics frameworks. Our framework highlights Sub-Saharan Africa as a pivotal site of responsible data science by promoting the practice of communalism, self-determination, and cultural preservation.

2:20-2:35: Beyond the Conversation: Designing the Architecture Around AI in Mental Health, Jina Suh

Abstract: General-purpose conversational AI has become so embedded in everyday life that it works like a utility, and it now shapes how millions of people experience distress. Yet we still evaluate it like a product, one conversation at a time. This talk draws on surveys, interviews, and lived-experience accounts from people who turned to AI in distress. It looks at what happens around the conversation, following the whole journey: why people came to AI in the first place, what happened during the interaction, and what they did afterward. A safety redirect can score well on a benchmark and still cause harm, and people whom AI referred to a helpline rarely called it. The talk then looks at the people who carry the consequences of each safety feature: clinicians, crisis lines, families, communities, and the reviewers behind the scenes. It also covers the value tensions any design has to work through, such as intervention versus autonomy. I argue that the design problem is architectural. It is about who is accountable, what gets measured, and what paths a person has to safety. Standards should apply to that architecture: consent given in advance, multiple paths to safety, and measuring the whole journey. The responses themselves should stay tailored to the person.

2:45-3:05: From Policy Models to World Action Models, Ge Yan

Abstract: Robot learning has long relied on policy models that map observations directly to actions, while world action models (WAMs) that jointly predict future observations and actions are emerging as a powerful alternative. In this talk, I will trace this shift through my work, from ManiFlow, a flow-matching manipulation policy, and UniFlow, a unified 2D and 3D multi-modal policy, to Flex-π, a multi-stream world action model that jointly predicts future video, DINO semantics, 3D point maps, and actions. A single Flex-π model can run as a fast action-only policy, a full world action model, or anything in between at inference time, and it more than doubles the real-world success rate of the strongest baseline on bimanual, dexterous, and high-precision manipulation tasks.

Biography: Ge Yan is a Ph.D. student at the University of Washington, advised by Dieter Fox, and works with the Toyota Research Institute. His research focuses on robot learning, in particular multi-modal policy and world action models that combine video, semantic, and 3D representations for dexterous manipulation. His recent work includes ManiFlow, UniFlow, and Flex-π. 

3:05-3:25: Large-Scale Reinforcement Learning in Simulation for Dexterous Robotic Assembly, Patrick Yin

Abstract: Robot learning needs large, diverse data to acquire dexterous, general-purpose skills. Physics simulation can generate that data at scale, but two central challenges remain. The first is generating meaningful behaviors in simulation without extensive human effort. The second is deciding what to train on as task complexity and the number of tasks grow. This talk presents three works that build on each other to address these challenges. OmniReset makes long-horizon reinforcement learning tractable by training from a broad distribution of reset states with a single reward function. It removes the need for demonstrations, curricula, and reward tuning, producing skills such as screwing, insertion, and assembly that transfer to real hardware from RGB cameras. SGS (Success-Guided Sampling) adaptively focuses training on the tasks and conditions where the policy is still learning. AlphaNIST combines these ideas to train a single policy on all 20 tasks of the NIST assembly board using only binary task rewards. It reaches 98.2% overall success and outperforms specialists trained on each task separately. Together, these results suggest that binary task rewards and multi-task training, both historically harder, become advantages at scale.

Biography: Patrick Yin is a Ph.D. student at the University of Washington’s Paul G. Allen School of Computer Science & Engineering, advised by Abhishek Gupta, and an NSF Graduate Research Fellow. He is also a research intern at NVIDIA. His research focuses on sim-to-real reinforcement learning for contact-rich robotic manipulation. Previously, he interned at Microsoft Research and completed his undergraduate studies at UC Berkeley, where he worked with Sergey Levine.

3:25-3:45: Cyber-Mechanical Intelligence: Creating Near-Gram Robots That Can Move Autonomously, Kyle Johnson

Abstract: Near-gram robots capable of autonomous movement could transform distributed sensing, environmental monitoring, and inspection in environments inaccessible to larger platforms, yet achieving true autonomy at sub-gram scales demands a fundamentally different design philosophy. This talk introduces Cyber-Mechanical Intelligence (CMI), a co-design paradigm that treats the location of intelligence within a robot as a design variable, distributing autonomous function across physical, electronic, and computational domains to meet task requirements while minimizing system-level resource cost. Three robotic systems are presented as demonstrations of CMI in practice. The Origami Microflier (414 mg) encodes aerodynamic decision-making into a bistable origami structure, enabling solar-triggered mid-air morphological reconfiguration and programmable dispersal of up to 98 m without continuous power. The MilliMobile achieves battery-free terrestrial locomotion from ambient solar and RF energy as low as 50 µW, and extends to onboard visual navigation through a lightweight embedded vision pipeline. The Coin-Copter offloads roll and pitch stabilization to passive flybar-rotor dynamics, reducing the sensing and control burden of a 1 g aerial vehicle while demonstrating closed-loop yaw control and a lift-to-weight ratio of 4.2. Together, these systems argue that intelligent behavior in resource-constrained robots is best understood as a property of the complete cyber-mechanical system, not of computation alone.

2:40-2:45: Introduction and Overview, Linda Shapiro

2:45-3:00: Online Neural Space Time Memory for Dynamic Novel View Synthesis, Baback Elmieh ↗

Abstract: Online novel view synthesis from multi-view streaming videos faces a fundamental trade-off: maintaining a persistent, long-horizon memory to reconstruct temporarily occluded regions while operating under strict real-time constraints. While Test-Time Training (TTT) offers a powerful memory mechanism, standard models mandate gradient-based memory updates at every frame to adapt to the changing motion in dynamic scenes. The computational cost of heavy memory updates precludes real-time application and can lead to instability over long contexts. Given that memory updates are more demanding than memory application and video content is largely redundant, we propose to decouple the frequencies of these two processes. Our approach performs periodic memory updates while applying the memory on a per-frame basis, using cross-view attention to manage deformations between the prior memory state and the current frame. To lock in the historical context, we introduce two critical mechanisms: an auxiliary Memory Loss that forces persistent internalization of the scene, and a Memory Caching strategy that regularizes active weights against catastrophic drift. Our method demonstrates state-of-the-art minute-scale memory persistence in online dynamic human scenes at amortized real-time speed. Visit webpage ↗

Biography: Baback Elmieh is a Computer Science PhD student at the University of Washington, where his research focuses on generative video models, real-time interactive systems, and persistent visual memory for dynamic 3D scenes. His recent work on Neural Space-Time Memory addresses a key challenge in immersive 3D telepresence: synthesizing consistent views of moving people by remembering visual details that are no longer visible to the cameras. Before returning to academia, he spent over two decades developing video games, real-time 3D rendering technologies, and AR/VR systems at Google, Meta, Qualcomm, and several startups. His work on Google’s Spotlight Stories earned an Academy Award nomination and an Emmy for innovation in interactive storytelling.

3:00-3:15: Toward Interactive Multimodal Agents, Zixian Ma ↗

Abstract: Multimodal agents have the potential to become a new interface through which people interact with digital and physical environments. While recent advances have enabled agents to perceive complex visual scenes and perform increasingly sophisticated tasks, building agents that can interact effectively with people and dynamic environments remains an open challenge.

In this talk, I will present my research toward interactive multimodal agents, progressing from perception to action to interaction. I will first introduce Molmo2, a multimodal foundation model for understanding and grounding visual information. I will then discuss how multimodal capabilities extend to tool use and autonomous action through LATTE and MolmoWeb, which enable agents to interact with tools and navigate websites visually. Finally, I will introduce our ongoing work on Interactive Visual Gym, an environment for training and evaluating agents that interact with humans in dynamic tasks such as cooking and navigation. Together, these works explore how to move beyond multimodal understanding and autonomous task execution toward agents that can continuously perceive, act, and collaborate with people.

Biography: Hi! I am a 4th-year Ph.D. student in Computer Science and Engineering at the University of Washington, advised by Prof. Ranjay Krishna and Prof. Dan Weld, and supported by the Qualcomm Innovation Fellowship and Two Sigma PhD Fellowship. I build open multimodal models (Molmo2, MolmoPoint), agents that act on the web and with tools (MolmoWeb, LATTE), and benchmarks of what these systems can and can’t do (Task Me Anything, CREPE). Currently, I’m especially interested in real-time, interactive multimodal models and agents.

3:15-3:30: Do Medical Language Models Preserve Correct Judgement Under Misleading Context?, Simon Zou

Abstract: Clinical and consumer-facing deployments of large language models (LLMs) can now achieve expert-level scores on medical question-answering benchmarks based on licensing exams. However, real-world questions from users rarely resemble the clean, single-turn exam questions in these benchmarks. In reality, interactions are often layered with patient-reported history, retrieved documents, and false claims. This thesis asks whether an LLM that answers a medical question correctly in a clean environment continues to be correct once such misleading context is added. In short, it often does not. To study this question at scale, this thesis presents MedMisBench, a benchmark of 10,932 medical question items paired with 48,889 misleading context option sentences spanning medical-reasoning, agentic-capability, and patient-journey question formats. Each injected sentence is constructed along two independent axes that separately capture the type of medical false claim and its apparent source. This way, failures can be attributed to what is false and who asserts it. Evaluated across 11 configurations of commercial, open-source, and medical-domain LLMs, mean answer accuracy falls from 71.1% on clean questions to 38.0% once an injection sentence is prefixed to one answer option, with a mean attack success rate of 51.5%. Fabricated treatment exceptions and claims attributed to clinical authorities are consistently the most effective types in lowering the accuracy of LLMs, with attack success rates as high as 64.1% and 69.5% respectively. A panel of 14 clinical reviewers spanning 7 countries judged that 38.2% of these injection-induced errors carried serious potential harm to a patient. These findings indicate that strong performance on existing medical benchmarks does not mean that a model will preserve sound clinical judgment once its context is contaminated by misleading information. By releasing MedMisBench as a static, reusable benchmark, this work provides a foundation for evaluating and eventually mitigating this gap between benchmark performance and real-world robustness.

Biography: I’m a first-year Ph.D. student in Electrical and Computer Engineering here at UW, advised by Professor Linda Shapiro. My research focuses primarily on medical applications of computer vision and AI, from evaluating whether medical language models hold on to correct answers under misleading context to building egocentric vision systems that automate welfare checks for laboratory animals. I’m co-first author of MedMisBench, the NeurIPS 2026 benchmark paper I’m presenting today. I hold an M.S. from the University of Washington and a B.A.Sc. in Systems Design Engineering from the University of Waterloo.

3:30-3:45: Measurement-Conditioned Virtual Try-On, Yuanhao Wang ↗

Abstract: Real-world clothing try-on serves two purposes: visualizing how a garment looks on the body, and seeing how a specific size actually fits—whether it stretches tightly or drapes loosely. Current Virtual Try-On (VTO) models address appearance but ignore fit; conditioned only on 2D images, they default to synthesizing deceptively well-fitted results regardless of garment dimensions. In this work, we present RealFIT, the first scalable framework for real-world, measurement-conditioned virtual try-on across complex apparel categories. To overcome the scarcity of metric-labeled data and the sim-to-real gap of 3D physics engines, we introduce a multi-tier data hierarchy combining web retail metadata, VLM-assisted annotations, and a newly curated real-world dataset with tape-measured ground truth. We further propose a flow-matching VTO architecture featuring a dedicated measurement encoder and a Localized Metric Contrastive (LMC) loss that contrasts velocity fields across sizing perturbations to sharpen fit sensitivity. Finally, we establish a systematic evaluation protocol and demonstrate that RealFIT achieves state-of-the-art photorealism, fit accuracy, and size controllability.

Biography: Yuanhao is a second-year PhD student in the Graphics and Imaging Lab at UW, co-advised by Prof. Ira Kemelmacher-Shlizerman and Prof. Steve Seitz. He is also a student researcher at Google. Before UW, he was a master’s student at Carnegie Mellon University advised by Prof. Fernando De la Torre, and an undergraduate student at Brown University advised by Prof. James Tompkin. His research focuses on computer vision and generative models.

2:40-2:45: Introduction and Overview, Franzi Roesner

View the Grand Challenge that relates to this topic

2:45-3:00: Online Safety in the Age of AI, Grace Brigham

Abstract: Coming soon

3:00-3:15: Protecting against malicious AI-powered image editing, Anshul Nasery

Abstract: Coming soon

3:15-3:30: Securing DNA Synthesizers against AI-Uplifted Bioweapon Creation, Henry Wong

Abstract: Coming soon

3:30-3:45: Securing DNA Synthesizers against AI-Uplifted Bioweapon Creation, Henry Wong

Abstract: Coming soon

3:50-3:55: Introduction and Overview, Magda Balazinska

3:55-4:15: R3Con: Context Representation for Complex Question Answering over Large Corpora, Michael Theologitis

Abstract: Coming soon

4:15-4:55: KathDB-FAO: Efficient Question Answering over Multimodal Data via Code Generation and Optimization, Guorui Xiao

Abstract: Coming soon

4:35-4:55: How People Debug Complex SQL With LLMs: A Qualitative Study, Nicole Sullivan

Abstract: Coming soon

3:50-3:55: Introduction and Overview, Jason Hoffman

3:55-4:15: EvoClinician: Self-Evolving System for White-Box Multimodal Clinical Reasoning, Yongkang Li

Abstract: Coming soon

4:15-4:35: Dissecting and directing pathology foundation models, Chanwoo Kim

Abstract: Foundation models (FMs) are central to recent advances in digital pathology, encoding histology images into dense embeddings for diagnostic classification, molecular alteration prediction, and clinical outcome modeling. However, the opacity of these embeddings renders such systems “black boxes,” limiting their clinical translation and scientific utility. Here, we introduce PICASSO (Pathology Image Concept Atlas built via SparSe dictiOnary learning), a framework that makes pathology FMs interpretable and controllable. It decomposes FM embeddings into human-interpretable visual concepts using a sparse autoencoder, a form of dictionary learning. Trained on more than 120 million tissue patches across 32 cancer types, PICASSO produces the first pan-cancer atlas of histomorphological concepts. Beyond interpretation, PICASSO enables concept editing, the targeted manipulation of concepts to interrogate and control model behavior. These capabilities enable diverse applications, including model auditing, biological discovery, and in silico interventions. PICASSO transforms FMs from black-box predictors into interpretable and controllable models that support biological insight, hypothesis generation, and virtual experimentation.

4:35-4:55: PeriPulse: Open-Ear Headphones for Robust Heart Rate Sensing During High-Intensity Exercise, Cynthia Dong

Abstract: Live cardiovascular monitoring has potential to unlock new interactive health experiences, but motion artifact noise and limited form factor options prevent users from accessing high-accuracy sensing. Leveraging the popularity of open-ear headphones and their unique contact pressure, advantageous physiological positioning, and potential for audio interaction, we present \device, an accurate, motion-robust headset which integrates neatly into users’ existing habits. We iteratively designed PeriPulse based on in-lab studies, then collected and trained models on a high-intensity exercise dataset of 29 participants, extracting heart rate with an error of 8.11 +- 8.30 BPM.

We then deploy models for real-time heart rate estimation and generate audio-based heart zone alerts with 9 participants. Users found the system accurate (error: 5.03 +- 2.16 BPM), appreciated verbal notifications over visual interfaces for zone training, and saw potential to consolidate other trackers into their headphones. Our results illustrate \device’s capacity for motion-robust sensing, and suggest opportunities for innovative end-user applications.

Flex-π: A Multi-Stream World-Action Model with Compute Flexibility
Authors: Ge Yan, Jinghao Liu, Yuzhi Fan, Lei Cai, Minwen Liao, Jesse Zhang, Dieter Fox
Presenter: Ge Yan
Lab/Research Area: RSE Lab, Robotics

Short Abstract: Developed Flex-π, a world-action model that jointly predicts future appearance, 3D geometry, and object semantics in one shared latent space, addressing a key limitation of existing world models that rely on RGB video alone. A single checkpoint runs either as a fast action-only policy or as a full world model, letting users trade latency for accuracy at deployment without retraining. On a real bimanual robot, it outperforms the strongest baselines by 2–6× on precise, dexterous tasks, including a robot repairing its own gripper with tight-margin insertions, and the full code, data, and models are open-sourced.

When Agents Slow Down: Understanding LLM Agents’ Test-Time Strategies via Elo-per-token Analysis
Authors: Kaiyuan Liu, Qiuyang Mang, Bo Peng, Wenhao Chai, Hanchen Li, Shreyas Pimpalgaonkar, Luke Zettlemoyer, Alex Dimakis, Alvin Cheung
Presenter: Kaiyuan Liu
Lab/Research Area: ZLab

Short Abstract: Large language model (LLM) agents allocate test-time compute adaptively as they revise solutions, use tools, explore alternatives, and decide when to stop. This test-time strategy makes it difficult to measure how agent performance scales. We study open-ended tasks that provide continuous scores for intermediate submissions, making progress observable throughout long trajectories. We propose Elo-per-token analysis, which tracks the best solution found at each token budget and uses a Bradley-Terry model to aggregate within-task orderings into Elo ratings across tasks with different score scales. We apply it to four general-purpose agents on four open-ended benchmarks, with sessions of up to 100M tokens, and to three feedback-driven LLM optimization harnesses in controlled single-task interventions. Independent sampling provides a theoretically characterized reference, for which Elo grows linearly with log compute. Against this reference, agents can initially convert tokens into Elo faster than independent sampling, but their marginal gains diminish and eventually fall below the reference. In contrast, the strongest historical human contestants improve superlinearly over contest time on shared AtCoder Heuristic Contest tasks, providing evidence of continual learning and substantial headroom after agents slow down. We define the scaling inflection point as the per-session budget where marginal Elo gains match the independent-sampling reference. Using this point as the per-session budget, we split 100M tokens across parallel sessions on FrontierCS Polyomino Packing, gaining +264 Elo over one long session and +355 over ten short sessions.

Forty Shades of Blue: Quality-Diversity Alignment via Mode-Conditioned Reinforcement Learning
Authors: Jiayi Yuan, Hangoo Kang, James Jihao Liu, Yejin Choi, Vikram Iyer, Liwei Jiang, Natasha Jaques
Presenter: Jiayi Yuan
Lab/Research Area: SocialRL Lab

Short Abstract: We present MoDA (Mode-conditioned Diversity Alignment), an online post-training RL algorithm that jointly optimizes generation quality and diversity, inspired by the coordination perspective in multi-agent reinforcement learning (MARL). MoDA trains a single shared LLM policy conditioned on abstract numbered roles, where each role acts as an agent competing to produce outputs distinct from the others. This formulation encourages mode-conditioned agents to explore complementary regions of the high-quality output space without requiring hand-crafted personas or architectural modifications.

Title: AutoSciency: Mapping Knowledge Gaps in Embedding Space
Authors: Jiayi Yuan, Aarav Talati, Emma Huang, Sota Kobata
Presenter: Jiayi Yuan
Lab/Research Area: SocialRL

Short Abstract: Prior work shows that unexpected combinations of content and context can predict impact, while some embedding methods provide scientific document embeddings for downstream tasks [4]. Our project builds on these ideas by asking whether gaps in embedding space can be translated into human-readable research hypotheses.

ArtScape: Detecting, Characterizing, and Interactively Exploring Urban Art at Scale Using Vision Language Models
Authors: Lucy Jiang, Jared Hwang, Jon E. Froehlich
Presenter: Jared Hwang
Lab/Research Area: Makeability Lab (HCI)

Short Abstract: Urban art brings cultural expression, community identity, and aesthetic enrichment to cities. Currently, there are no systematic ways of exploring urban art at scale. We introduce ArtScape, a VLM-based interactive tool that lets users view, search, filter, and tour four categories of art (murals, mosaics, sculptures, graffiti). To comprehensively and automatically detect urban artwork throughout a region, we designed a custom five-stage pipeline that uses VLMs to detect and describe urban artwork within 360° panoramic Street View images, achieving a micro-F1 of 0.83 on an urban art dataset we labeled. ArtScape categorizes, describes, and scores the reachability and interestingness of each artwork while deduplicating based on proximity and similarity. We evaluated this tool with 12 participants spanning urban planners, artists, sighted users, and blind and low vision users. Our findings show that ArtScape’s at-scale AI artwork curation enables granular and aggregate urban art discovery, thereby helping people learn about their communities and city-wide trends, motivating in-person urban exploration, and expanding perceptions of what constitutes urban art.

How People Debug Complex SQL With LLMs: A Qualitative Study
Authors: Nicole Sullivan, Ella Cao, Leilani Battle, Magda Balazinska
Presenter: Nicole Sullivan
Lab/Research Area: Database Group

Short Abstract: We lack a fundamental understanding of how LLMs impact the SQL query debugging workflow. In response, we conducted a user study with 16 participants of varying SQL expertise, examining how they debug complex SQL queries with and without LLM support. Qualitative analyses of our study data revealed that while participants trusted the LLM, they varied widely in how and whether they delegated debugging tasks to the LLMs. Additionally, participants enforced specific collaborative roles for the LLM such as TA or sounding board, which participants controlled by withholding context from the LLM. These findings suggest that LLM-based tools could enable preset role selection, broader context management, and flexible automation features, including low automation options, to support SQL debugging moving forward.

FlexiCodes: Embedded De Bruijn Patterns for Deformation Tracking on Elastic 3D-Printed Objects with a Monocular Camera
Authors: Daniel Campos Zamora, Raiden Santos*, Ria Agarwal*, Jerry Cao, Jon Froehlich, Chang Xiao
Presenter: Daniel Campos Zamora
Lab/Research Area: Makeability Lab, HCI

Short Abstract: Non-rigid interfaces allow for soft, deformable inputs using malleable materials, which enable interactions that would be infeasible with traditional rigid controllers. However, tracking and decoding the deformations typically requires embedded sensing hardware or complex external setups (e.g., stereo cameras, calibrated projector setups). We present FlexiCodes a novel approach to deformable interaction that leverages 3D printing to embed De Bruijn patterns onto elastic objects and computer vision to track surface deformation. Our three-part system consists of design, fabrication, and computer vision components that allow for vast design possibilities for flat and volumetric forms, preserve the ability to tune mechanical properties, and enables live interactions, respectively. We perform a technical evaluation and validate our approach through a series of proof-by-demonstration scenarios.

Thinking while Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents
Authors: Zachary Englhardt
Presenter: Zachary Englhardt, Justin Chae, Bilal Bhurgri
Lab/Research Area: Ubicomp Lab

Short Abstract: Voice agents face a fundamental tension. The reasoning, retrieval, and tool use that make foundation models capable are iterative and slow, while conversation demands responses on a millisecond timescale. Small real-time models meet the latency bar but cannot match foundation models on complex tasks, leaving current voice agents to trade away either responsiveness or capability. We introduce conversational infill, in which a small Talker model both immediately generates contextually grounded responses to hide the latency of an external Reasoner model and fluently integrates streamed Reasoner knowledge into its responses during inference. We curate a 290,571-example synthetic dataset spanning six domains and show that the task is learnable across seven widely used small language models ranging from 135M to 1.7B parameters. Our system, ConvFill, sustains millisecond-level time-to-first-response while closing the accuracy gap to within 6.3% of the corresponding frontier Reasoner. In a live user study on Apple M2 hardware (n=18), participants rated ConvFill on par with frontier models overall, preferred it for retrieval tasks, and ranked it significantly more responsive.

GazeRL: Cognitive Reward Shaping for Scanpath Prediction
Authors: Jiexin Ding, Xinyun Liu, Ruotong Yu, Zhihan Zhang, Shwetak Patel, Yue Jiang, Akshay Gadre
Presenter: Jiexin Ding
Lab/Research Area: Ubicomp Lab

Short Abstract: GazeRL fine-tunes a vision-language model (Qwen2.5-VL-3B) to predict human visual-search scanpaths using reinforcement learning with cognitively grounded rewards: decaying bottom-up saliency, ramping top-down semantics, and object-level inhibition of return. On COCO-Search18, it produces scanpaths that are both accurate and human-like diverse, ranking best or second-best on every metric that survives our audit.

TBD
Authors: TBD
Presenter: TBD
Lab/Research Area: TBD

Short Abstract: TBD