How can we harness the University of Washington’s research excellence and culture of collaboration to advance computing in ways that truly serve humanity?
Over the past year, Allen School researchers have coalesced around a grassroots effort to find out. They identified a set of grand challenges to which they would apply their deep expertise and engage a broad network of collaborators to advance innovation with the potential for significant societal impact.
First on the list: artificial intelligence. On the one hand, it has the potential to increase productivity and accelerate discovery, but on the other, it may lead to unintended consequences — especially for those who don’t fit the default user profile.
For the initial installment of a new Q&A series highlighting the grand challenges — and the researchers working on potential solutions — we spoke with professors Katharina Reinecke, whose research focuses on AI ethics and human-computer interaction, and Noah A. Smith, whose research at the Allen School and Allen Institute for AI (Ai2) centers around machine learning and natural language processing. Together with their colleagues across campus and beyond, they aim to answer the question, “How do we design AI in a way that is transparent and equally beneficial to all?”
If we want to understand and evaluate these systems, along with the problems and the assumptions that have gone into them — which will ultimately lead to better improved systems in future generations of the technology — we’ve got to have openness and transparency.
The Transparent AI grand challenge emphasizes open source, open data and open processes as pillars of future AI tools. What should people expect these tools to look like in 10 or 15 years?
Noah A. Smith: One way I think about what things could look like in 10 to 15 years is infrastructure that’s adaptable at a local scale, and I really like the analogy to open source software. Anyone should be able to change an AI system, track those changes, understand the consequences and experiment. It has to be a fully open stack — the data, the models, the training recipes, the evaluation, the processes — they’ll have to be open and reproducible.
What we have today is basically an interface where you can interact with an AI system. In the future, people will also need a fundamental knowledge and an understanding of how the systems work, and how the data affects what the model can do.
Katharina Reinecke: I totally agree, though even if some of the models remain closed, I think we can still achieve some kind of transparency. For example, we’ve been envisioning AI tools that might become much more open about their biases. We just published a paper in which we showed people a simple overview of these value biases that any model has, and when you make people aware of that, they actually become much more cautious about relying and over-relying on the AI. This neutrality assumption is always something that really bothers me, and I feel like that is something we can address even when the rest of the model remains closed.
What are some of these biases in AI models, and how can we address them?
KR: Most people think it’s all about the training data — if only we could have more diverse training data, then AI would be much better for everybody. But I actually think the biases come in already at a much earlier point. For example, the idea that AI should help us become more efficient, that’s a very Western idea in itself. In terms of structural barriers, there is the lack of diverse training data but I think it’s also just the lack of diverse viewpoints and experiences in tech companies who decide that there is a need for certain products.
NS: The tendencies and biases in these systems are partly due to the data, but not just the data. I can point to work from my lab about tokenization. It’s the very, very early stage where a language model takes text and breaks it into a sequence of pieces. It turns out that the way you break the text into pieces has a huge effect on the cost and the experience that people have, depending on what language and writing system they use. The ability to study and understand these kinds of biases is accelerated by the availability of open pipelines — it gives you many more lenses on the problem and potential solutions.
What are you most excited about with regard to the future of open and equally beneficial AI?
KR: I actually see that AI could have so many benefits, if only we let society play a role in shaping it. Many people really want to use AI. They see the benefit of it such as there’s all these medical cures that could happen, it could address climate change, all sorts of things. These are indisputably beneficial to society, but really figuring out how society can have a seat at the table to discuss what is being developed, how it is being developed, and why, as well as how it is being rolled out — that is a really exciting opportunity for research.
I actually see that AI could have so many benefits, if only we let society play a role in shaping it.
NS: I agree with these points about broader agency and giving communities and individuals the power to shape tools around their own needs and their own norms. I think that’s deeply linked to better science. I’m an AI scientist, and these things are artifacts worthy of scientific study. If we want to do that, if we want to understand and evaluate these systems, along with the problems and the assumptions that have gone into them — which will ultimately lead to better improved systems in future generations of the technology — we’ve got to have openness and transparency. Cultural responsiveness and open AI systems are deeply linked.
Learn more about the Allen School’s work on transparent and beneficial AI, and explore more grand challenges in computing.