As technology becomes increasingly embedded into every aspect of our lives, University of Washington researchers are exploring how to balance convenience and connection with the security, privacy and safety risks that may arise from new technologies and their applications in new contexts.
In the latest installment of our Q&A series highlighting grand challenges in computing and the researchers working on potential solutions, we spoke with Allen School professors Franziska Roesner (Ph.D., ‘14), who co-directs the Allen School’s Security and Privacy Research Lab, and Sewoong Oh, whose research focuses on differential privacy as well as secure and robust machine learning. Along with their collaborators across campus, they aim to develop a safer, more secure and privacy-preserving future through technologies that better serve people and society.
Thinking about security, privacy and safety is not just a technology question; it’s also a very human question for lots of different reasons. Understanding the actual social, societal and personal context matters a lot.
How do you anticipate potential security and privacy concerns arising from new technologies?
Franziska Roesner: The security mindset is something that we teach our students, it’s a different way of thinking about systems. Instead of asking, “How do we make it faster and have more features?”, we ask, “How might somebody misuse it? What happens if it gets input that is different from what I expected? What are some assumptions that I might have intentionally, or worse, accidentally baked into it that somebody could manipulate?” Before the designs are set in stone, we try to think about how the technology might be misused by an adversary or go wrong.
Sewoong Oh: Technology harms don’t always involve an intentional adversary. Incentives and security and privacy needs are not always aligned for companies, and that can also cause trouble. For example, one side of it is that the speed of things — including emerging technologies — is getting exponentially faster, maybe for incentive reasons. Companies don’t necessarily need to slow down, but they need to pay equal attention to things that may not make money, such as protecting copyright and preserving privacy. This is where we are looking to regulators to help enforce things.
Attackers and their tactics are constantly changing, so how do we ensure that our defenses also evolve to keep up with increasingly sophisticated threats?
FR: To take a concrete example, in the artificial intelligence era, there’s concern about the models getting better and better at finding computer security vulnerabilities, or exploiting computer security vulnerabilities that have maybe been there all along, but people didn’t have quite the scale of operation of finding and exploiting them. The approach there isn’t just that we also need the defenders to find them faster and fix them faster.
Instead, some of the coolest research steps out of the arms race and says, if we build systems in this other way, or we use certain programming languages, or we use verification techniques, we can actually make it so that an entire class of vulnerabilities can be removed, rather than having that back and forth.
SO: Opening up some of their process to the research community would actually benefit companies in the long run. Even these days, when you feel like everything could be vetted through your AI and tested, I see cases where companies are still relying on security by obfuscation — meaning they’re hoping that people don’t figure out that they’re doing it one way and fingers crossed. For example, recently it was discovered that the decryption key for the reasoning traces [of LLMs] are shared across users, contexts and models. This is a serious flaw in security, like using the same password for all your accounts.
An open system would have discovered such flaws much earlier. They are so afraid of whatever might leak or the competition catching up that everything is closed, which I think is particularly harmful for security and privacy.
The grand challenge brings together researchers working across different disciplines in computer science. Why does security, privacy and safety research lend itself to such interdisciplinary endeavors?
FR: As a whole, thinking about security, privacy and safety is not just a technology question; it’s also a very human question for lots of different reasons. Understanding the actual social, societal and personal context matters a lot for understanding what are the actual problems you’re trying to solve and what solutions are effective. For example, journalists’ work can be at odds with some of the assumptions that security tools aiming to help journalists make. Another big branch of recent work in computer security has been around intimate partner violence-type threat models, where some of the traditional assumptions — such as your attacker doesn’t have physical access to your device — actually don’t hold in an intimate partner kind of setting.
These complex problems in security, privacy and safety require bringing together researchers from a lot of different perspectives such as folks working in cryptography on much more theoretical questions to those working in human-computer interaction.
Companies don’t necessarily need to slow down, but they need to pay equal attention to things that may not make money, such as protecting copyright and preserving privacy.
SO: One example of this kind of collaboration happening within security and privacy research right now is mathematicians and theoreticians tackling safety and alignment with AI systems. It is challenging to find a mathematical principle that fits this problem in any way that is actionable but also aligns with human values. I think this speaks to how what we’re trying to solve in this grand challenge is so difficult and unlikely to be solved by any one domain. Instead, it requires experts getting together and developing completely new principles and new designs — that’s when this merge of different fields happens naturally out of necessity, and that’s when breakthroughs spark.
Learn more about the Allen School’s work on security, privacy and safety research, and explore more grand challenges in computing.