Security, AI, and the Future of Data Ownership with Dawn Song
The Evolution of Cybersecurity
Dawn Song, a professor of computer science at UC Berkeley, explores the complex landscape of security vulnerabilities, emphasizing that software systems will likely always contain bugs. As we refine our ability to build more secure systems, the primary threat is shifting up the stack.
Humans as the Weakest Link
- Social Engineering: Attackers are increasingly exploiting human psychology through phishing and manipulation rather than attacking code directly.
- AI for Defense: Song discusses using NLP-driven chatbots as a personal security agent to detect, challenge, and interact with potential attackers to protect users from social engineering.
Adversarial Machine Learning
This section addresses the fragility of deep learning models.
Types of Attacks
• Inference Stage: Manipulating inputs with imperceptible perturbations to force machine learning systems to misclassify images, such as traffic signs in autonomous driving.
• Training Stage: Inserting poisoned data to create backdoors that only trigger for specific inputs chosen by the attacker.
"The adversarial examples show us that we are still at a very early stage of really developing robust and generalizable machine learning methods."
Defense Strategies
• Rich Representations: Shifting away from fragile patterns toward more rich and nuanced representations similar to human vision.
• Spatial/Temporal Consistency: Leveraging natural constraints, such as spatial consistency in segmentation, to detect adversarial interference.
Data Ownership and Future Visions
Song argues for a reevaluation of data rights to align with economic growth principles similar to physical property law.
• Ownership: Clearly defining digital data ownership enables users to dictate how their information is used—balancing utility and privacy.
• Confidentiality: Techniques like differential privacy add noise to training processes to prevent sensitive data extraction from models.
• Responsible AI: The ultimate goal is a distributed secure computing fabric where data can be leveraged in a privacy-preserving and responsible manner.