AI Security and Data Minimisation - Information Commissioner's Office Webinar Insights
This article provides valuable insights from a webinar hosted by the Information Commissioner's Office on AI security risks, data minimisation techniques, and privacy-preserving methods. Experts discuss the challenges of AI security, the importance of a risk-based approach, and strategies to mitigate both existing and novel risks posed by AI technologies.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
AI, Security and Data Minimisation Information Commissioner s Office
Introductions Ahmed Razek Principal Technology Advisor Alister Pearson Senior Policy Officer Professor Reuben Binns Associate Professor at the University of Oxford, former Research Fellow in AI
Agenda What security risks does AI introduce? What data minimisation and privacy-preserving techniques are available for AI systems? Call to action Question and answer session
AI webinar series Selected published guidance ICO and The Alan Turing Institute, Explaining decisions made with AI ICO, Guidance on AI and data protection Webinars 1. AI, accountability and governance (September) 2. AI, lawfulness, fairness, and transparency (October) 3. AI, security and data minimisation 4. AI and individual rights (circa Dec)
What security risks does AI introduce? Key takeaways There is no one size fits all approach to security AI exacerbates existing security risks and poses novel ones. Take a risk-based approach to assessing the security of your AI system
There is no one size fits all approach to security The appropriate security measures you should adopt depend on the level and types of risks that arise from specific processing activities. For example, compare the security risks associated with an AI chatbot for a local library service with the risks associated with an AI chatbot on a payment page.
AI exacerbates existing security risks and poses novel ones Where AI exacerbates existing issues: Third-party code relationships with suppliers. Integrating your AI system with your existing IT components. Wider range of people involved in building and deploying AI systems. What should you do? Subscribe to security advisories to be notified of vulnerabilities or adhere to coding standards and instituting source code review processes. Separate the machine learning development environment from the rest of your IT infrastructure where possible (eg by using virtual machines or containers). Extend existing approaches to cover AI (eg staff training)
AI exacerbates existing security risks and poses novel ones Where AI poses novel issues: Membership inference Model inversion Black box and white box attacks What should you do? Avoid overfitting Monitor API requests
Take a risk-based approach to assessing security If you train models and provide them to others, you should assess whether those models may contain personal data or are at risk of revealing it if attacked and take appropriate steps to mitigate these risks. You should assess whether the training data contains identified or identifiable personal data of individuals, either directly or by those who may have access to the model. You should assess the means that may be reasonably likely to be used, considering the vulnerabilities described above. As this is a rapidly developing area, you should stay up-to-date with the state of the art in both methods of attack and mitigation.
What data minimisation and privacy-preserving techniques are available for AI systems? Key takeaways: The principle of data minimisations says that personal data must be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed. There are different considerations about the data minimisation principle during the training phase and the inference stage in the lifecycle of an AI system. You should consider balancing the need for greater accuracy with the need to gather limited personal data.
The data minimisation principle Article 5(1)(c) of the GDPR says: Personal data shall be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (data minimisation) At first glance it may be difficult to see how AI systems can comply with the data minimisation. The key is that you only process the personal data you need for your purpose.
Consider your different purposes during the training stage and inference stage Supervised machine learning approaches use personal data in two main phases: The training phase, when training data is used to develop models based on past examples; and The inference phase, when the model is used to make a prediction or classification about new instances. How should you minimise personal data in the training phase? Assess which features are relevant for your purpose and only process that data. Ensure you only keep data that is needed for a specific purpose (ie do not retain data on the off-chance that it might be useful in the future) Consider privacy enhancing methods (eg perturbation, synthetic data and federated learning). How should you minimise personal data at the inference stage? Convert personal data into less human readable; formats. Make inferences locally; and Privacy preserving query approaches
Balancing data minimisation and statistical accuracy In general, when an AI system learns from data (as is the case with ML models), the more data it is trained on, the more statistically accurate it will be. However, generally speaking, the more data points collected about each person, and the more people whose data is included in the data set, the greater the risks to those individuals, even if the data is collected for a specific purpose. So if you can achieve sufficient accuracy with fewer data points or fewer individuals being included (or both), you should do so.
Call to action 2. We are also conducting an assessment of the usability and effectiveness of the explaining decisions made with AI guidance. 1. We are developing a toolkit targeted at risk practitioners to help them assess their AI systems. We are currently looking for your views about what the tool should look like. You should have received details about how you can get involved in this. If you haven t and want to get involved, email explain@ico.org.uk If you would like to share your views, email AI@ico.org.uk
