Ethics Pen-Testing for AI Social Good
A proposal to use ethics pen-testing to enhance AI systems for social good. Addressing questions on defining social good, roles of knowledge, potential side-effects, and striving to improve. Exploring the concept of social good in terms of substantive and procedural aspects. Advocating for an adversarial approach to foster continuous improvement.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Ethics pen-testing We can t make AI systems good but we have to make them better Bettina Berendt DTAI, Dept. Computer Science, KU Leuven https://people.cs.kuleuven.be/~bettina.berendt/ AI in Asia AI for Social Good, March 6, 2017, Tokyo
Context: Approaches to AI and ethics IEEE Ethically Aligned Design, Asilomar AI Principles, : All Artificial Intelligence and Autonomous Systems should embody the highest ideals of human rights prioritize the maximum benefit to humanity and the natural environment be in the service of widely shared ethical ideals, and for the benefit of all humanity Machine learning poses specific problems ( AI safety , verified AI ) AI for Social Good: The use (and development) of AI methods specificallyto further the social good e.g. about homelessness, climate change, healthcare, drugs,
Summary Premise 1: AI systems are primarily created by engineers academics or practitioners. Premise 2: The creators want their systems to increase the social good, and believe that they do. Question: Are they right in believing this? Proposal: Questions designed to break this belief goal: improve the systems
What is pen-testing? A penetration test, colloquially known as a pen test, is an authorised simulated attack on a computer system that looks for security weaknesses, potentially gaining access to the system's features and data. Note: No system is 100% secure. The point is not to pass all pen- tests (this is impossible), the point is to get better through trying.
The proposal: Ethics pen-testing for AI for Social Good What are we talking about? What is social good ? What is good ? Define Who defines the social good / the problem? Dissect the method What role does knowledge play? What are the side-effects? Do all problems have solutions? How can the answers to the questions in the second block break the requirements of the answer to the questions in the first block? Be adversarial How can we do better? Keep going
What is the social good / common good? Substantive that which is shared by and beneficial to all or most members of a given community specifies precisely what factors or values are beneficial and shared Provocation: part and parcel of all terror regimes Procedural the outcome that is achieved through collective participation in the formation of a shared will Usually contains ethical dilemmas that cannot be resolved out of context and/or for good
What is the social good / common good? Substantive that which is shared by and beneficial to all or most members of a given community specifies precisely what factors or values are beneficial and shared Provocation: part and parcel of all terror regimes Procedural the outcome that is achieved through collective participation in the formation of a shared will Usually contains ethical dilemmas that cannot be resolved out of context and/or for good
What is the social good / common good? Substantive that which is shared by and beneficial to all or most members of a given community specifies precisely what factors or values are beneficial and shared Provocation: part and parcel of all terror regimes Procedural the outcome that is achieved through collective participation in the formation of a shared will Usually contains ethical dilemmas that cannot be resolved out of context and/or for good
Who defines the good / the problem? Example: What is the drug problem ? And who was involved in formulating it, and under what constraints? People use drugs Poor people use drugs Drug users commit crimes Drug users get ill, homeless, Drug users die There aren t enough drugs available.
Who defines the good / the problem? Example: What is the drug problem ? And who was involved in formulating it, and under what constraints? People use drugs Poor people use drugs Drug users commit crimes Drug users become homeless, ill, Drug users die There aren t enough drugs available.
What role does knowledge play? Imparting knowledge is often overrated by AIers: Is it suitable to addressing the problem? is often underrated by AIers (when compared to robots) War on drugs (Nixon, 1971) Current debate on filter bubbles is not always a good thing may interfere with autonomy & with society s duty to care
What role does knowledge play? Imparting knowledge is often overrated by AIers: Is it suitable to addressing the problem? is often underrated by AIers (when compared to robots) War on drugs (Nixon, 1971) AI recommendations: Current debate on filter bubbles is not always a good thing may interfere with autonomy & with society s duty to care
What role does knowledge play? Imparting knowledge is often overrated by AIers: Is it suitable to addressing the problem? is often underrated by AIers (when compared to robots) War on drugs (Nixon, 1971) AI recommendations: Current debate on filter bubbles is not always a good thing may interferewith autonomy & with society s duty to care
What are the side-effects? Accumulation of personal data privacy violations? Conservative bias of machine learning discrimination? Affirmative bias reinforcing what you criticised?
What are the side-effects? Accumulation of personal data privacy violations? Conservative bias of machine learning discrimination? Affirmative bias reinforcing what you criticised?
What are the side-effects? Accumulation of personal data privacy violations? Conservative bias of machine learning discrimination? Affirmative bias reinforcing what you criticised?
Do all problems have solutions? Engineering Problem solving approach Dealing with social problems Goal function well-defined often involves unresolvable ethical dilemmas, continued re-negotiation Method can be black box requires fairness, transparency, accountability Decomposition, Delegation modular; can be fully delegated Usually interdependent, (at least some) participation required Ethics codes and assessments Canonical; often a burdensome afterthought starting points; ethics by design Solvability Problems can be solved. Some problems can only be addressed.
Do all problems have solutions? Engineering Problem solving approach Dealing with social problems Goal function well-defined often involves unresolvable ethical dilemmas, continued re-negotiation Method can be black box requires fairness, transparency, accountability Decomposition, Delegation modular; can be fully delegated Usually interdependent, (at least some) participation required Ethics codes and assessments Canonical; often a burdensome afterthought starting points; ethics by design Solvability Problems can be solved. Some problems can only be addressed. Solutionism
How can we make systems better? Identify and involve stakeholders throughout Employ proportionality thinking: suitable & necessary, balance of interests? Draw on the state of the art e.g. anonymisation, discrimination-aware methods Focus on whole systems instead of parts (e.g. algorithms) Ask questions, early and again Be ready to be wrong, and embrace learning!
