Teaching Machines to Learn by Metaphors at Technion Israel Institute of Technology
This study explores the concept of teaching machines using metaphors, focusing on learning by induction, transfer learning approaches, common inductive bias, instances, and features. The research offers a new approach to transfer learning through metaphors, aiming to enhance the understanding and application of machine learning techniques.
Uploaded on Oct 09, 2024 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Teaching Machines to Learn by Metaphors Omer Levy & Shaul Markovitch Technion Israel Institute of Technology
Transfer Learning Target (New) Source (Original)
Transfer Learning Approaches Common Inductive Bias Common Instances Common Features
Example -3 -2 0 2 3
Example -3 -2 0 2 3 0 4 9
Example -3 -2 0 2 3 ??= ??2 0 4 9
Common Inductive Bias -3 -2 0 2 3 0 4 9
Common Inductive Bias -3 -2 0 2 3 0 4 9
Common Instances -3 -2 0 2 3 0 4 9
Common Features 3 2 4 9 -2 -3 ??= ??2
Metaphors Target (New) Source (Original)
Source Concept Learner Target Metaphor Learner ?? ?? ? ? +/-
? is a perfect metaphor if: 1. ? is label preserving ???? = ??? ?? 2. ? is distribution preserving ??~?? ? ??~??
Theorem If ? is a perfect metaphor - and - ? is a source hypothesis with ?? error - then - ??? = ?? ?? is a target hypothesis with ?? error
The Metaphor Theorem If ? is an ?-perfect metaphor - and - ? is a source hypothesis with ?? error - then - ??? = ?? ?? is a target hypothesis with ? + ?? error
Redefine Transfer Learning Given source and target datasets, find a target hypothesis ? such that ?? is as small as possible.
Redefine Transfer Learning Given source and target datasets, find an ?-perfect metaphor ? such that ? is as small as possible.
Concept Learning Framework Search Algorithm Hypothesis Space Data Evaluation Function
Metaphor Learning Framework Source Search Algorithm Metaphor Space Target Evaluation Function ?
Metaphor Evaluation 1. ? is label preserving ???? = ??? ?? 2. ? is distribution preserving ??~?? ? ??~??
Metaphor Evaluation 1. ? is label preserving Empirical error over target dataset 2. ? is distribution preserving Statistical distance between ? ?? and ??
Metaphor Evaluation ?? ? ??,??
Metaphor Evaluation ?? ? ??+,??+
Metaphor Evaluation ?? ? ?? ,??
Metaphor Evaluation ?? ? ??+,??++ ?? ? ?? ,??
Metaphor Spaces General Few Degrees of Freedom Representation-Specific Bias
Dictionary-Based Metaphors cheese queso
Linear Transformations ? ? = ? ?+ ?
Which metaphor space should I use? Automatic Selection of Metaphor Spaces
Which metaphor space should I use? Automatic Selection of Metaphor Spaces Occam s Razor
Which metaphor space should I use? Automatic Selection of Metaphor Spaces Occam s Razor Structural Risk Minimization
Automatic Selection of Metaphor Spaces ?1 1 ?2 2 ?3 3 ?4 4
Automatic Selection of Metaphor Spaces ?1 1 60% ?2 90% 2 ?3 91% 3 ?4 70% 4
Reference Methods Baseline Target Only Identity Metaphor Merge State-of-the-Art Frustratingly Easy Domain Adaptation Daum , 2007 MultiTask Learning Caruana, 1997; Silver et al, 2010 TrAdaBoost Dai et al, 2007
