Evolution of Machine Learning and Deep Learning in AI

undefined
 
Neural Networks and Deep Learning
 
Is AI Finally Arriving?
 
CSIRO ASTRONOMY AND SPACE SCIENCE
 
Arkadi Kosmynin
 
6 December 2018
 
 
 
 
Machine Learning
Neural Networks and Deep Learning  |  Arkadi Kosmynin
Wikipedia: Machine learning (ML) is the study of algorithms and mathematical
models that computer systems use to progressively improve their performance on
a specific task.
 
Main idea: learn behaviour of algorithm (model) from data.
 
 
 
 
Machine Learning
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
Unsupervised
 
 
 
 
 
Recommended resources
 
Java
Weka 3: Data Mining Software in Java
Weka is a collection of machine learning algorithms for data mining tasks. It contains tools
for data preparation, classification, regression, clustering, association rules mining, and
visualization.
https://www.cs.waikato.ac.nz/ml/weka/
Python
scikit-learn: Machine Learning in Python
Is a free software machine learning library for the Python programming language.
It features various classification, regression and clustering algorithms and is
designed to interoperate with the Python numerical and scientific libraries
NumPy and SciPy.
https://scikit-learn.org/stable/
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
 
 
 
 
AlexNet at ImageNet 2012
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
Krizhevsky, A., Sutskever, I. and Hinton, G. E.
ImageNet Classification with Deep Convolutional Neural Networks
Advances in Neural Information Processing 25, MIT Press, Cambridge, MA
 
ImageNet Project:
14,000,000+ hand
annotated images
20,000+ categories
1,000,000+ images have
bounding boxes
ImageNet Large Scale Visual
Recognition Challenge
(ILSVRC) on 1,000 categories
“Good” error rate in 2011
was around 25%
By 2015 NNs outperformed
humans
*
 
 
 
 
 
Progress
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
 
 
 
 
How do they do it?
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
 
 
 
 
Combining neurons into a network…
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
Input
layer
 
Hidden
layer 1
 
Hidden
layer 2
 
Output
layer
 
Credit: Arden Dertat, Applied Data Learning – Part 1: Artificial Neural Networks
 
 
 
 
Gradient descent
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
Training
 
Starting from the first level, calculate
output: multiply input by weights and
apply the activation function
 
Repeat on the next levels using output
from the previous level as input
 
Calculate loss value
 
Calculate weights’ gradients for the last
level and adjust weights in directions of
the gradients
 
Repeat on the levels to the left using
known gradients on the processed levels
 
Repeat until converges or you loose
patience
 
 
 
 
Main types of neural networks
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
Multilayer Perceptron networks
The “classic” ones
 
Convolutional networks
Good for image processing
Have special layers for input filtering/subsampling
These layers work as automatic feature extractors
 
Recurrent networks
Good for sequence processing
Remember state
 
Long Short Term Memory (and Gated Recurrent Unit) networks
Even better for sequence processing
Learn what to remember longer
 
Combined networks
NNs are relatively easy to combine
 
 
 
 
Deep Learning
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
“When you hear the term deep
learning, just think of a large deep
neural net. Deep refers to the number
of layers typically and so this kind of
the popular term that’s been adopted
in the press. I think of them as deep
neural networks generally.”
Andrew Ng
Deep Learning for Building Intelligent
Computer Systems
2016
 
“For most flavors of the old generations of learning algorithms … performance will
plateau. … deep learning … is the first class of algorithms … that is scalable. …
performance just keeps getting better as you feed them more data”
 
 
 
 
 
Tools
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
Credit: Microsoft
 
 
 
 
 
Are we there yet?
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
AdaNet (Google, 2018) – learns architecture
 
“…effectively learns to perform
iterative reasoning processes that
are directly inferred from the data
in an end-to-end approach.”
98.9% (record) accuracy
Requires up to 5x less data for
training
Is computationally efficient
 
 
 
 
 
Some fun
 
A Neural Algorithm of Artistic
Style
Leon A. Gatys, Alexander S.
Ecker, Matthias Bethge
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
 
 
 
 
 
Recommended resources
 
Stanford University Computer Vision Course on YouTube
(presented mostly by Andrej Karpathy)
https://www.youtube.com/playlist?list=PLf7L7Kg8_FNxHATtLwDceyh72QQL9pvpQ
Natural Language Processing with Deep Learning (Stanford
University, Christopher D. Manning et. al.)
https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6
MIT 6.S191 Lectures on YouTube
Andrew Ng Talks
Microsoft Cognitive Toolkit
https://www.microsoft.com/en-us/cognitive-toolkit/
Keras: The Python Deep Learning library
https://keras.io/
 
 
Neural Networks and Deep Learning  |  Arkadi Kosmynin
undefined
 
Arkadi Kosmynin
t
 
+61 2 9372 4633
e
 
Arkadi.Kosmynin@csiro.au
 
Thank you
 
CSIRO ASTRONOMY AND SPACE SCIENCE
Slide Note
Embed
Share

Exploring the evolution of machine learning and deep learning in artificial intelligence through neural networks, with insights on supervised, unsupervised, and reinforcement learning. Learn about recommended resources like Java Weka and Python scikit-learn for data mining tasks. Delve into advancements in image recognition like AlexNet at ImageNet 2012 and the progress in ImageNet classification error rates from 2012 to 2017.

  • Machine Learning
  • Deep Learning
  • Neural Networks
  • Image Recognition
  • Data Mining

Uploaded on Sep 26, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Neural Networks and Deep Learning Is AI Finally Arriving? Arkadi Kosmynin 6 December 2018 CSIRO ASTRONOMY AND SPACE SCIENCE

  2. Machine Learning Wikipedia: Machine learning (ML) is the study of algorithms and mathematical models that computer systems use to progressively improve their performance on a specific task. Main idea: learn behaviour of algorithm (model) from data. Data samples Labels Optimisation problem X1 = (x11, x12, x13 x1N) y1 ?? ?? ??? y2 Training set X2 ? y3 X3 Results z4 More practical X4 y4 X5 Test set z5 y5 ?? ??2 ??? X6 ? z6 y6 Neural Networks and Deep Learning | Arkadi Kosmynin

  3. Machine Learning Supervised Unsupervised Semi-Supervised Relatively small part of data is labelled. Use mix of supervised and unsupervised learning to guess new labels and then use data samples where new labels were assigned with high enough confidence for next round of learning with a larger labelled sample set. Reinforcement Learning Software agents interacting with environment and learning in order to maximise some cumulative reward Neural Networks and Deep Learning | Arkadi Kosmynin

  4. Recommended resources Java Weka 3: Data Mining Software in Java Weka is a collection of machine learning algorithms for data mining tasks. It contains tools for data preparation, classification, regression, clustering, association rules mining, and visualization. https://www.cs.waikato.ac.nz/ml/weka/ Python scikit-learn: Machine Learning in Python Is a free software machine learning library for the Python programming language. It features various classification, regression and clustering algorithms and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy. https://scikit-learn.org/stable/ Neural Networks and Deep Learning | Arkadi Kosmynin

  5. AlexNet at ImageNet 2012 ImageNet Project: 14,000,000+ hand annotated images 20,000+ categories 1,000,000+ images have bounding boxes ImageNet Large Scale Visual Recognition Challenge (ILSVRC) on 1,000 categories Good error rate in 2011 was around 25% By 2015 NNs outperformed humans* Krizhevsky, A., Sutskever, I. and Hinton, G. E. ImageNet Classification with Deep Convolutional Neural Networks Advances in Neural Information Processing 25, MIT Press, Cambridge, MA Neural Networks and Deep Learning | Arkadi Kosmynin

  6. Progress ImageNet Classification Error (Top 5) 16.4 11.7 6.7 3.57 3.08 2.25 2012 (ALEXNET) 2013 (ZF) 2014 (GOOGLENET) 2015 (RESNET) 2016 (GOGLENET-V.4) 2017 (SE-RESNET) Neural Networks and Deep Learning | Arkadi Kosmynin

  7. How do they do it? Neural Networks and Deep Learning | Arkadi Kosmynin

  8. Combining neurons into a network Input layer Hidden layer 1 Output layer Hidden layer 2 Credit: Arden Dertat, Applied Data Learning Part 1: Artificial Neural Networks Neural Networks and Deep Learning | Arkadi Kosmynin

  9. Gradient descent Training Starting from the first level, calculate output: multiply input by weights and apply the activation function Repeat on the next levels using output from the previous level as input Calculate loss value Calculate weights gradients for the last level and adjust weights in directions of the gradients Repeat on the levels to the left using known gradients on the processed levels Repeat until converges or you loose patience Neural Networks and Deep Learning | Arkadi Kosmynin

  10. Main types of neural networks Multilayer Perceptron networks The classic ones Convolutional networks Good for image processing Have special layers for input filtering/subsampling These layers work as automatic feature extractors Recurrent networks Good for sequence processing Remember state Long Short Term Memory (and Gated Recurrent Unit) networks Even better for sequence processing Learn what to remember longer Combined networks NNs are relatively easy to combine Neural Networks and Deep Learning | Arkadi Kosmynin

  11. Deep Learning When you hear the term deep learning, just think of a large deep neural net. Deep refers to the number of layers typically and so this kind of the popular term that s been adopted in the press. I think of them as deep neural networks generally. Andrew Ng Deep Learning for Building Intelligent Computer Systems 2016 For most flavors of the old generations of learning algorithms performance will plateau. deep learning is the first class of algorithms that is scalable. performance just keeps getting better as you feed them more data Neural Networks and Deep Learning | Arkadi Kosmynin

  12. Tools Credit: Microsoft Neural Networks and Deep Learning | Arkadi Kosmynin

  13. Are we there yet? AdaNet (Google, 2018) learns architecture Stanford University, 2018 Compositional Attention Networks for Machine Reasoning Drew A. Hudson, Christopher D. Manning effectively learns to perform iterative reasoning processes that are directly inferred from the data in an end-to-end approach. 98.9% (record) accuracy Requires up to 5x less data for training Is computationally efficient Q: Do the block in front of the tiny yellow cylinder and the tiny thing that is to the right of the large green shiny object have the same color? A: No Neural Networks and Deep Learning | Arkadi Kosmynin

  14. Some fun A Neural Algorithm of Artistic Style Leon A. Gatys, Alexander S. Ecker, Matthias Bethge Neural Networks and Deep Learning | Arkadi Kosmynin

  15. Recommended resources Stanford University Computer Vision Course on YouTube (presented mostly by Andrej Karpathy) https://www.youtube.com/playlist?list=PLf7L7Kg8_FNxHATtLwDceyh72QQL9pvpQ Natural Language Processing with Deep Learning (Stanford University, Christopher D. Manning et. al.) https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6 MIT 6.S191 Lectures on YouTube Andrew Ng Talks Microsoft Cognitive Toolkit https://www.microsoft.com/en-us/cognitive-toolkit/ Keras: The Python Deep Learning library https://keras.io/ Neural Networks and Deep Learning | Arkadi Kosmynin

  16. Thank you Arkadi Kosmynin t +61 2 9372 4633 e Arkadi.Kosmynin@csiro.au CSIRO ASTRONOMY AND SPACE SCIENCE

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#