Understanding Keras Functional API for Neural Networks

Slide Note
Embed
Share

Explore the Keras Functional API for building complex neural network models that go beyond sequential structures. Learn how to create computational graphs, handle non-sequential models, and understand the directed graph of computations involved in deep learning. Discover the flexibility and power of the Functional API in designing advanced neural networks.

dale
dale Follow

Uploaded on May 15, 2024 | 1 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

  1. The Functional API in Keras CSE 4392 Neural Networks and Deep Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1

  2. The Functional API Here is a model created using the Sequential() method: model = tf.keras.Sequential([keras.Input(shape = input_shape), layers.Dense(30, activation='tanh'), layers.Dense(50, activation='tanh'), layers.Dense(number_of_classes, activation='softmax')]) Here is the same model created using the Functional API: inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs, outputs) 2

  3. Computational Graphs inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs, outputs) The functional API specifies a computational graph: a directed graph of computations. The computations start with the input of the model. The computations eventually produce the output of the model. The intermediate steps are expressed as applications of layers to quantities that have already been computed. The model is specified by stating its inputs and outputs. 3

  4. Computational Graphs inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs, outputs) The functional API specifies a computational graph: a directed graph of computations. Here is the graph corresponding to the model defined above. Dense (30, tanh) Dense (50, tanh) Dense Output Layer inputs outputs x1 x2 Input Layer 4

  5. Non-Sequential Models As the name indicates, the keras.Sequential() method defines models that are sequential. The layers form a sequence, where each layer takes its input from the previous layer and provides its output as input to the next layer. There are many commonly used types of models that are not sequential. You can define such models with the Functional API, but not with the Sequential() method. Here is an example: x1 Dense Output Layer inputs x1 x2 x3 outputs Input Layer Dense (30, tanh) Dense (50, tanh) Concat 5

  6. Non-Sequential Models In this example, the output layer takes as input a concatenation of x1 and x2. So, the output x1 of the first hidden unit is given as input to two different layers. We cannot create such a model with a call to keras.Sequential(). x1 Dense Output Layer inputs x1 x2 x3 outputs Input Layer Dense (30, tanh) Dense (50, tanh) Concat 6

  7. Non-Sequential Models inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) x3 = layers.Concatenate()((x1,x2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x3) model = keras.Model(inputs, outputs) This code creates the model below, using the Functional API. Note the use of a Concatenate layer, that concatenates the outputs of two different layers. x1 Dense Output Layer inputs x1 x2 x3 outputs Input Layer Dense (30, tanh) Dense (50, tanh) Concat 7

  8. Supporting More Operations inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = tf.square(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x1) model = keras.Model(inputs, outputs) The Functional API can also be used to specify operations that are not supported by pre-defined Keras layers. We will see later that another way to do this is to define our own Keras layers. In the example above, we get x2 by squaring each value in x1. There is no pre-defined Keras layer for doing that, but there is a pre-defined Tensorflow function called square(). 8

  9. Checking the Graph inputs1 = keras.Input(shape=input_shape) inputs2 = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs1) x2 = layers.Concatenate()((x1,inputs2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs1, outputs) The keras.Model() method checks whether the computational graph specifies how to map inputs to outputs. In the code above, the keras.Model() arguments specify that the input is inputs1, and the output is outputs. However, knowing inputs1 does not suffice to compute outputs. If you run this code, keras.Model() will produce an error. 9

  10. Checking the Graph inputs1 = keras.Input(shape=input_shape) inputs2 = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs1) x2 = layers.Concatenate()((x1,inputs2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs1, outputs) This code defines the model shown below. As we see, inputs1 is not sufficient to compute outputs. We also need inputs2. inputs1 x1 Input Layer 1 Dense (30, tanh) Dense x2 outputs concat Output Layer inputs2 Input Layer 2 10

  11. Checking the Graph inputs1 = keras.Input(shape=input_shape) inputs2 = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs1) x2 = layers.Concatenate()((x1,inputs2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model((inputs1, inputs2), outputs) We fix the error by listing both inputs1 and inputs2 as model input. Knowing both inputs the model can compute output. inputs1 x1 Input Layer 1 Dense (30, tanh) Dense x2 outputs concat Output Layer inputs2 Input Layer 2 11

  12. Functional API: Recap So far we have been using the Sequential API to define models in Keras. However, the Sequential API has limitations. We can only define sequential models. We are getting to a point in the semester where we will study types of models that are not sequential. Encoder-Decoder RNNs for sequence-to-sequence translation. Transformers. The Functional API allows us to define and use such models. 12

Related


Introduction to Keras for Deep Learning

Introduction to Keras for Deep Learning

koslow_s
Introduction to Neural Networks in IBM SPSS Modeler 14.2

Introduction to Neural Networks in IBM SPSS Modeler 14.2

anouk
Understanding Artificial Neural Networks From Scratch

Understanding Artificial Neural Networks From Scratch

inara
Understanding Advanced Classifiers and Neural Networks

Understanding Advanced Classifiers and Neural Networks

elwynto
Graph Neural Networks

Graph Neural Networks

kornelia
Machine Learning and Artificial Neural Networks for Face Verification: Overview and Applications

Machine Learning and Artificial Neural Networks for Face Verification: Overview and Applications

lipp195
Exploring Neural Quantum States and Symmetries in Quantum Mechanics

Exploring Neural Quantum States and Symmetries in Quantum Mechanics

adelinda
Understanding Spiking Neurons and Spiking Neural Networks

Understanding Spiking Neurons and Spiking Neural Networks

kooper
Understanding Neural Network Watermarking Technologies

Understanding Neural Network Watermarking Technologies

alad
Assistive Speech System for Individuals with Speech Impediments Using Neural Networks

Assistive Speech System for Individuals with Speech Impediments Using Neural Networks

albi
Understanding Network Analysis: Whole Networks vs. Ego Networks

Understanding Network Analysis: Whole Networks vs. Ego Networks

kayliee
Understanding Back-Propagation Algorithm in Neural Networks

Understanding Back-Propagation Algorithm in Neural Networks

kamilah
New Approaches in Learning Complex-Valued Neural Networks

New Approaches in Learning Complex-Valued Neural Networks

eliyahu
A Deep Dive into Neural Network Units and Language Models

A Deep Dive into Neural Network Units and Language Models

rowyn
Understanding Mechanistic Interpretability in Neural Networks

Understanding Mechanistic Interpretability in Neural Networks

maksymilian
Computational Physics (Lecture 18)

Computational Physics (Lecture 18)

lorelai
Understanding Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)

Understanding Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)

imaan
Introduction to Deep Learning: Neural Networks and Multilayer Perceptrons

Introduction to Deep Learning: Neural Networks and Multilayer Perceptrons

ellie
Advancing Physics-Informed Machine Learning for PDE Solving

Advancing Physics-Informed Machine Learning for PDE Solving

luciana
Learning a Joint Model of Images and Captions with Neural Networks

Learning a Joint Model of Images and Captions with Neural Networks

cosimo
Detecting Image Steganography Using Neural Networks

Detecting Image Steganography Using Neural Networks

cynthia
Neural Networks for Learning Relational Information

Neural Networks for Learning Relational Information

tyranba
Evolution of Neural Networks through Neuroevolution by Ken Stanley

Evolution of Neural Networks through Neuroevolution by Ken Stanley

kol_v
Introduction to Neural Networks in Machine Learning

Introduction to Neural Networks in Machine Learning

ame_sho

More Related Content