Understanding Keras Functional API for Neural Networks
Explore the Keras Functional API for building complex neural network models that go beyond sequential structures. Learn how to create computational graphs, handle non-sequential models, and understand the directed graph of computations involved in deep learning. Discover the flexibility and power of the Functional API in designing advanced neural networks.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
The Functional API in Keras CSE 4392 Neural Networks and Deep Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1
The Functional API Here is a model created using the Sequential() method: model = tf.keras.Sequential([keras.Input(shape = input_shape), layers.Dense(30, activation='tanh'), layers.Dense(50, activation='tanh'), layers.Dense(number_of_classes, activation='softmax')]) Here is the same model created using the Functional API: inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs, outputs) 2
Computational Graphs inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs, outputs) The functional API specifies a computational graph: a directed graph of computations. The computations start with the input of the model. The computations eventually produce the output of the model. The intermediate steps are expressed as applications of layers to quantities that have already been computed. The model is specified by stating its inputs and outputs. 3
Computational Graphs inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs, outputs) The functional API specifies a computational graph: a directed graph of computations. Here is the graph corresponding to the model defined above. Dense (30, tanh) Dense (50, tanh) Dense Output Layer inputs outputs x1 x2 Input Layer 4
Non-Sequential Models As the name indicates, the keras.Sequential() method defines models that are sequential. The layers form a sequence, where each layer takes its input from the previous layer and provides its output as input to the next layer. There are many commonly used types of models that are not sequential. You can define such models with the Functional API, but not with the Sequential() method. Here is an example: x1 Dense Output Layer inputs x1 x2 x3 outputs Input Layer Dense (30, tanh) Dense (50, tanh) Concat 5
Non-Sequential Models In this example, the output layer takes as input a concatenation of x1 and x2. So, the output x1 of the first hidden unit is given as input to two different layers. We cannot create such a model with a call to keras.Sequential(). x1 Dense Output Layer inputs x1 x2 x3 outputs Input Layer Dense (30, tanh) Dense (50, tanh) Concat 6
Non-Sequential Models inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = layers.Dense(50, activation='tanh')(x1) x3 = layers.Concatenate()((x1,x2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x3) model = keras.Model(inputs, outputs) This code creates the model below, using the Functional API. Note the use of a Concatenate layer, that concatenates the outputs of two different layers. x1 Dense Output Layer inputs x1 x2 x3 outputs Input Layer Dense (30, tanh) Dense (50, tanh) Concat 7
Supporting More Operations inputs = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs) x2 = tf.square(x1) outputs = layers.Dense(number_of_classes, activation="softmax")(x1) model = keras.Model(inputs, outputs) The Functional API can also be used to specify operations that are not supported by pre-defined Keras layers. We will see later that another way to do this is to define our own Keras layers. In the example above, we get x2 by squaring each value in x1. There is no pre-defined Keras layer for doing that, but there is a pre-defined Tensorflow function called square(). 8
Checking the Graph inputs1 = keras.Input(shape=input_shape) inputs2 = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs1) x2 = layers.Concatenate()((x1,inputs2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs1, outputs) The keras.Model() method checks whether the computational graph specifies how to map inputs to outputs. In the code above, the keras.Model() arguments specify that the input is inputs1, and the output is outputs. However, knowing inputs1 does not suffice to compute outputs. If you run this code, keras.Model() will produce an error. 9
Checking the Graph inputs1 = keras.Input(shape=input_shape) inputs2 = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs1) x2 = layers.Concatenate()((x1,inputs2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model(inputs1, outputs) This code defines the model shown below. As we see, inputs1 is not sufficient to compute outputs. We also need inputs2. inputs1 x1 Input Layer 1 Dense (30, tanh) Dense x2 outputs concat Output Layer inputs2 Input Layer 2 10
Checking the Graph inputs1 = keras.Input(shape=input_shape) inputs2 = keras.Input(shape=input_shape) x1 = layers.Dense(30, activation='tanh')(inputs1) x2 = layers.Concatenate()((x1,inputs2)) outputs = layers.Dense(number_of_classes, activation="softmax")(x2) model = keras.Model((inputs1, inputs2), outputs) We fix the error by listing both inputs1 and inputs2 as model input. Knowing both inputs the model can compute output. inputs1 x1 Input Layer 1 Dense (30, tanh) Dense x2 outputs concat Output Layer inputs2 Input Layer 2 11
Functional API: Recap So far we have been using the Sequential API to define models in Keras. However, the Sequential API has limitations. We can only define sequential models. We are getting to a point in the semester where we will study types of models that are not sequential. Encoder-Decoder RNNs for sequence-to-sequence translation. Transformers. The Functional API allows us to define and use such models. 12
