How to create Models in Keras?

In this article, we are going to learn about how to create Models in Keras

Introduction

Keras is an open-source library in Python which provides APIs for building Artificial Neural Network Models with great flexibility. Modelling in Keras can be done either using the Functional API or the Keras Sequential Model. Keras module is also available under the popular Tensorflow Library.

Latest version and installation

The latest version of keras as of writing this article is 2.1.0.

Keras can be installed from PyPI repository using pip.

Advantages of Keras for Modelling

• Keras is used for fast implementation due to the simple API it exposes

• Keras is Flexible and robust and provides both simple and advanced workflows for building models

• Provides industry strength API for building robust models

• Can also be used for quick prototyping and research work

There are two types of Keras Models:

1. Keras Sequential Model

As the name suggest Keras Sequential model lets us build models layer by layer in a sequential manner.

This helps to create models that are limited to single input and output. It is used for building simple layers.

Example snippet to show use of Keras Sequential Model.

<div class="code-mirror  language-python" contenteditable="plaintext-only" spellcheck="false" style="outline: none; overflow-wrap: break-word; overflow-y: auto; white-space: pre-wrap;"><span class="token keyword">from</span> keras<span class="token punctuation">.</span>models <span class="token keyword">import</span> Sequential
<span class="token keyword">from</span> keras<span class="token punctuation">.</span>layers <span class="token keyword">import</span> Dense
model<span class="token operator">=</span>Sequential<span class="token punctuation">(</span><span class="token punctuation">)</span>
model<span class="token punctuation">.</span>add<span class="token punctuation">(</span>Dense<span class="token punctuation">(</span><span class="token number">128</span><span class="token punctuation">,</span>input_shape<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">8</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
model<span class="token punctuation">.</span>add<span class="token punctuation">(</span>Dense<span class="token punctuation">(</span><span class="token number">64</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
model<span class="token punctuation">.</span>add<span class="token punctuation">(</span>Dense<span class="token punctuation">(</span><span class="token number">32</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
</div>

2.Keras Functional API

Keras Functional API provides more flexibility in terms of creating models. It can help us to create models with multiple inputs and outputs. Layers can be added and shared across as in graphs.

Keras Functional API can be used as blueprint and the graph and nodes can be used wherever required.

Example of Keras Function API -

<div class="code-mirror  language-python" contenteditable="plaintext-only" spellcheck="false" style="outline: none; overflow-wrap: break-word; overflow-y: auto; white-space: pre-wrap;"><span class="token keyword">from</span> keras<span class="token punctuation">.</span>layers <span class="token keyword">import</span> Input<span class="token punctuation">,</span> Dense<span class="token punctuation">,</span>Concatenate
<span class="token keyword">from</span> keras<span class="token punctuation">.</span>models <span class="token keyword">import</span> Model
<span class="token keyword">import</span> numpy <span class="token keyword">as</span> np

<span class="token comment"># This returns a tensor</span>
inputs <span class="token operator">=</span> Input<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">784</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span>

<span class="token comment"># a layer instance is callable on a tensor, and returns a tensor</span>
x <span class="token operator">=</span> Dense<span class="token punctuation">(</span><span class="token number">64</span><span class="token punctuation">,</span> activation<span class="token operator">=</span><span class="token string">'relu'</span><span class="token punctuation">)</span><span class="token punctuation">(</span>inputs<span class="token punctuation">)</span>
x <span class="token operator">=</span> Dense<span class="token punctuation">(</span><span class="token number">64</span><span class="token punctuation">,</span> activation<span class="token operator">=</span><span class="token string">'relu'</span><span class="token punctuation">)</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span>
predictions <span class="token operator">=</span> Dense<span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">,</span> activation<span class="token operator">=</span><span class="token string">'softmax'</span><span class="token punctuation">)</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span>

<span class="token comment"># This creates a model that includes</span>
<span class="token comment"># the Input layer and three Dense layers</span>
model <span class="token operator">=</span> Model<span class="token punctuation">(</span>inputs<span class="token operator">=</span>inputs<span class="token punctuation">,</span> outputs<span class="token operator">=</span>predictions<span class="token punctuation">)</span>
model<span class="token punctuation">.</span><span class="token builtin">compile</span><span class="token punctuation">(</span>optimizer<span class="token operator">=</span><span class="token string">'rmsprop'</span><span class="token punctuation">,</span>
   loss<span class="token operator">=</span><span class="token string">'categorical_crossentropy'</span><span class="token punctuation">,</span>
   metrics<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">'accuracy'</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span>model<span class="token punctuation">.</span>summary<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>

<span class="token comment"># Keras with muliple inputs</span>
input1 <span class="token operator">=</span> Input<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">32</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
input2 <span class="token operator">=</span> Input<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">64</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
x <span class="token operator">=</span> Concatenate<span class="token punctuation">(</span>axis<span class="token operator">=</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">(</span><span class="token punctuation">[</span>input1<span class="token punctuation">,</span> input2<span class="token punctuation">]</span><span class="token punctuation">)</span>
output <span class="token operator">=</span> Dense<span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">,</span> activation<span class="token operator">=</span><span class="token string">'softmax'</span><span class="token punctuation">)</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span>
model10 <span class="token operator">=</span> Model<span class="token punctuation">(</span><span class="token punctuation">[</span>input1<span class="token punctuation">,</span> input2<span class="token punctuation">]</span><span class="token punctuation">,</span>output<span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span>model10<span class="token punctuation">.</span>summary<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
</div>

Keras using Tensorflow API [ Functional API ]

Keras is available under Tensorflow as tf.keras API. The functional API is basically centered around building directed acyclic graph (DAG) of layers. It is a graph of layers.

Example using tensorflow keras with mnist model:

<div class="code-mirror  language-python" contenteditable="plaintext-only" spellcheck="false" style="outline: none; overflow-wrap: break-word; overflow-y: auto; white-space: pre-wrap;"><span class="token keyword">import</span> tensorflow <span class="token keyword">as</span> tf
<span class="token keyword">from</span> tensorflow <span class="token keyword">import</span> keras
<span class="token keyword">from</span> tensorflow<span class="token punctuation">.</span>keras <span class="token keyword">import</span> layers

inputs <span class="token operator">=</span> keras<span class="token punctuation">.</span>Input<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">784</span><span class="token punctuation">,</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
image_inputs <span class="token operator">=</span> keras<span class="token punctuation">.</span>Input<span class="token punctuation">(</span>shape<span class="token operator">=</span><span class="token punctuation">(</span><span class="token number">32</span><span class="token punctuation">,</span> <span class="token number">32</span><span class="token punctuation">,</span> <span class="token number">3</span><span class="token punctuation">)</span><span class="token punctuation">)</span>
dense_layer <span class="token operator">=</span> layers<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">64</span><span class="token punctuation">,</span> activation<span class="token operator">=</span><span class="token string">"relu"</span><span class="token punctuation">)</span>
x <span class="token operator">=</span> dense_layer<span class="token punctuation">(</span>inputs<span class="token punctuation">)</span>
x <span class="token operator">=</span> layers<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">64</span><span class="token punctuation">,</span> activation<span class="token operator">=</span><span class="token string">"relu"</span><span class="token punctuation">)</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span>
outputs <span class="token operator">=</span> layers<span class="token punctuation">.</span>Dense<span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">(</span>x<span class="token punctuation">)</span>
model <span class="token operator">=</span> keras<span class="token punctuation">.</span>Model<span class="token punctuation">(</span>inputs<span class="token operator">=</span>inputs<span class="token punctuation">,</span> outputs<span class="token operator">=</span>outputs<span class="token punctuation">,</span> name<span class="token operator">=</span><span class="token string">"mnist"</span><span class="token punctuation">)</span>

<span class="token comment">#TRAINING AND INFERENCE</span>
<span class="token punctuation">(</span>x_train<span class="token punctuation">,</span> y_train<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">(</span>x_test<span class="token punctuation">,</span> y_test<span class="token punctuation">)</span> <span class="token operator">=</span> keras<span class="token punctuation">.</span>datasets<span class="token punctuation">.</span>mnist<span class="token punctuation">.</span>load_data<span class="token punctuation">(</span><span class="token punctuation">)</span>

x_train <span class="token operator">=</span> x_train<span class="token punctuation">.</span>reshape<span class="token punctuation">(</span><span class="token number">60000</span><span class="token punctuation">,</span> <span class="token number">784</span><span class="token punctuation">)</span><span class="token punctuation">.</span>astype<span class="token punctuation">(</span><span class="token string">"float32"</span><span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token number">255</span>
x_test <span class="token operator">=</span> x_test<span class="token punctuation">.</span>reshape<span class="token punctuation">(</span><span class="token number">10000</span><span class="token punctuation">,</span> <span class="token number">784</span><span class="token punctuation">)</span><span class="token punctuation">.</span>astype<span class="token punctuation">(</span><span class="token string">"float32"</span><span class="token punctuation">)</span> <span class="token operator">/</span> <span class="token number">255</span>
model<span class="token punctuation">.</span><span class="token builtin">compile</span><span class="token punctuation">(</span>
   loss<span class="token operator">=</span>keras<span class="token punctuation">.</span>losses<span class="token punctuation">.</span>SparseCategoricalCrossentropy<span class="token punctuation">(</span>from_logits<span class="token operator">=</span><span class="token boolean">True</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
   optimizer<span class="token operator">=</span>keras<span class="token punctuation">.</span>optimizers<span class="token punctuation">.</span>RMSprop<span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span>
   metrics<span class="token operator">=</span><span class="token punctuation">[</span><span class="token string">"accuracy"</span><span class="token punctuation">]</span><span class="token punctuation">,</span>
<span class="token punctuation">)</span>

history <span class="token operator">=</span> model<span class="token punctuation">.</span>fit<span class="token punctuation">(</span>x_train<span class="token punctuation">,</span> y_train<span class="token punctuation">,</span> batch_size<span class="token operator">=</span><span class="token number">32</span><span class="token punctuation">,</span> epochs<span class="token operator">=</span><span class="token number">10</span><span class="token punctuation">,</span> validation_split<span class="token operator">=</span><span class="token number">0.3</span><span class="token punctuation">)</span>
test_scores <span class="token operator">=</span> model<span class="token punctuation">.</span>evaluate<span class="token punctuation">(</span>x_test<span class="token punctuation">,</span> y_test<span class="token punctuation">,</span> verbose<span class="token operator">=</span><span class="token number">2</span><span class="token punctuation">)</span>

<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"Test loss:"</span><span class="token punctuation">,</span> test_scores<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
<span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"Test accuracy:"</span><span class="token punctuation">,</span> test_scores<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">)</span>
</div>

Advantages of Functional API

• In Functional API the input shape and dtype are created in advance using Input. There will be an internal check by the API for the specifications passed for the input and dtype and it also raises helpful suggestions.

• The graph can be plotted and we can access the intermediate nodes in a graph. Such as

features_list = [layer.output for layer in vgg19.layers] feat_extraction_model = keras.Model(inputs=vgg19.input, outputs=features_list)

• A functional API is serializable and can be stored as a single file since it is a data structure.

Disadvantages of Functional API

• Dynamic architectures are not supported since the Functional API treats a network as a directed acyclic graph.

Conclusion

Keras is a powerful tool/ library for deep learning model implementation both for industry and as well as for prototyping owing to its simplicity and robustness. While the Sequential API helps to build quick models, where more control is needed the Functional API comes to the rescue.