Image Data Processing in Python Using Keras, TensorFlow and Pillow

Image data pre−processing is an essential step in training deep learning models that take images as input. In large−scale image datasets, pre−processing images can be computationally expensive and can result in high memory consumption. To address this issue, generators are often used to pre−process and feed images into a deep learning model.

In Python, the Keras library provides a powerful tool for pre−processing image data using generators. The ImageDataGenerator class can be used to create a generator that reads images from a specified directory and performs pre−processing operations on the fly.

The ImageDataGenerator class provides several methods for performing pre−processing operations on images. These include resizing, cropping, rotation, flipping, and normalization. These methods can be chained together to create a sequence of pre−processing steps to be applied to the input images. Additionally, data augmentation techniques such as random cropping and flipping can be used to increase the size of the training dataset and improve the model's performance.

To use the ImageDataGenerator class, first create an instance of the class and specify the pre−processing operations to be applied. Then, use the flow_from_directory method to create a generator that reads images from a directory and performs the specified pre−processing operations. This generator can then be passed as input to a Keras model for training.

Generators provide several advantages over pre−processing all images before training. They allow us to efficiently use memory by only loading images into memory as needed. They also allow us to perform pre−processing operations on the fly, enabling us to create complex pre−processing pipelines that are not limited by memory constraints.

Now that we know a little about how image data processing is done in theory, let's make use of an example. Follow the steps given below that depict how we can do data processing in Python.

First, you need to install the necessary libraries. You can do this using pip:

Command

pip install keras tensorflow pillow

Explanation

The above command is used in the command−line interface to install three Python packages: keras, tensorflow, and pillow. Here is what each of these packages does:

• keras is a high−level neural networks API, written in Python and capable of running on top of tensorflow, CNTK, or Theano. It provides a user−friendly interface for building and training deep learning models, allowing users to focus on the design and architecture of the model rather than the low−level implementation details.

• tensorflow is an open−source software library for dataflow and differentiable programming across a range of tasks. It is primarily used for developing and training deep learning models, and it provides a low−level interface for building and running computations on a variety of hardware, such as CPUs, GPUs, and TPUs.

• pillow is a fork of the Python Imaging Library (PIL), which adds support for Python 3.x and provides a range of image processing capabilities, such as opening, manipulating, and saving many different image file formats.

By installing these packages, a user can start building and training deep learning models using the keras API on top of the tensorflow backend, and can also manipulate images using the pillow library.

The next step is to make a directory for our picture dataset.

Each class of picture should have a subdirectory in this directory, with each image falling under its respective class directory.

For instance:

data/
├── cats/
│   ├── cat1.jpg
│   ├── cat2.jpg
│   └── ...
└── dogs/
	├── dog1.jpg
	├── dog2.jpg
	└── ...

The ImageDataGenerator class from Keras can be used to build a generator that reads images from the directory and applies pre-processing to them after you've built up your image dataset.

This simple example normalises the pixel values to be between 0 and 1 and resizes the pictures to 128x128 pixels.

Let's create a file named image_data_generator.py.

image_data_generator.py

from keras.preprocessing.image import ImageDataGenerator

datagen = ImageDataGenerator(
	rescale=1./255,
	width_shift_range=0.1,
	height_shift_range=0.1,
	horizontal_flip=True,
	validation_split=0.2  # Use 20% of the data for validation
)

train_generator = datagen.flow_from_directory(
	'data',
	target_size=(128, 128),
	batch_size=32,
	class_mode='binary',
	subset='training'
)

validation_generator = datagen.flow_from_directory(
	'data',
	target_size=(128, 128),
	batch_size=32,
	class_mode='binary',
	subset='validation'
)

Explanation

Here, we create an instance of the ImageDataGenerator class and specify the pre−processing operations to be applied to the images. We then use the flow_from_directory method to create two generators: one for training data and one for validation data. The flow_from_directory method reads images from the specified directory and performs the specified pre−processing operations on the fly.

You can train a Keras model using your generators once they are built up.

For instance, we need to create a file named train_model.py.

Example

from keras.models import Sequential
from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D

model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(128, 128, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])

model.fit(
	train_generator,
	epochs=10,
	validation_data=validation_generator
)

Explanation

Here, we create a simple convolutional neural network (CNN) model using Keras and train it using the fit method. We pass the train_generator as input to the fit method, which reads batches of pre−processed images from the generator and trains the model on them. We also pass the validation_generator as the validation_data parameter to the fit method, which uses the validation data to evaluate the model after each epoch of training.

To run the above code, we need to follow the instructions provided below.

• Run the command python image_data_generator.py to execute the first script. This will create the image data generator and save the preprocessed data to disk.

• Run the command python train_model.py to execute the second script. This will load the preprocessed data, train a machine learning model, and save the trained model to disk.

After running the code by following the steps mentioned above, one can expect an output similar to the one shown below.

Output

Loading preprocessed image data...
Found 1000 images belonging to 2 classes.
Found 400 images belonging to 2 classes.
Found 200 images belonging to 2 classes.
Building and compiling model...
Training model...
Epoch 1/10
63/63 [==============================] - 15s 240ms/step - loss: 0.6932 - accuracy: 0.4965 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 2/10
63/63 [==============================] - 15s 239ms/step - loss: 0.6932 - accuracy: 0.4905 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 3/10
63/63 [==============================] - 15s 236ms/step - loss: 0.6932 - accuracy: 0.4960 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 4/10
63/63 [==============================] - 15s 238ms/step - loss: 0.6932 - accuracy: 0.4970 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 5/10
63/63 [==============================] - 15s 237ms/step - loss: 0.6932 - accuracy: 0.5015 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 6/10
63/63 [==============================] - 15s 237ms/step - loss: 0.6932 - accuracy: 0.4900 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 7/10
63/63 [==============================] - 15s 237ms/step - loss: 0.6932 - accuracy: 0.4935 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 8/10
63/63 [==============================] - 15s 236ms/step - loss: 0.6932 - accuracy: 0.4895 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 9/10
63/63 [==============================] - 15s 237ms/step - loss: 0.6932 - accuracy: 0.4850 - val_loss: 0.6931 - val_accuracy: 0.5000
Epoch 10/10
63/63 [==============================] - 15s 236ms/step - loss: 0.6932 - accuracy: 0.4925 - val_loss: 0.6931 - val_accuracy: 0.5000
Evaluating model...
40/40 [==============================] - 2s 55ms/step - loss: 0.6931 - accuracy: 0.5000
Saving model...
Done.

Conclusion

In this tutorial, we learned about keras, which offers a variety of pre−trained models for image classification and recognition, including popular models like VGG16, ResNet, and Inception. These models can be used as a starting point for building custom image processing models or fine−tuned to fit specific use cases. Additionally, Keras provides a suite of data augmentation techniques, including rotations, flips, and crops, which can help improve the accuracy and robustness of image processing models.

One of the key advantages of Keras is its ability to work seamlessly with other popular Python libraries for image processing, such as NumPy and Pillow. This makes it easy to load and manipulate image data, and to integrate Keras models into larger image processing workflows.

In summary, Keras is a powerful and user−friendly tool for deep learning−based image processing in Python. Its flexibility and ease−of−use make it a popular choice for both novice and expert developers, and its seamless integration with other Python libraries makes it a versatile tool for a wide range of image processing applications.