How can Keras be used for feature extraction using a sequential model using Python?

Keras is a high-level deep learning API built on top of TensorFlow that simplifies building and training neural networks. One powerful application is feature extraction, where we extract intermediate layer outputs from a trained model to use as features for other tasks.

What is Feature Extraction?

Feature extraction involves using a pre-trained or partially trained model to extract meaningful representations from data. Instead of using the final output, we capture intermediate layer outputs that contain learned features.

Installation

Install TensorFlow which includes Keras ?

pip install tensorflow

Creating a Sequential Model for Feature Extraction

First, let's create a sequential model with convolutional layers ?

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

print("Creating sequential model...")
initial_model = keras.Sequential([
    keras.Input(shape=(250, 250, 3)),
    layers.Conv2D(32, 5, strides=2, activation="relu", name="conv1"),
    layers.Conv2D(32, 3, activation="relu", name="conv2"),
    layers.Conv2D(32, 3, activation="relu", name="conv3"),
])

print("Model architecture:")
initial_model.summary()

Creating sequential model...
Model architecture:
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv1 (Conv2D)              (None, 123, 123, 32)     2432      
                                                                 
 conv2 (Conv2D)              (None, 121, 121, 32)     9248      
                                                                 
 conv3 (Conv2D)              (None, 119, 119, 32)     9248      
                                                                 
=================================================================
Total params: 20,928
Trainable params: 20,928
Non-trainable params: 0
_________________________________________________________________

Building the Feature Extractor

Now we create a feature extractor that outputs intermediate layer features ?

print("Creating feature extractor...")
feature_extractor = keras.Model(
    inputs=initial_model.inputs,
    outputs=[layer.output for layer in initial_model.layers],
)

print("Feature extractor created successfully!")
print("Number of output layers:", len(feature_extractor.outputs))

Creating feature extractor...
Feature extractor created successfully!
Number of output layers: 3

Extracting Features from Test Data

Let's extract features using sample input data ?

# Create sample input data
x = tf.ones((1, 250, 250, 3))
print("Input shape:", x.shape)

# Extract features
print("Extracting features...")
features = feature_extractor(x)

# Display feature shapes
for i, feature in enumerate(features):
    print(f"Layer {i+1} output shape: {feature.shape}")

Input shape: (1, 250, 250, 3)
Extracting features...
Layer 1 output shape: (1, 123, 123, 32)
Layer 2 output shape: (1, 121, 121, 32)
Layer 3 output shape: (1, 119, 119, 32)

Accessing Specific Layer Features

You can also extract features from specific layers ?

# Extract features from specific layers only
specific_extractor = keras.Model(
    inputs=initial_model.inputs,
    outputs=[initial_model.layers[0].output, initial_model.layers[2].output]
)

specific_features = specific_extractor(x)
print("First layer features shape:", specific_features[0].shape)
print("Third layer features shape:", specific_features[1].shape)

First layer features shape: (1, 123, 123, 32)
Third layer features shape: (1, 119, 119, 32)

Key Benefits

• Transfer Learning ? Use pre-trained features for new tasks

• Dimensionality Reduction ? Extract compact representations

• Feature Analysis ? Understand what the model learns at each layer

• Computational Efficiency ? Reuse learned features instead of training from scratch

Conclusion

Keras makes feature extraction straightforward by creating a new Model with the same inputs but different outputs. This technique is essential for transfer learning and understanding deep learning models. The extracted features can be used for classification, clustering, or visualization tasks.