Demonstrate a basic implementation of 'tf.keras.layers.Dense' in Python

TensorFlow is a machine learning framework provided by Google. It is an open-source framework used with Python to implement algorithms, deep learning applications, and much more. The Dense layer is one of the most fundamental layers in neural networks, performing matrix multiplication between inputs and weights.

The tensorflow package can be installed on Windows using the below line of code ?

pip install tensorflow

Keras is a high-level deep learning API written in Python that runs on top of TensorFlow. It provides essential abstractions for building neural networks quickly and efficiently.

Importing Required Libraries

First, let's import the necessary TensorFlow and Keras modules ?

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
print("TensorFlow version:", tf.__version__)

Creating a Custom Dense Layer

We can create a custom Dense layer by inheriting from layers.Layer. This demonstrates how the built-in Dense layer works internally ?

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

class CustomDense(layers.Layer):
    def __init__(self, units=32):
        super(CustomDense, self).__init__()
        self.units = units
    
    def build(self, input_shape):
        self.w = self.add_weight(
            shape=(input_shape[-1], self.units),
            initializer="random_normal",
            trainable=True,
        )
        self.b = self.add_weight(
            shape=(self.units,), 
            initializer="random_normal", 
            trainable=True
        )
    
    def call(self, inputs):
        return tf.matmul(inputs, self.w) + self.b

# Create input and output
inputs = keras.Input((4,))
outputs = CustomDense(10)(inputs)

print("Creating Keras model with custom Dense layer...")
model = keras.Model(inputs, outputs)
print("Model created successfully!")
print("Model summary:")
print(model.summary())

Using Built-in Dense Layer

Here's how to use TensorFlow's built-in Dense layer, which is equivalent to our custom implementation ?

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# Using built-in Dense layer
inputs = keras.Input((4,))
dense_layer = layers.Dense(10, activation='linear')
outputs = dense_layer(inputs)

model = keras.Model(inputs, outputs)
print("Built-in Dense layer model created!")
print("Model summary:")
print(model.summary())

Key Components Explained

The Dense layer implementation involves three key methods:

• __init__: Initializes the layer with the number of units (neurons)
• build: Creates the weight matrix and bias vector based on input shape
• call: Performs the forward pass computation (matrix multiplication + bias)

Comparison

Conclusion

The Dense layer performs linear transformation using weights and biases. Creating custom layers helps understand the internal workings, while built-in layers provide optimized implementations for production use.