Emotion Based Music Player: A Python Project in Machine Learning

Introduction

Music is a universal language. Despite cultures and languages, it connects emotions and brings people together. Today, you can personalize your music depending on your moods, emotions, and preferences.

This article will teach us how to build our emotion-based music player. The idea is simple to recognize a user's emotion and provide a customized playlist. For this, we need some machine language algorithms. The algorithms will recognize emotion patterns and the user's niche to suggest songs that perfectly match their mood.

Technology and music have enough potential to heal emotions through the power of music.

This project will offer an emotion-based music system. The system will sort down songs that fit your mood, and you don't have to search for songs. An emotion-based music player will help you manage mental stress. It would also help you to manage emotions.

Choosing a Dataset

A dataset is essential while training a model using machine learning algorithms. A dataset in this project is essential. It contains data such as the state of emotion and information about various songs.

Creating a dataset contains various emotions such as happiness, calmness, anger, sadness, loneliness, etc. The dataset can help train the model to understand and predict the user's emotional state. Then the model will classify the emotions and suggest appropriate songs.

The dataset will help the model recommend songs with positive lyrics and upbeat, cheerful songs. Similarly, songs with a low pitch, calmness, and slow pace are considered melancholic. This type of song describes sadness, loneliness, or emptiness.

Here we will be using two datasets −

• Face Emotion Recognition Dataset (FER 2013) for detecting the facial emotions

  https://www.kaggle.com/datasets/msambare/fer2013

• Spotify songs dataset

  https://www.kaggle.com/datasets/musicblogger/spotify-music-data-to-identify-themoods

Preprocessing The Data

Training a machine learning model needs both audio and emotional data. You need a person who has some knowledge of music theory and signal processing.

In Audio Data, you need information like tempo, pitch, and rhythm. It will help the model to understand the musical data. Extracting this data is a challenging task. More accurate data may be needed to maintain the accuracy level of the model.

Loading the Datasets

img_shape = 48
batch_size = 64

train_data_path = '/Users/someswarpal/Downloads/archive-2/train/'
test_data_path = '/Users/someswarpal/Downloads/archive-2/test/'

Data Preprocessing

Example

train_preprocessor = ImageDataGenerator(
   rescale = 1 / 255.,
   # Data Augmentation
   rotation_range=10,
   zoom_range=0.2,
   width_shift_range=0.1,
   height_shift_range=0.1,
   horizontal_flip=True,
   fill_mode='nearest',
)
test_preprocessor = ImageDataGenerator(
   rescale = 1 / 255.,
)

train_data = train_preprocessor.flow_from_directory(
   train_data_path,
   class_mode="categorical",
   target_size=(img_shape,img_shape),
   color_mode='rgb',
   shuffle=True,
   batch_size=batch_size,
   subset='training',
)

test_data = test_preprocessor.flow_from_directory(
   test_data_path,
   class_mode="categorical",
   target_size=(img_shape,img_shape),
   color_mode="rgb",
   shuffle=False,
   batch_size=batch_size,
)

Output

Found 28709 images belonging to 7 classes.
Found 7178 images belonging to 7 classes.

Machine Learning Model

The machine learning model for the emotion-based music player project uses supervised learning. It involves training the model with pre-labeled data. The labeled data includes the mental states of people and the songs that go with those moods.

The decision tree method classifies information by breaking it up into smaller and smaller groups based on certain rules. Branching the decision lets the model make a series of choices based on the information it gets and guess how the user feels.

Creating the CNN Model

def Create_CNN_Model():

   model = Sequential()
   
   #CNN1
   model.add(Conv2D(32, (3,3), activation='relu', input_shape=(img_shape, img_shape, 3)))
   model.add(BatchNormalization())
   model.add(Conv2D(64,(3,3), activation='relu', padding='same'))
   model.add(BatchNormalization())
   model.add(MaxPooling2D(pool_size=(2,2), padding='same'))
   model.add(Dropout(0.25))
   
   #CNN2
   model.add(Conv2D(64, (3,3), activation='relu', ))
   model.add(BatchNormalization())
   model.add(Conv2D(128,(3,3), activation='relu', padding='same'))
   model.add(BatchNormalization())
   model.add(MaxPooling2D(pool_size=(2,2), padding='same'))
   model.add(Dropout(0.25))
   
   #CNN3
   model.add(Conv2D(128, (3,3), activation='relu'))
   model.add(BatchNormalization())
   model.add(Conv2D(256,(3,3), activation='relu', padding='same'))
   model.add(BatchNormalization())
   model.add(MaxPooling2D(pool_size=(2,2), padding='same'))
   model.add(Dropout(0.25))
   
   #Output
   model.add(Flatten())
   
   model.add(Dense(1024, activation='relu'))
   model.add(BatchNormalization())
   model.add(Dropout(0.25))
   
   model.add(Dense(512, activation='relu'))
   model.add(BatchNormalization())
   model.add(Dropout(0.25))
   
   model.add(Dense(256, activation='relu'))
   model.add(BatchNormalization())
   model.add(Dropout(0.25))
   
   model.add(Dense(128, activation='relu'))
   model.add(BatchNormalization())
   model.add(Dropout(0.25))
   
   model.add(Dense(64, activation='relu'))
   model.add(BatchNormalization())
   model.add(Dropout(0.25))
   
   model.add(Dense(32, activation='relu'))
   model.add(BatchNormalization())
   model.add(Dropout(0.25))
   
   model.add(Dense(7,activation='softmax'))
   
return model

Compiling the CNN Model

CNN_Model = Create_CNN_Model()
CNN_Model.compile(optimizer="adam", loss='categorical_crossentropy',
metrics=['accuracy'])

Training the Model

CNN_history = CNN_Model.fit( train_data , validation_data= test_data , epochs=50,
batch_size= batch_size,
callbacks=callbacks, steps_per_epoch= steps_per_epoch,
validation_steps=validation_steps)

Accuracy and Graph

Example

CNN_Score = CNN_Model.evaluate(test_data)

print(" Test Loss: {:.5f}".format(CNN_Score[0]))
print("Test Accuracy: {:.2f}%".format(CNN_Score[1] * 100))

def plot_curves(history):

   loss = history.history["loss"]
   val_loss = history.history["val_loss"]
   
   accuracy = history.history["accuracy"]
   val_accuracy = history.history["val_accuracy"]
   
   epochs = range(len(history.history["loss"]))
   
   plt.figure(figsize=(15,5))
   
   #plot loss
   plt.subplot(1, 2, 1)
   plt.plot(epochs, loss, label = "training_loss")
   plt.plot(epochs, val_loss, label = "val_loss")
   plt.title("Loss")
   plt.xlabel("epochs")
   plt.legend()
   
   #plot accuracy
   plt.subplot(1, 2, 2)
   plt.plot(epochs, accuracy, label = "training_accuracy")
   plt.plot(epochs, val_accuracy, label = "val_accuracy")
   plt.title("Accuracy")
   plt.xlabel("epochs")
   plt.legend()
   plot_curves(CNN_history)
   
   #plt.tight_layout()

Output

Training the Testing Emotion

CNN_Predictions = CNN_Model.predict(test_data)

# Choosing highest probalbilty class in every prediction
CNN_Predictions = np.argmax(CNN_Predictions, axis=1)

Testing on Random Dataset

Example

test_preprocessor = ImageDataGenerator(
   rescale = 1 / 255.,
)

test_generator = test_preprocessor.flow_from_directory(
   test_data_path,
   class_mode="categorical",
   target_size=(img_shape,img_shape),
   color_mode="rgb",
   shuffle=True,
   batch_size=batch_size,
)

Random_batch = np.random.randint(0, len(test_generator) - 1)

Random_Img_Index = np.random.randint(0, batch_size - 1 , 10)

fig, axes = plt.subplots(
   nrows=2, ncols=5, figsize=(25, 10),
   subplot_kw={'xticks': [], 'yticks': []}
)

for i, ax in enumerate(axes.flat):
   Random_Img = test_generator[Random_batch][0][Random_Img_Index[i]]
   Random_Img_Label = np.argmax(test_generator[Random_batch][1][Random_Img_Index[i]])
   
   Model_Prediction = np.argmax(CNN_Model.predict( tf.expand_dims(Random_Img,
   axis=0) , verbose=0))
   ax.imshow(Random_Img)
   if Emotion_Classes[Random_Img_Label] == Emotion_Classes[Model_Prediction]:
      color = "green"
   else:
      color = "red"
   ax.set_title(f"True: {Emotion_Classes[Random_Img_Label]}\nPredicted:
   {Emotion_Classes[Model_Prediction]}", color=color)
plt.show()
plt.tight_layout()

Output

Emotion Detection

Emotion detection is the main part of the emotion-based music player system. Proper emotion detection helps to predict the emotion of the user accurately.

Emotions can be captured using a webcam or selfie camera. This will provide real-time emotions of the user.

Python has a library, OpenCV, for processing emotions through pictures or videos. OpenCV helps analyze facial expression changes. This is done by tracking the movement of the eyebrows, eyes, and mouth shape of the user.

Mapping real-time facial expressions will accurately provide personalized recommendations based on the user's emotions.

Loading the Emotion Dataset

Music_Player = pd.read_csv("/Users/someswarpal/Downloads/data_moods.csv")
Music_Player = Music_Player[['name','artist','mood','popularity']]
Music_Player.head()

Displaying the Contents of The Dataset

Play = Music_Player[Music_Player['mood'] == 'Happy' ]
Play = Play.sort_values(by="popularity", ascending=False)
Play = Play[:5].reset_index(drop=True)
display(Play)

Genre Detection

Genre detection is also important for this project. It allows the model to learn the user's musical preferences. By using genre detection, the model can suggest similar genre songs to the user. It helps user expand their musical tastes.

Recommending the Music

The ML model predicts the user's emotions and recommends songs based on the emotions.

A well-trained ML model ensured a well-suited song based on the user's mood and emotions. It ensures the recommended songs are well-suited to the user's mood and emotions.

# Making Songs Recommendations Based on Predicted Class
def Recommend_Songs(pred_class):

   if( pred_class=='Disgust' ):
      Play = Music_Player[Music_Player['mood'] =='Sad' ]
      Play = Play.sort_values(by="popularity", ascending=False)
      Play = Play[:5].reset_index(drop=True)
      display(Play)
   
   if( pred_class=='Happy' or pred_class=='Sad' ):
      Play = Music_Player[Music_Player['mood'] =='Happy' ]
      Play = Play.sort_values(by="popularity", ascending=False)
      Play = Play[:5].reset_index(drop=True)
      display(Play)
   
   if( pred_class=='Fear' or pred_class=='Angry' ):
      Play = Music_Player[Music_Player['mood'] =='Calm' ]
      Play = Play.sort_values(by="popularity", ascending=False)
      Play = Play[:5].reset_index(drop=True)
      display(Play)
   
   if( pred_class=='Surprise' or pred_class=='Neutral' ):
      Play = Music_Player[Music_Player['mood'] =='Energetic' ]
      Play = Play.sort_values(by="popularity", ascending=False)
      Play = Play[:5].reset_index(drop=True)
      display(Play)

Print list of moods and ask user to select one

print("Select a mood by entering a number:\n1. Disgust\n2. Happy/Sad\n3. Fear/Angry\n4.
Surprise/Neutral")
selected_mood = int(input("Enter a number for the mood you want to select: "))

# Display recommended songs based on selected mood
if selected_mood == 1:
   Recommend_Songs('Disgust')
elif selected_mood == 2:
   Recommend_Songs('Happy')
elif selected_mood == 3:
   Recommend_Songs('Fear')
elif selected_mood == 4:
   Recommend_Songs('Surprise')
else:
   print("Invalid input. Please enter a number between 1 and 4.")

Discovering a New Song

Song finding is essential to the emotion-based music player project. It lets users discover new songs that fit their tastes. The machine learning model offers new or lesser-known songs based on the user's listening history.

Suggesting a new song improves the user's listening experience and encourages musical exploration. Song discovery can also help discover songs of other languages with the same genre and mood.

Adding Additional Features

The essential features of the music system are emotion detection, genre detection, and song recommendation. You can update your music system by adding up several other features like

• Search for a song based on emotion detection.

• Rate the song based on their emotion.

• Add suggestions for better training the model.

• Share your emotion-based playlist with others or make it public.

• Save or download the songs for future use. (can be a premium version)

Conclusion

An emotion-based music player perfectly combines emotion, technology, and music. It provides a unique music experience based on their feelings and musical taste. The model uses ML techniques to suggest the best songs matching the user's emotions.

A well-trained model ensures that the songs it suggests match the user's feelings. The most important parts of the music system are feeling detection, genre recognition, and song suggestions.

You can, however, look for songs based on how you feel, rate them, share playlists, and save or download music. Emotion-based music players can help you calm down and deal with mental worry.