- Trending Categories
- Data Structure
- Networking
- RDBMS
- Operating System
- Java
- MS Excel
- iOS
- HTML
- CSS
- Android
- Python
- C Programming
- C++
- C#
- MongoDB
- MySQL
- Javascript
- PHP
- Physics
- Chemistry
- Biology
- Mathematics
- English
- Economics
- Psychology
- Social Studies
- Fashion Studies
- Legal Studies

- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who

# Non-Negative Matrix Factorization

## Introduction

Non-Negative Matrix Factorization (NMF) is a supervised algorithm used to represent data into lower dimensions which reduces the number of features while preserving enough basic information to construct the original matrix from the reduced feature space.

In this article, we will be going explore more about NMF and how it can be useful.

## Non-Negative Matrix Factorization

NMF is used to reduce the dimensions of the input matrix or corpus. It uses factor analysis which gives less importance to less relevant words. The decomposition of the original matrix(which is a non-negative matrix) thus creates a product of two non-negative coefficients with a rank lower than that of the original matrix.

### Importance of NMF

NMF belongs to the category of algebra-based algorithms that are used to determine the hidden state.

It can be used for topic modeling TF-IDF.

NMF can easily extract sparsely populated data and factors.

Below is a representation of Non-Negative matrix factorization in topic modeling

Matrix 1 (H+): Topic and words

Matrix 3 (W+): Documents and topics

## Representation of NMF

Let us have one input matrix M of shape p x q. The matrix factorization topic modeling will decompose the matrix M into two matrices R and S of shapes p x t and t x q.

Thus, we have three matrices as described below.

Matrix M − shape (p x q) Represents the document term matrix

Matrix R − shape (p x t ) Represents the word embedding matrix

Matrix S − shape (t x q ) Weight of each word in a sentence is represented in each column

## Mathematic modeling of NMF

NMF is an unsupervised ML technique that computes the distance between the elements. There are different methods to calculate the distance. Two such methods are discussed below.

KL Divergence − It is used to determine the closeness of two distributions on quantitative aspects. Thus, if two words are similar and close the value of KL divergence tends to zero otherwise it increases.

The general formula for KL Divergence is given as

$$\mathrm{D_{K\:L}(p(x)||q\left ( x \right ))=\sum _{x\epsilon X}p(x)ln\frac{p(x)}{q(x))}}$$

Euclidean Distance - The distance between two points in space can be given as

$$\mathrm{d(p,q)=\sqrt{(q_{1}-p_{1})^{2}+(q_{2}-p_{2})^{2}}}$$

### Advantages of Non-Negative Matrix Factorization

It can handle missing data while minimizing the cost function and does not consider missing data as zeros.

It can work by breaking down a higher-complex matrix into a lower-dimension matrix. It is considered better than LDA.

## Conclusion

Non Negative matrix Factorization is a widely used technique for dimension reduction, especially in fields related to Natural Language and Machine Learning. It is faster, and easier and produces better results than those of its rivals like LDA.

- Related Articles
- Program to find maximum non negative product in a matrix in Python
- Non-negative set subtraction in JavaScript
- Non-negative Integers without Consecutive Ones in C++
- Finding the longest non-negative sum sequence using JavaScript
- Number of non-negative integral solutions of sum equation in C++
- Replace Negative and Positive Matrix Elements with 0 & 1 in Java
- Finding square root of a non-negative number without using Math.sqrt() JavaScript
- How to convert negative values in a matrix to 0 in R?
- Return the non-negative square-root of an array element-wise in Numpy
- How to convert values in alternate rows to negative in matrix in R?
- Return the negative infinity Norm of the matrix in Linear Algebra in Python
- Python Program that filters out non-empty rows of a matrix
- Count Negative Numbers in a Column-Wise and Row-Wise Sorted Matrix using Python?
- Minimum Factorization in C++
- What is prime factorization?