Explaining the Language in Natural Language

If you have ever talked with chatbots and used language translation tools, then you will know that these tools work exactly like a real human. This is possible because they use Natural Language Processing (NLP) techniques to understand the natural languages that humans use for communication. However, this is quite complex because each language has a different nature and structure along with various contexts.

Natural Language Processing (NLP) uses a number of techniques to get outputs as close as the natural language. Some of these techniques include −

• Lemmatization − It is the process that reduces the words to their root forms or lemma. As an example, the word bravery would be reduced to its base form brave.

• Tokenization − It is the process that breaks down the sentences into individual words called tokens. These tokens are processed by algorithms to give useful data.

• Stemming − It is the process that removes the prefix and suffix from a word. As an example, the word playing would be reduced to play by stemming the suffix.

Also, Natural Language Processing (NLP) can be used for a number of things like text summarization, sentence similarity, text classification, text generation from keywords and translation, etc.

Example 1

In this example, we will see how sentences can be classified on the basis of sentiments using NLP. For this, we will use the textblob library from the nltk module of Python.

Algorithm

• Step 1 − Import NaiveBaiyeClassifier and textblob module.

• Step 2 − Create a test set with strings, where each string has a keyword associated with it that denotes its sentiment. In the example we have directly used the string positive for positive sentiment and negative for negative sentiment.

• Step 3 − Create an object of the NBC class and pass the test set to it.

• Step 4 − Use the classify() method to pass a string and test it for its sentiment.

#import NaiveBayesClassifier and textblob module
from textblob.classifiers import NaiveBayesClassifier as NBC
from textblob import TextBlob

#prepare the test_set with corresponding sentiment
test_set = [
               ('I am tired of this attitude.', 'Negative'),
               ('He is the worst person I know!', 'Negative'),
               ('Your marks were very poor in Chemistry.', 'Positive'),
               ('I love United Kingdom.', 'Positive'),
               ('This is a great movie.', 'Positive'),
               ("What a delicious treat!", 'Positive'),
               ('I do not like her at all.', 'Negative')]


#create an object of NBC Class
model = NBC(test_set) 

#pass the string to be tested
print(model.classify("This is the best website."))
print(model.classify("I do not like Java as much as Python."))

Output

Positive
Negative

You can see in the output that the data classification method of NLP rightly assesses the sentences for their sentiments.

Example 2

In this example, we will see how a language is recognized using NLP techniques. For this, we will use the langdetect library of Python.

Algorithm

• Step 1 − Import the detect module from the langdetect library of Python.

• Step 2 − Provide the text whose language has to be detected.

• Step 3 − Print the result using the detect module.

#import the detect module
from langdetect import detect as dt

#provide the strings to be tested 
example_one = "This is a nice language."
example_two = "comment allez-vous?"   #means how are you in French

#display the result
print(dt(example_one))
print(dt(example_two))

Output

en
fr

You can see that the code returns the codes en and fr for the languages English and French respectively.

Conclusion

Although NLP does not always provide exact outputs, it is a great tool that is being used extensively in the field of data science, artificial intelligence and machine learning. Also, since it allows us to communicate with computers, it is a highly useful tool to enable illiterate and physically challenged individuals to engage with technology. With the multitude of data being generated today, use of Natural Language Processing tools is very important to efficiently analyze and use it.