Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Explaining the Language in Natural Language
If you have ever talked with chatbots and used language translation tools, then you will know that these tools work exactly like a real human. This is possible because they use Natural Language Processing (NLP) techniques to understand the natural languages that humans use for communication. However, this is quite complex because each language has a different nature and structure along with various contexts.
Natural Language Processing (NLP) uses a number of techniques to get outputs as close as the natural language. Some of these techniques include −
Lemmatization − It is the process that reduces the words to their root forms or lemma. As an example, the word bravery would be reduced to its base form brave.
Tokenization − It is the process that breaks down the sentences into individual words called tokens. These tokens are processed by algorithms to give useful data.
Stemming − It is the process that removes the prefix and suffix from a word. As an example, the word playing would be reduced to play by stemming the suffix.
Also, Natural Language Processing (NLP) can be used for a number of things like text summarization, sentence similarity, text classification, text generation from keywords and translation, etc.
Example 1
In this example, we will see how sentences can be classified on the basis of sentiments using NLP. For this, we will use the textblob library from the nltk module of Python.
Algorithm
Step 1 − Import NaiveBaiyeClassifier and textblob module.
Step 2 − Create a test set with strings, where each string has a keyword associated with it that denotes its sentiment. In the example we have directly used the string positive for positive sentiment and negative for negative sentiment.
Step 3 − Create an object of the NBC class and pass the test set to it.
Step 4 − Use the classify() method to pass a string and test it for its sentiment.
#import NaiveBayesClassifier and textblob module
from textblob.classifiers import NaiveBayesClassifier as NBC
from textblob import TextBlob
#prepare the test_set with corresponding sentiment
test_set = [
('I am tired of this attitude.', 'Negative'),
('He is the worst person I know!', 'Negative'),
('Your marks were very poor in Chemistry.', 'Positive'),
('I love United Kingdom.', 'Positive'),
('This is a great movie.', 'Positive'),
("What a delicious treat!", 'Positive'),
('I do not like her at all.', 'Negative')]
#create an object of NBC Class
model = NBC(test_set)
#pass the string to be tested
print(model.classify("This is the best website."))
print(model.classify("I do not like Java as much as Python."))
Output
Positive Negative
You can see in the output that the data classification method of NLP rightly assesses the sentences for their sentiments.
Example 2
In this example, we will see how a language is recognized using NLP techniques. For this, we will use the langdetect library of Python.
Algorithm
Step 1 − Import the detect module from the langdetect library of Python.
Step 2 − Provide the text whose language has to be detected.
Step 3 − Print the result using the detect module.
#import the detect module from langdetect import detect as dt #provide the strings to be tested example_one = "This is a nice language." example_two = "comment allez-vous?" #means how are you in French #display the result print(dt(example_one)) print(dt(example_two))
Output
en fr
You can see that the code returns the codes en and fr for the languages English and French respectively.
Conclusion
Although NLP does not always provide exact outputs, it is a great tool that is being used extensively in the field of data science, artificial intelligence and machine learning. Also, since it allows us to communicate with computers, it is a highly useful tool to enable illiterate and physically challenged individuals to engage with technology. With the multitude of data being generated today, use of Natural Language Processing tools is very important to efficiently analyze and use it.
44 Views
