Find frequency of each word in a string in Python

As a part of text analytics, we frequently need to count words and assign weightage to them for processing in various algorithms. In this article, we will explore three different approaches to find the frequency of each word in a given sentence.

Using Counter from Collections

The Counter class from the collections module provides an elegant way to count word frequencies. It creates a dictionary where keys are words and values are their counts ?

from collections import Counter

line_text = "Learn and practice and learn to practice"
freq = Counter(line_text.split())
print("Word frequencies:", freq)
print("Most common words:", freq.most_common())

Word frequencies: Counter({'and': 2, 'practice': 2, 'Learn': 1, 'learn': 1, 'to': 1})
Most common words: [('and', 2), ('practice', 2), ('Learn', 1), ('learn', 1), ('to', 1)]

Using FreqDist from NLTK

The Natural Language Toolkit (NLTK) provides the FreqDist function which creates a frequency distribution object. This approach is particularly useful for natural language processing tasks ?

from nltk import FreqDist

text = "Learn and practice and learn to practice"
words = text.split()
fdist = FreqDist(words)
print("Frequency distribution:", fdist)
print("Most common words:", fdist.most_common())

Frequency distribution: <FreqDist with 5 samples and 7 outcomes>
Most common words: [('and', 2), ('practice', 2), ('Learn', 1), ('learn', 1), ('to', 1)]

Using Dictionary and List Comprehension

This approach manually counts word occurrences using Python's built-in count() method and creates a dictionary using zip() ?

text = "Learn and practice and learn to practice"
words = text.split()
word_freq = [words.count(word) for word in words]
frequency_dict = dict(zip(words, word_freq))
print("Word frequency dictionary:", frequency_dict)

Word frequency dictionary: {'Learn': 1, 'and': 2, 'practice': 2, 'learn': 1, 'to': 1}

Comparison of Methods

Conclusion

For most word frequency tasks, Counter from collections is the most efficient choice. Use FreqDist for advanced NLP operations, and the dictionary method helps understand the underlying logic of word counting.