Python - Text Munging


Advertisements

Munging in general means cleaning up anything messy by transforming them. In our case we will see how we can transform text to get some result which gives us some desirable changes to data. At a simple level it is only about transforming the text we are dealing with.

Example

In the below example we plan to shuffle and then rearrange all the letters of a sentence except the first and the last one to get the possible alternate words which may get generated as a mis-spelled word during writing by a human. This rearrangement helps us in

import random

import re

def replace(t):
    inner_word = list(t.group(2))
    random.shuffle(inner_word)
    return t.group(1) + "".join(inner_word) + t.group(3)
text = "Hello, You should reach the finish line."
print re.sub(r"(\w)(\w+)(\w)", replace, text)

print re.sub(r"(\w)(\w+)(\w)", replace, text)

When we run the above program we get the following output −

Hlleo, You slouhd raech the fsiinh lnie.
Hlleo, You suolhd raceh the fniish line.

Here you can see how the words are jumbled except for the first and the last letters. By taking a statistical approach to wrong spelling we can decided what are the commonly mis0spelled words and supply the correct spelling for them.

Useful Video Courses


Video

Python Online Training

187 Lectures 17.5 hours

Malhar Lathkar

Video

Python Essentials Online Training

55 Lectures 8 hours

Arnab Chakraborty

Video

Learn Python Programming in 100 Easy Steps

136 Lectures 11 hours

In28Minutes Official

Video

Python with Data Science

Best Seller

75 Lectures 13 hours

Eduonix Learning Solutions

Video

Python 3 from scratch to become a developer in demand

Best Seller

70 Lectures 8.5 hours

Lets Kode It

Video

Python Data Science basics with Numpy, Pandas and Matplotlib

Most Popular

63 Lectures 6 hours

Abhilash Nelson

Advertisements