Flattening JSON objects in Python

JSON (JavaScript Object Notation) is a lightweight data interchange format. It is extensively used in web applications for transmitting data between the server and the client. JSON data often comes in a nested format, which can be difficult to manipulate. Flattening JSON objects involves converting complex hierarchical JSON structures to simpler structures. This process is often required when analyzing JSON data or transforming it into a different format. In this blog post, we will explore the process of flattening JSON objects in Python.

Syntax

Python has a built-in JSON module that provides functions to encode and decode JSON data.

data= <json-data-here>
flattened_data = flatten(data, "_")

We have an example using an external library called flatten-json. Install it using

pip install flatten-json

Algorithm

The process of flattening a JSON object involves the following steps −

• Create an empty dictionary to store the flattened data.

• Traverse the JSON object recursively.

• Make recursive calls to the function if the value is a dictionary.

• Add the key-value pair to the dictionary generated in step 1 if the value is not a dictionary.

Example

Consider the following nested JSON object −

{
   "name": "John",
   "age": 30,
   "address": {
      "street": "123 Main St",
      "city": "New York",
      "state": "NY"
   }
}

We can flatten this object using the following Python code −

import json

def flatten_json(nested_json):
   flattened_json = {}

   def flatten(x, name=''):
      if type(x) is dict:
         for a in x:
            flatten(x[a], name + a + '_')
      else:
         flattened_json[name[:-1]] = x

   flatten(nested_json)
   return flattened_json

nested_json = {
   "name": "John",
   "age": 30,
   "address": {
      "street": "123 Main St",
      "city": "New York",
      "state": "NY"
   }
}

flattened_json = flatten_json(nested_json)

print(json.dumps(flattened_json, indent=4))

Output

{
   "name": "John",
   "age": 30,
   "address_street": "123 Main St",
   "address_city": "New York",
   "address_state": "NY"
}

Explanation

• Create the flatten json function, which accepts nested JSON objects as input and produces a flattened JSON object.

• To hold the flattened data, the method generates an empty dictionary called flattened json.

• Then define an inner function called flatten that accepts two arguments: name, the current key prefix, and x, the current value. This function is invoked repeatedly with the current value and the current key prefix if the current value is a dictionary.

• The current key-value pair is added to the flattened json dictionary if the current value is not a dictionary.

• Lastly, use the input JSON object to perform the flatten method and return the flattened json dictionary.

To flatten this JSON object, we can use the flatten_json library as follows −

import json
from flatten_json import flatten

json_data = """
{
   "id": 1,
   "name": "John",
   "age": 30,
   "friends": [
      {
         "id": 2,
         "name": "Jane",
         "age": 28
      },{
         "id": 3,
         "name": "Bob",
         "age": 35,
         "location": {
            "city": "New York",
            "state": "NY"
         }
      }
   ]
}
"""

# Convert JSON string to Python object
data = json.loads(json_data)

# Flatten JSON object
flattened_data = flatten(data, "_")

print(flattened_data)

Output

{'id': 1, 'name': 'John', 'age': 30, 'friends_0_id': 2, 
'friends_0_name': 'Jane', 'friends_0_age': 28, 'friends_1_id': 3, 
'friends_1_name': 'Bob', 'friends_1_age': 35, 
'friends_1_location_city': 'New York', 'friends_1_location_state': 
'NY'}

The flatten function in this case has flattened the JSON object and separated the keys by using the specified separator, in this case an underscore (_). After combining the keys from the parent object and the keys from the nested object, the values are set accordingly.

Applications

Flattening JSON objects is a common task in web development, data analysis, and machine learning. It can help simplify data processing, reduce complexity, and make data easier to work with. Some specific applications of flattened JSON objects include −

• Storing JSON data in a relational database

• Processing JSON data with SQL

• Analyzing JSON data with machine learning algorithms

• Visualizing JSON data with charts and graphs

Conclusion

JSON data structures may be made simpler and more manageable by flattening JSON objects, which is a valuable method. There are other additional techniques available, and the flatten json package in Python offers a quick and effective approach to flatten JSON data. Your data processing procedures may be enhanced, and your applications can be made more effective and scalable, by learning how to flatten JSON objects.