Flattening JSON objects in Python

JSON (JavaScript Object Notation) is a lightweight data interchange format extensively used in web applications for transmitting data between servers and clients. JSON data often comes in nested formats, which can be difficult to manipulate and analyze. Flattening JSON objects involves converting complex hierarchical JSON structures into simpler, flat key-value pairs.

What is JSON Flattening?

JSON flattening transforms nested objects into a single-level dictionary where nested keys are combined using a separator (commonly underscore). This process is essential when working with databases, data analysis, or when you need to convert complex JSON into tabular formats.

Method 1: Custom Function Approach

You can create a custom recursive function to flatten JSON objects ?

import json

def flatten_json(nested_json, separator='_'):
    flattened_json = {}

    def flatten(x, name=''):
        if type(x) is dict:
            for key in x:
                flatten(x[key], name + key + separator)
        elif type(x) is list:
            for i, item in enumerate(x):
                flatten(item, name + str(i) + separator)
        else:
            flattened_json[name[:-1]] = x

    flatten(nested_json)
    return flattened_json

# Example nested JSON
nested_json = {
    "name": "John",
    "age": 30,
    "address": {
        "street": "123 Main St",
        "city": "New York",
        "state": "NY"
    },
    "hobbies": ["reading", "swimming"]
}

flattened_json = flatten_json(nested_json)
print(json.dumps(flattened_json, indent=2))

{
  "name": "John",
  "age": 30,
  "address_street": "123 Main St",
  "address_city": "New York",
  "address_state": "NY",
  "hobbies_0": "reading",
  "hobbies_1": "swimming"
}

Method 2: Using flatten-json Library

The flatten-json library provides a more robust solution for complex JSON structures ?

pip install flatten-json

import json
from flatten_json import flatten

# Complex nested JSON with arrays
json_data = {
    "id": 1,
    "name": "John",
    "age": 30,
    "friends": [
        {
            "id": 2,
            "name": "Jane",
            "age": 28
        },
        {
            "id": 3,
            "name": "Bob",
            "age": 35,
            "location": {
                "city": "New York",
                "state": "NY"
            }
        }
    ]
}

# Flatten JSON object with underscore separator
flattened_data = flatten(json_data, "_")

# Display flattened result
for key, value in flattened_data.items():
    print(f"{key}: {value}")

id: 1
name: John
age: 30
friends_0_id: 2
friends_0_name: Jane
friends_0_age: 28
friends_1_id: 3
friends_1_name: Bob
friends_1_age: 35
friends_1_location_city: New York
friends_1_location_state: NY

Comparison of Methods

Common Use Cases

Flattening JSON objects is particularly useful for ?

• Database Storage Converting JSON to relational database tables

• Data Analysis Processing JSON with pandas DataFrames

• CSV Export Converting nested JSON to flat CSV format

• Machine Learning Preparing features from nested JSON data

Conclusion

Flattening JSON objects simplifies complex hierarchical data structures into manageable key-value pairs. Use custom functions for simple cases or the flatten-json library for complex nested structures with arrays and deep nesting.