How to use Boto3 to get the list of schemas present in AWS account

In this article, we will see how a user can get the list of all schemas present in an AWS account using the Boto3 library. AWS Glue Data Catalog stores schema information that can be retrieved programmatically.

Problem Statement

Use boto3 library in Python to get the list of all schemas available in an AWS Glue Data Catalog.

Prerequisites

Before running the code, ensure you have:

• AWS credentials configured (via AWS CLI, IAM roles, or environment variables)

• Boto3 library installed: pip install boto3

• Appropriate IAM permissions for Glue operations

Approach

The solution follows these steps:

• Step 1: Import boto3 and botocore exceptions to handle errors

• Step 2: Create an AWS session with proper region configuration

• Step 3: Create an AWS client for glue service

• Step 4: Use list_schemas() function to retrieve all schemas

• Step 5: Handle exceptions appropriately

Example Code

The following code fetches the list of all schemas ?

import boto3
from botocore.exceptions import ClientError

def list_of_schemas():
    session = boto3.session.Session()
    glue_client = session.client('glue')
    try:
        schemas_name = glue_client.list_schemas()
        return schemas_name
    except ClientError as e:
        raise Exception("boto3 client error in list_of_schemas: " + e.__str__())
    except Exception as e:
        raise Exception("Unexpected error in list_of_schemas: " + e.__str__())

print(list_of_schemas())

Output

{
    'Schemas': [
        {
            'RegistryName': 'employee_details',
            'SchemaName': 'employee_table',
            'SchemaArn': 'string',
            'Description': 'Schema for employees record',
            'Status': 'AVAILABLE',
            'CreatedTime': 'string',
            'UpdatedTime': 'string'
        },
        {
            'RegistryName': 'security_details',
            'SchemaName': 'security_table',
            'SchemaArn': 'string',
            'Description': 'Schema for security record',
            'Status': 'AVAILABLE',
            'CreatedTime': 'string',
            'UpdatedTime': 'string'
        }
    ],
    'ResponseMetadata': {...}
}

Enhanced Example with Error Handling

Here's an improved version that extracts only the schema names for easier viewing ?

import boto3
from botocore.exceptions import ClientError

def get_schema_names():
    try:
        session = boto3.session.Session()
        glue_client = session.client('glue')
        
        response = glue_client.list_schemas()
        
        if 'Schemas' in response:
            schema_names = [schema['SchemaName'] for schema in response['Schemas']]
            return schema_names
        else:
            return []
            
    except ClientError as e:
        print(f"AWS Client Error: {e}")
        return None
    except Exception as e:
        print(f"Unexpected error: {e}")
        return None

# Get and display schema names
schemas = get_schema_names()
if schemas is not None:
    print(f"Found {len(schemas)} schemas:")
    for schema in schemas:
        print(f"- {schema}")
else:
    print("Failed to retrieve schemas")

Key Points

• The list_schemas() function returns schemas with status AVAILABLE only

• Schemas with status DELETING are not included in the response

• If no schemas exist, the function returns an empty list in the Schemas key

• Always handle ClientError exceptions for AWS?specific errors

Conclusion

Using Boto3's list_schemas() method provides an efficient way to retrieve all available schemas from AWS Glue Data Catalog. Proper error handling ensures robust application behavior when working with AWS services.