How to get the details of multiple function definitions in a database from AWS Data catalog using Boto3

Let's see how a user can get the details of multiple function definitions from AWS Glue Data Catalog.

Example

Problem Statement: Use boto3 library in Python to get the details of multiple function definitions present in a database(s) from AWS Glue Data Catalog.

Approach/Algorithm to solve this problem

• Step 1: Import boto3 and botocore exceptions to handle exceptions.

• Step 2: database_name and regular_pattern are optional parameters. If details are not provided for these, the function fetches the definition of all the functions present in AWS User account. If database_name is provided but regular_pattern is not provided, then it fetches all the functions in a given database. If both the parameters are provided, then it fetches the definition of matched function as per regular_pattern. If only regular_pattern is provided, it fetches all the functions that match with the regular_pattern present in the AWS user account

• Step 3: Create an AWS session using boto3 lib. Make sure region_name is mentioned in the default profile. If it is not mentioned, then explicitly pass the region_name while creating the session.

• Step 4: Create an AWS client for glue.

• Step 5: Call get_multiple_function_definition and pass the database_name as DatabaseName and regular_pattern as Pattern parameter.

• Step 6: It returns the definition of multiple functions as per the provided parameter.

• Step 7: Handle the generic exception if something went wrong while checking the function.

Example Code

The following code fetches the definition of multiple functions −

import boto3
from botocore.exceptions import ClientError

def get_multiple_function_definition(database_name =None, regular_pattern = None):
   session = boto3.session.Session()
   glue_client = session.client('glue')
   try:
      response = glue_client.get_user_defined_functions(DatabaseName=database_name,Pattern= regular_pattern)
      return response
   except ClientError as e:
      raise Exception("boto3 client error in get_multiple_function_definition: " + e.__str__())
   except Exception as e:
      raise Exception("Unexpected error in get_multiple_function_definition: " + e.__str__())
a = get_multiple_function_definition('employee')
print(a)

Output

{
   'UserDefinedFunctions':[{
      'FunctionName': 'insert_employee_record',
      'DatabaseName': 'employee',
      'ClassName': 'InsertEmployee',
      'OwnerName': 'string',
      'OwnerType': 'USER'|'ROLE'|'GROUP',
      'CreateTime': datetime(2021,03,15),
      'ResourceUris':[
         {
            'ResourceType': 'JAR'|'FILE'|'ARCHIVE',
            'Uri': 'string'
         },
      ]
   }]
}

Ashish Anand

e

Updated on: 15-Apr-2021

71 Views

Kickstart Your Career

Get certified by completing the course

Get Started