How to use Boto3 to start the scheduler of a crawler in AWS Glue Data Catalog

In this article, we will see how a user can start the scheduler of a crawler in AWS Glue Data Catalog.

Example

Start the scheduler of a crawler available in AWS Glue Data Catalog.

 Problem Statement: Use boto3 library in Python to start the scheduler of a crawler.

Approach/Algorithm to solve this problem

• Step 1: Import boto3 and botocore exceptions to handle exceptions.

• Step 2: crawler_name is the required parameter in this function.

• Step 3: Create an AWS session using boto3 lib. Make sure region_name is mentioned in the default profile. If it is not mentioned, then explicitly pass the region_name while creating the session.

• Step 4: Create an AWS client for glue.

• Step 5: Now use the start_crawler_schedule function and pass the parameter crawler_name as CrawlerName.

• Step 6: It returns the response metadata and sets the schedule state of the crawler as SCHEDULED. If the status of the crawler is running or the schedule state is already SCHEDULED, then it throws the exception - SchedulerRunningException.

• Step 7: Handle the generic exception if something went wrong while starting the scheduler of a crawler.

Example Code

The following code starts the scheduler of a crawler −

import boto3
from botocore.exceptions import ClientError

def start_scheduler_of_a_crawler(crawler_name)
   session = boto3.session.Session()
   glue_client = session.client('glue')
   try:
      response = glue_client.start_crawler_schedule(CrawlerName=crawler_name)
      return response
   except ClientError as e:
      raise Exception("boto3 client error in start_scheduler_of_a_crawler: " + e.__str__())
   except Exception as e:
      raise Exception("Unexpected error in start_scheduler_of_a_crawler: " + e.__str__())
print(start_scheduler_of_a_crawler("Data Dimension"))

Output

{'ResponseMetadata': {'RequestId': '73e50130-*****************8e', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Sun, 28 Mar 2021 07:26:55 GMT', 'content-type': 'application/x-amz-json-1.1', 'content-length': '2', 'connection': 'keep-alive', 'x-amzn-requestid': '73e50130-***************8e'}, 'RetryAttempts': 0}}