How to use Boto3 to start a workflow in AWS Glue Data Catalog


In this article, we will see how a user can start a workflow in AWS Glue Data Catlog.

Example

Problem Statement: Use boto3 library in Python to start a workflow.

Approach/Algorithm to solve this problem

  • Step 1: Import boto3 and botocore exceptions to handle exceptions.

  • Step 2: workflow_name is parameter in this function.

  • Step 3: Create an AWS session using boto3 lib. Make sure region_name is mentioned in the default profile. If it is not mentioned, then explicitly pass the region_name while creating the session.

  • Step 4: Create an AWS client for glue.

  • Step 5: Now use start_workflow_run function and pass the parameter workflow_name as Name.

  • Step 6: It returns the RunId and response metadata and starts the workflow.

  • Step 7: Handle the generic exception if something went wrong while starting a workflow.

Example Code

The following code starts a workflow −

import boto3
from botocore.exceptions import ClientError

def start_a_workflow(workflow_name)
   session = boto3.session.Session()
   glue_client = session.client('glue')
   try:
      response = glue_client.start_workflow_run(Name=workflow_name)
      return response
   except ClientError as e:
      raise Exception("boto3 client error in start_a_workflow: " + e.__str__())
   except Exception as e:
      raise Exception("Unexpected error in start_a_workflow: " + e.__str__())
print(start_a_workflow("test-daily"))

Output

{'RunId': 'wr_64e880240692fddd5e1b19aed587f856bc20a96f54bc', 'ResponseMetadata': {'RequestId': '782e953b-8ee3-4876-9b2c-cd35e147b513', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Sun, 28 Mar 2021 08:11:02 GMT', 'content-type': 'application/x-amz-json-1.1', 'content-length': '79', 'connection': 'keep-alive', 'x-amzn-requestid': '782e953b-********************************13'}, 'RetryAttempts': 0}}

Updated on: 15-Apr-2021

553 Views

Kickstart Your Career

Get certified by completing the course

Get Started
Advertisements