Workflows are used to determine the process for execution. The main purpose of workflow is to prepare for executing the data flows and to set the state of system, once the data flow execution is completed.
The Batch jobs in ETL projects are similar to the Workflows with the only difference that the job does not have parameters.
Various objects can be added to a work flow. They are −
You can also make a work flow call other work flow or a work flow can call itself.
Note − In workflow, steps are executed in a left to right sequence.
Suppose there is a fact table that you want to update and you have created a data flow with the transformation. Now, if you want to move the data from source system, you have to check the last modification for fact table so that you extract only rows that are added after last update.
In order to achieve this, you have to create one script, which determines the last update date and then pass this as input parameter to the data flow.
You also have to check if the data connection to a particular fact table is active or not. If it is not active, you need to setup a catch block, which automatically sends an email to the administrator to notify about this problem.