Data Services administration includes creating real time and batch jobs, scheduling jobs, embedded data flow, variables and parameters, recovery mechanism, data profiling, Performance tuning, etc.
You can create real time jobs to process real time messages in Data Services designer. Like a batch job, real time job extracts the data, transforms and loads it.
Each real time job can extract data from a single message. You can also extract data from other sources like tables or files.
Real time jobs are not executed with the help of triggers unlike batch jobs. They are executed as real time services by administrators. Real time services wait for messages from access server. Access server receives this message and passes it to real time services, which is configured to process the message type. Real time services execute the message and returns the result and continues to process the messages until they get an instruction to stop executing.
Transforms like the branches and control logic are used more often in real time job, which is not the case with batch jobs in designer.
Real time jobs are not executed in response of a schedule or internal trigger unlike the batch jobs.
Real time jobs can be created using same objects like data flows, work flows, loops, conditionals, scripts, etc.
You can use the following data models for creating real time jobs −
You can create a real time job with single data flow in its real time processing loop and it includes a single message source and single message target.
Creating Real Time job using single data model −
To create Real Time job using single data model, follow the given steps.
Step 1 − Go to Data Services Designer → Project New → Project → Enter the Project Name
Step 2 − Right click on the white space in Project area → New Real time job.
Workspace shows two components of Real time job −
It shows the beginning and the end of real time job.
Step 3 − To create a real time job with single data flow, select data flow from the tool palette on the right pane and drag it to the work space.
Click inside the loop, you can use one message source and one message target in real time processing loop. Connect the starting and the end marks to the data flow.
Step 4 − Add configure objects in data flow as required and save the job.
This allows you to create a real time job with multiple data flow in its real time processing loop. You also need to ensure that the data in each data model is fully processed before it moves to the next message.
You can test the real time job by passing the sample message as source message from the file. You can check if the Data Services generate the expected target message.
To ensure that your job gives you the expected result, you can execute the job in view data mode. Using this mode, you can capture output data to make sure that your real time job is working fine.
Embedded data flow is known as data flows, which are called from another data flow in the design. The embedded data flow can contain multiple number of source and targets but only one input or output pass data to main data flow.
The following types of embedded data flows can be used −
One Input − Embedded data flow is added at the end of dataflow.
One Output − Embedded data flow is added at the beginning of a data flow.
No input or output − Replicate an existing data flow.
Embedded data flow can be used for the following purpose −
To simplify the data flow display.
If you want to save the flow logic and reuse it in other data flows.
For debugging, wherein you create sections of data flow as embedded data flow and execute them separately.