Monday, December 5, 2011

Looping in InfoSphere DataStage Sequencer

Looping in any Data Processing Flow is must and I discuss how it can be used with in InfoSphere Datastage with the help of  some example.
I have data available in a File (any format) as DataStage can process can pass file id as parameter reading from another file. File ID will keep on increasing  and we take advantage of datastage looping instead of adding job or change sequencer.Contents in the File can be S.No and File Id and use this File ID as Parameter to the parallel Job.
To achieve this can start with getting total number of lines in the file using DataStage External Routine which contains Command = “wc -l”:” “:FileName:”| awk ‘{print $1}’” and get the number of lines. Routine can be triggered by nall DSExecute(“UNIX”, Command, Output, SystemReturnCode)
This will provide "Upper Limit" for the Loop.

File Id can be retrieved with startLoop.$Counter variable with the help of grep and awk command as mentioned above and will get file ID for each iteration.
This is how DataStage Job will look Like 

Will cover Loop in Transformer in a follow-up.

Disclaimer: The postings on this site are my own and don't necessarily represent IBM's positions, strategies or opinions

No comments:

Post a Comment