Looping in any Data Processing Flow is must and I discuss how it can be used with in InfoSphere Datastage with the help of some example.
I have data available in a File (any format) as DataStage can process can pass file id as parameter
reading from another file. File ID will keep on increasing and we take advantage of datastage
looping instead of adding job or change sequencer.Contents in the File can be S.No and File Id and use this File ID as Parameter to the parallel Job.
To achieve this can start with getting total number of lines in the file using DataStage External Routine which contains Command = “wc -l”:” “:FileName:”| awk ‘{print $1}’” and get the number of lines. Routine can be triggered by nall DSExecute(“UNIX”, Command, Output, SystemReturnCode)
This will provide "Upper Limit" for the Loop.
File Id can be retrieved with startLoop.$Counter variable with the help of grep and awk command as mentioned above and will get file ID for each iteration.
This is how DataStage Job will look Like
Will cover Loop in Transformer in a follow-up.