Within InfoSphere DataStage Ulimits
are determined by the dsrpc daemon process for all process spawned by
the Conductor on an SMP environment. When we introduce
additional processing nodes (MPP) Ulimits need to be addressed on those
nodes separately. On
UNIX/Linux MPP configuration the ulimit are set for the user running
the processes on the processing nodes, They are not inherited from the
Conductor like the rest of the environment. In the event of not setting ulimit on these nodes we get error like "Fatal Error: Need to be able to open at least 16 files"
Each
users that "owns" processes on the remote processing nodes (where the
Conductor spawns processes on remote processing nodes as that user)
needs to have their ulimit set correctly on EACH processing node. This
can be done by either setting the default ulimit for everyone, or
setting each user's ulimit specifically. Like in AIX it can be achieved by modifying the /etc/security/limits file's default settings as root user:
default:
fsize = -1
core = 2097151
cpu = -1
data = -1
rss = -1
stack = 65536
nofiles = 102400
nofiles_hard = 102400
The default section is used to set the defaults for all users, You can override each user in the same manner. On 64-bit AIX should not set nofiles to -1 (unlimited) or nofiles_hard to unlimited. It can result an integer overflow for
Information Server and you will get a variation of the errors.
-Ritesh