Defines the number of cores allocated for each process. May be overwritten by psenvironment_PSI_TPP.

Only unused nodes are considered for spawning new processes. In addition, the nodes chosen for the current job will be locked for further jobs, consequently no additional processes will be started on this nodes until the current job terminates.

This variable does not define, how many processes of a job will be placed per node. See also PSI_OVERBOOK and set maxproc of psiadmin(8).

A list of environment variables which should be exported to remote processes during spawning. Some environment variables are exported by default: HOME, USER, SHELL, TERM, LD_LIBRARY_PATH, LD_PRELOAD, MPID_PSP_MAXSMALLMSG, In addition, all variables named PSP_*, __PSI_* or OMP_* are exported. Therefore, the variable OMP_NUM_THREADS is exported automatically.

Furthermore PWD is set correctly for remote processes. In addition, the environment used for partitioning the cluster (i.e. PSI_NODES, PSI_HOSTFILE or PSI_HOSTS and PSI_NODES_SORT) is propagated to remote processes.

Defines the nodes building the partition used to spawn new processes to. Depending on the variable PSI_NODES_SORT the ordering may be relevant. If the number of processes to spawn exceed the number of nodes in the partition, some nodes may get more than one process.

See also PSI_HOSTS and PSI_HOSTFILE.

Space separated list of hostnames on which new processes should be spawned on. Similar to PSI_NODES, but with hostnames instead of logical ParaStation node numbers. If PSI_NODES is set too, it is dominant over PSI_HOSTS.

See also PSI_NODES and PSI_HOSTFILE.

The name of a file containing a list of nodes' hostnames which should be used for spawning. Similar to PSI_HOSTS but the actual information is within the file instead of the environment variable. If PSI_NODES or PSI_HOSTS are set too, they are dominant over PSI_HOSTFILE.

See also PSI_NODES and PSI_HOSTFILE.

This variable controls the behavior of ParaStation when placing processes on nodes. If PSI_LOOP_NODES_FIRST is not defined, ParaStation first of all will try to use all available CPUs on a node for the current job. If necessary, more processes will be placed on the next nodes. If PSI_LOOP_NODES_FIRST is defined, ParaStation will place one process per node, and if more processes as available nodes are requested, it will start putting an additional process on each node, as long as all processes are placed; or the placement couldn't be fullfilled, e.g. due to the fact that not enough CPUs are available.

This variable defines the sorting criterion used to reorder the nodes building a virtual partition. This order will be used to spawn remote processes. The following values of mode are recognized:

If defined, more processes per node will be placed than CPUs available, if necessary. If undefined, only as many processes will be placed on a node as unused CPUs (= number(CPU) - number(currently running processes)) are available.

See also set maxproc of psiadmin(8), which takes precedence over PSI_OVERBOOK.

Defines the number of cores allocated per process. If undefined, defaults to 1.

See also psenvironment_OMP_NUM_THREADS.

If defined, new job start request will be queued, if not enough resources are currently available. See Chapter 3 and psmstart(1) for more details.

Preceding arguments for remote processes. For example: use PSI_RARG_PRE_0=/usr/bin/time to execute the process chain /usr/bin/time <yourApplication> <yourArgs> on the remote nodes.

This parameter defines after how many PMI_BARRIER_TMOUT cycles a job will be terminated, if not all processes have joined the PMI barrier. Defaults to 1.

The parameter should remain at the default value in production environments. This parameter's primary use is for diagnostic purposes as it allows the user to observe slower clients join an PMI barrier over multiple timeout periods. As such, the parameter helps administrators identify possible filesystem or network issues that occur on specific client nodes.

The PMI_BARRIER_TMOUT variable defines the delay (in seconds) allowed for each process to successfully join an PMI barrier. If not all processes joined, a corresponding warning is printed to stdout.

If PMI_BARRIER_TMOUT is not set, the timeout will be 60sec + (# of processes * 0.5µsec). If PMI_BARRIER_TMOUT equals -1, no barrier timeout is used and the job will not terminate because of failure to join the barrier from any one process. If PMI_BARRIER_TMOUT is set to num, then the timeout is set to num seconds.

See also ParaStation MPI Administrator's Guide.

If set, suppress pinning of processes, even if enabled globally (value irrelevant).

If set, suppress binding to memory-node, even if enabled globally (value irrelevant).

Using this environment variable, it is possible to define the communication library to use, independent of the variables mentioned above. This library must match the currently available interconnect and protocol, otherwise an error will occur.

The library name must be specified using the full path and filename, e.g. PSP_LIB=/opt/parastation/lib64/libpsport4openib.so.

A comma or space separated list of networks enabled to do optimized ParaStation communication using the p4sock protocol or TCP. Each network is a resolvable hostname in the chosen network, the IP address of a host in this network or the IP address of this network. The corresponding network has to be bound to a NIC of the current node.

If PSP_NETWORK is set, each network should be bound to a distinct NIC. This card then is used in order to do communication operations. If more than one network is given, the first one found to be bound to a local NIC is used.

If PSP_NETWORK is not set, ParaStation uses the NIC bound to the IP address, the local hostname resolves to.

Retry counter for all connect() calls within the pscom library. Default is 3.

TCP listen() backlog length. Only required for pscom library version below version 5.0.34.

The actual backlog is the minimum of PSP_TCP_BACKLOG and net.core.somaxconn, defined by the operating system.

If set to 1, use "on demand" connections with PSP_OPENIB. This means, establish connections between ranks and allocate there associated communication buffers with the first byte send. This could cause application aborts at any time, if the application runs out of resources (e.g. a final all to one communication pattern could fail)! Default is to establish all connections at startup time (inside MPI_Init()) which assures, that there are enough resources available for all connections. If not, MPI_Init() will fail.

These variables define the TCP buffer size used for TCP sockets. Defaults to 32k.

If set to 1 (default), the socket option NODELAY will be used for TCP sockets.

control the size of the TCP backlog when listening for new connections.

If set to 1, call sched_yield() in polling loops instead of busy polling. This might improve shared memory performance a lot, when there is more than one process per CPU core running, but slowdown communication performance in the common case of one process per core. (see also overbooking)

Control the path MTU of InfiniBand connections. Default is 3 which correspond to 1024 bytes. (1 = 256 bytes, 2 = 512 bytes, 3 = 1024 bytes)

These variables define the InfiniBand buffer counts used for InfiniBand connections. (Default = 16)

If set, psilogger will forward all input to the process with the corresponding rank within the process group. The default is to give all available input to process 0.

If set, psilogger will print a message about the user and system time consumed by each process of the parallel task upon exit of this process.

If set, psilogger gives information about the source of the received output, i.e. it will prepend every output by “[id]:”, where id is the rank of the printing process within the process group. Usually the id coincides with the MPI-rank. If PSI_LOGGERDEBUG is also set, every output is prepended by “[id, len]”, where id is the rank again and len is the length of the printed message in bytes.

If set, psilogger will not print out the message “PSIlogger: done” at the end of a parallel run.

If set, psilogger gives debug output about connecting and detaching clients as well as received output from the clients.

If set, debug output of the psiforwarder about connected programs, received input and received output is printed.

Length (in bytes) of the largest message sent without rendezvous.

Define the method used in order to spawn remote processes. The possible values are:

Comma separated list of hostnames. Used for MPID_PSP_START=SSH only.

defines (beside others) the path to the required preload library to enable the TCP bypass. It must be set to /opt/parastation/lib64/libp4tcp.so

defines the debug mask controlling the process management information. The following bits are defined:

These debug flags may be set as hex numbers, e.g. PSI_DEBUGMASK=0x07.

defines the debugging level for the ParaStation psport4 library. Higher values generally give more output.