— Administrator's Guide

Release 5.0.7-4

August 2011

Reproduction in any manner whatsoever without the written permission of ParTec Cluster Competence Center GmbH is strictly forbidden.

All rights reserved. ParTec and ParaStation are registered trademarks of ParTec Cluster Competence Center GmbH. The ParTec logo, the ParaStation logo and the ParaStation MPI logo are trademarks of ParTec Cluster Competence Center GmbH. Linux is a registered trademark of Linus Torvalds. All other marks and names mentioned herein may be trademarks or registered trademarks of their respective owners.

The ParaStation MPI Administrator's Guide and ParaStation MPI User's Guide provide detailed information about ParaStation MPI. Installation and configuration of ParaStation MPI including process management and communication libraries are described in-depth by the ParaStation MPI Administrator's Guide. Usage of MPI and related tools are described by the ParaStation MPI User's Guide.

Though it may seem hard to believe, this manual might contain errors. We welcome any reports on errors or problems that are found. We also would appreciate suggestions on improving this book. Please direct all comments and problems to .

The most up-to-date version of this document is available at http://docs.par-tec.com.


Share your knowledge with others. It's a way to achieve immortality.

 --Dalai Lama

Table of Contents

1. Preface
About this document
This book's audience
ParaStation MPI overview
The history of ParaStation MPI
2. Technical overview
Runtime daemon
Kernel modules
3. Installation
Directory structure
Installation via RPM packages
Installing the documentation
Installing MPI
Further steps
Uninstalling ParaStation MPI
4. Configuration
Configuration of the ParaStation MPI system
Enable optimized network drivers
Testing the installation
5. Insight ParaStation MPI
ParaStation pscom communication library
ParaStation protocol p4sock
Directory /proc/sys/ps4/state
Directory /proc/sys/ps4/ether
Directory /proc/sys/ps4/local
Controlling process placement
Using the ParaStation MPI queuing facility
Exporting environment variables for a task
Using non-ParaStation MPI applications
ParaStation MPI TCP bypass
Controlling ParaStation MPI communication paths
Authentication within ParaStation MPI
Homogeneous user ID space
Single system view
Parallel shell tool
Nodes and CPUs
Integration with AFS
Integrating external queuing systems
Integration with PBS PRO
Integration with OpenPBS
Integration with Torque
Integration with LSF
Integration with LoadLeveler
Copying files in parallel
Using ParaStation MPI accounting
Using ParaStation MPI process pinning
Using memory binding
Spawning processes belonging to all groups
Changing the default ports for psid(8)
Relocating packages during installation
6. Troubleshooting
Problem: psiadmin returns error
Problem: node shown as "down"
Problem: cannot start parallel task
Problem: bad performance
Problem: different groups of nodes are seen as up or down
Problem: cannot start process on front end
Warning issued on task startup
Problem: pssh fails
Problem: psid does not startup, reports port in use
Problem: processes cannot access files on remote nodes
Warning: PSI barrier timeout
I. Reference Pages
parastation.conf — the ParaStation MPI configuration file
psiadmin — the ParaStation MPI administration and management tool
psid — the ParaStation MPI daemon. The organizer of the ParaStation MPI software architecture.
test_config — verify the ParaStation4 configuration file.
test_nodes — test physical connections within a cluster.
test_pse — test virtual connections within a cluster.
p4stat — display information about the p4sock protocol.
p4tcp — configure the ParaStation4 TCP bypass.
psaccounter — Write accounting information from the ParaStation MPI psid to the accounting files.
psaccview — Print ParaStation MPI accounting information.
mlisten — display multicast pings from the ParaStation MPI daemon psid(8)
A. Quick Installation Guide
B. ParaStation license
C. Upgrading ParaStation4 to ParaStation MPI, version 5
Building and installing ParaStation MPI packages
Changes to the runtime environment