Portable Instrumentation Library for MPI (MPICL)

Click here to see the number of accesses to this library.

MPICL is a subroutine library for collecting information on communication and user-defined events in message-passing parallel programs written in C or FORTRAN. In particular, for MPI programs it uses the MPI profiling interface to automatically intercept calls to MPI communication routines, eliminating the need to add more than a few statements to the source code in order to collect the information. By using the MPI_Pcontrol interface to the instrumentation commands, a single version of the MPI program can be used whether the instrumentation library is linked with the executable or not.

MPICL instruments the application code, primarily using C routines that query the system clock and save event-specific information in internal buffers, and incurs more overhead than a good, direct, instrumentation of the underlying system would. However, great care has been taken to minimize this overhead, and versions of the instrumentation layer have been used successfully in numerous performance studies over the last 9 years. Being able to quickly port and use MPICL, not having to depend on the existence or correctness of vendor-supplied performance tools, has been crucial in many of these studies.

MPICL is typically used in one of two ways. It can be used to collect profile data, summarizing the number of occurrences, the associated volume statistics, and the time spent in communication and user-defined events for each processor. It can also be used to collect detailed traces of each event, which can then be viewed using the ParaGraph visualization tool. MPICL uses the PICL trace data format, described in the technical report referenced below. This has been updated to include new MPI-specific events, but the structure of the format is unchanged.

MPICL is built on top of the Portable Instrumented Communication Library (PICL), a software packag e that provided a portable message-passing interface in the days before the MPI standard. The PICL message-passing commands simply call the underlying native commands on each machine on which it is implemented. While an MPI user need know nothing about PICL message-passing, this does mean that MPICL can be used to collect performance data for non-MPI programs. But information on communication events is collected only if MPI or PICL message-passing commands are used, or if the user instruments the message-passing layer using MPICL instrumentation commands.

Disclaimer

MPICL was developed by P. H. Worley at Oak Ridge National Laboratory and has been used by the author for a variety of performance evaluation studies since its initial incarnation in 1997. However, MPICL IS RESEARCH SOFTWARE WITHOUT ANY GUARANTEE OR WARRANTY THAT IT IS GOOD FOR ANYTHING OR SAFE TO USE. It is being made available at this time to coordinate with the release of the new MPI-aware version of ParaGraph. Please notify and acknowledge the author in any research or publications utilizing MPICL, or any part of the code. Suggestions, bug reports, and (especially) new ports are also appreciated.

Ports

At the current time, the MPI instrumentation layer has been used successfully on the HP/Convex Exemplar, IBM SP, Intel Paragon, SGI/Cray Origin 2000, SGI/Cray T3E, and on a network of workstations using MPICH. MPICL is easily ported to any standards-compliant version of MPI. The machine dependent code primarily deals with the Fortran/C interface and, possibly, the use of a higher resolution system clock.

The underlying PICL library works on a much larger set of machines and communication libraries, many of which are obsolete:

Note that the non-MPICL aspects of PICL are not exercised very often, and some of these implementations may no longer work perfectly.

The most recent version of the library can be obtained from the MPICL web page. Questions and comments may be addressed to worleyph@ornl.gov.