.TH NAG_MPI_UN 3NAG .SH NAME nag_mpi_un \- NAG Parallel Library (MPI-based version), Release 2, Users' Note .ce 7 .B "NAG Parallel Library (MPI-based version), Release 2 .B "FDFUA02DM .B "Fujitsu AP3000 .B "Users' Note .SH "Introduction" This document is essential reading for every user of the NAG Parallel Library implementation specified in the title. It provides implementation-specific detail that augments the information provided in the NAG Parallel Library Manual. NAG recommends that users read the following minimum reference material before calling any library routine: .in +5 .ti -4 (a) Essential Introduction .ti -4 (b) Tutorial (MPI version) .ti -4 (c) Chapter Introduction .ti -4 (d) Implementation-specific Users' Note .in -5 Items (a), (b) and (c) are included in the NAG Parallel Library Manual; item (d) is this document. Each NAG Parallel Library site is supplied with at least one copy of each of the above. A postscript version of (a) and (b) and a Unix man version of (d) are available. .SH "General Information" .SS "Fujitsu APruntime and Fujitsu MPI/AP" Parallel programs using the NAG Parallel Library require Fujitsu APruntime to be installed on an AP series computer. APruntime enables a partition, which consists of a group of processor nodes, to be handled as an independent distributed memory parallel computer. A crucial component of APruntime is the `aprun' command, which can be used to run parallel programs on multiple nodes of a partition using different communication modes. APruntime also provides several runtime libraries to support the execution of parallel programs. In particular, the APruntime message-passing kernel library enables concurrent processes to communicate using either .nf (a) a simple, AP-Net dedicated protocol - this is referred to as `user level' communication mode, or (b) a TCP/IP protocol - this is referred to as `system level' communication mode .fi Using the `user level' communication mode enables programs to access the AP-Net communication network directly without any system call. While this maximizes the performance of communication operations, it also restricts the number of processes on any processor node to only one. Using the `system level' communication mode results in slower communication operations, but enables multiple processes to run on a node simultaneously. Additionally, networks other than the AP-Net - for example, Ethernet - can be used with the `system level' communication mode. Further information on Fujitsu APruntime can be found in the APruntime User's Guide. In addition to APruntime, parallel programs using the NAG Parallel Library require MPI/AP to be installed. Fujitsu MPI/AP V1.0 is based on and conforms to the specifications of MPI 1.1. Exceptions are the MPI routines `MPI_CANCEL' and `MPI_TEST_CANCELED' which are not supported. MPI/AP is based on the message-passing kernel library supplied as part of APruntime and thus can be used in both `user level' and `system level' communication modes. Further information can be found in the Fujitsu MPI/AP User's Guide. The NAG Parallel Library has been tested and can be used with both `user level' and `system level' communication modes, and in the latter case both the AP-Net and Ethernet can be used as interconnection networks. .SS "Selecting Compiler Options" In order to achieve optimal performance, the NAG Parallel Library was built from its source codes using, amongst other things, the following compiler switches: .nf -dalign - enforces 8-byte data alignment and thus allows double word load/store operations -fns - selects the SPARC nonstandard floating-point mode .fi As a result, the user MUST use the `-dalign' and `-fns' switches for compiling all program modules. Additionally, the `-fsimple=1' compiler switch, which selects floating-point optimization preferences, was used to build the libraries. This choice implies that the user program must not change the rounding/trapping modes after program startup. It is recommended to set the compiler switch `-fast' when compiling all user program modules. This switch selects a combination of compiler options, including `-dalign' and `-fns', which optimizes the speed of execution without excessive compilation time. .SS "Accessing the Library" The NAG Parallel Library consists of six different libraries. Assuming that .nf (a) libnagmpi.a, libnagmpiz.a, libnagpblas.a, libnagmpiblacs.a, libnagtools.a and libnagfls.a have been installed in a directory in the search path of the linker (b) the Fujitsu MPI/AP library libmpi.a has been installed in /opt/FSUNmpiap/lib (c) the Fujitsu APruntime libraries libmpl.a and libemi.a have been installed in /opt/FSUNaprun/lib and libthread.a in /usr/lib (Note: these libraries may be shareable (.so) rather than archive (.a) libraries.) .fi then a user may link to the NAG Parallel Library as follows: .nf f77 -fast driver.f -lnagmpi -lnagpblas -lnagmpiz -lnagtools \ -lnagmpiblacs -L/opt/FSUNmpiap/lib -lmpi \ -L/opt/FSUNaprun/lib -lmpl -lemi -lthread -lnagfls .fi where driver.f is the user's application program. .SS "Running NAG Parallel Library Programs" The `aprun' command has to be used to run a user program, say a.out, linked with the NAG Parallel Library on multiple nodes of a partition in a Fujitsu AP3000 system, e.g. aprun -nproc 4 a.out instructs the system to execute the executable program a.out in four parallel processes. Using the `-nproc' option leaves the task of mapping parallel processes onto the available processor nodes to the operating system. Alternatively, the `aprun' option `-procpe' can be used to specify explicitly this mapping. Furthermore, the `aprun' options `-ult' or `-slt' can be used to choose the `user level' or `system level' communication modes, respectively. If neither of these options is used, the system-dependent default communication mode, which can be either `-ult' or `-slt', is selected automatically. Note that selecting the `user level' communication mode limits the number of parallel processes to the number of processor nodes exclusively available to the user program. More detailed information on the `aprun' command can be found in the APruntime User's Guide. Note also that it may be necessary to execute the `aprun' command from within a batch request of NQS/AP in order to be able to access parallel partitions. See the NQS/AP Handbook for further information. .SS "Optimizing Performance" The NAG Parallel Library uses standard sequential software components, such as the Basic Linear Algebra Subroutines (BLAS), wherever suitable. While the NAG Parallel Library provides portable implementations of these components, significant performance gains can often be attained by linking the user program with machine-optimized implementations. For instance, if the Sun Performance Library is available on a system, it can be linked with the user program using the `-xlic_lib=sunperf' compiler option. Note also that specifying the compiler option `-fast' on the Fujitsu AP3000 automatically links the user program with an optimized library of selected common elementary function routines. .SS "Notes" The standard input, standard output and standard error output of the parallel program execution command `aprun' are connected to all processes performing a NAG Parallel Library program. However, these connections are guaranteed only from the initial call of the NAG Parallel Library routine Z01AAFP until the final call of Z01ABFP. Read/write operations from/to standard input/output before the initial call of Z01AAFP and after the final call of Z01ABFP may fail. It is therefore recommended to call Z01AAFP at the very beginning and Z01ABFP at the very end of the user program. See also the Notes in Chapter 4 of the Fujitsu MPI/AP User's Guide. The size of the receive buffer when using the AP-Net communication system is limited to 2 Mb. By default, this buffer space is divided into 128 packets of 16 kb size. Since each message occupies at least one packet, at most 128 messages can be stored in the receive buffer simultaneously. Exceeding this limit results in MPLIB: Can't get receive buffer or MPLIB: Receive Buffer Overflow error messages and in the aborting of the parallel program execution. This problem is often caused by a large number of small messages which cannot be received quickly enough by the destination processes. In this case, the problem can usually be overcome by decreasing the packet size, which increases the number of messages that can be stored in the receive buffer. This is achieved by setting the environment variable APRUN_MAXPACKETSIZE appropriately, e.g. setenv APRUN_MAXPACKETSIZE 4096 Note that the packet size must be a power of 2 and larger than 256. .SS "Example Programs" The example programs and data and the results obtained during the preparation of this specific implementation are supplied so you can try them yourself, or use one as a template for your own program. Note that the example material has been adapted, if necessary, from that printed in the NAG Parallel Library Manual, so that programs are suitable for execution with this implementation. The example programs are most easily accessed by using the script `runex', which will copy an example program and its data, compile it, link it with the libraries and run it. For example, give the command runex f04ebfp to run the F04EBFP example program. This will place the results in the subdirectory examples/newresults. .SS "On-line User Documentation" The following information is available in man page form: Users' Note - nag_mpi_un.3 The following information is available as ASCII files: .nf List of Contents - contents.ascii Reserved Names - reserved.ascii .fi The following information is available in postscript form: .nf Release 2 News - news.ps Future Developments - future.ps Essential Introduction - essint.ps Tutorial - tutorial.ps (step by step instructions for using the NAG Parallel Library) .fi .SH "Routine-specific Information" .ti -4 (a) X02 The constants referred to in the NAG Parallel Library Manual have the following values in this implementation: The values of the machine constants are: The basic parameters of the model .nf .in +4 X02BHF = 2 X02BJF = 53 X02BKF = -1021 X02BLF = 1024 X02DJF = .TRUE. .in -4 Derived parameters of floating-point arithmetic .in +4 X02AJF = 1.1102230246252D-16 X02AKF = 2.2250738585072D-308 X02ALF = 1.7976931348623D+308 X02AMF = 2.2250738585072D-308 X02ANF = 4.4501477170147D-308 .in -4 Parameters of other aspects of the computing environment .in +4 X02AHF = 1.4272476927060D+45 X02BBF = 2147483647 X02BEF = 15 X02DAF = .FALSE. .in -4 .fi .SH "Additional Services from NAG" .ti -4 (a) Documentation Each NAG Parallel Library site is provided with a printed copy of the NAG Parallel Library Manual. Additional manuals are available at prices published in the NAG documentation order form. The NAG Parallel Library is also supported by a Hypertext system at extra cost which allows the user to browse through the Library documentation. Users will be advised of the availability of this system for different platforms as they become available. If you require further information please contact NAG. .SH "Support from NAG" .ti -4 (a) Contact with NAG Queries concerning this document or the implementation generally should be directed initially to your local Advisory Service. If you have difficulty in making contact locally, you can write to NAG directly, at one of the supplied addresses. Users subscribing to the support service are encouraged to contact one of the NAG Response Centres (see below). .ti -4 (b) NAG Response Centres The NAG Response Centres are available for general enquiries from all users and also for technical queries from sites with an annually licensed product or support service. The Response Centres are open during office hours, but contact is possible by fax, email and phone (answering machine) at all times. When contacting a Response Centre please quote your NAG user reference and NAG product code. .ti -4 (c) Network Network, NAG's newsletter, is produced periodically and sent free of charge to sites with a supported product or service. .ti -4 (d) NAG Website The NAG Website is an information service providing items of interest to users and prospective users of NAG products and services. The information is regularly updated and reviewed and includes implementation availability, descriptions of products, downloadable software and technical reports. The NAG Website can be accessed at .nf http://www.nag.co.uk/ or http://www.nag.com/ (in the USA) .fi .SH "NAG Users Association" NAGUA, the NAG Users Association, is a self-financing, non-profitmaking body. It exists to promote communications between NAG and users of its products and services. It provides information to NAG on the requirements of users, who are in turn kept informed of developments in services. Membership is available to any institution or individual holding a licence for any NAG product or service. Members receive discounts on the registration fees at conferences and workshops. For an information pack and membership application form, please contact NAGUA at the supplied address. .SH "Appendix - supplied addresses" .nf NAG Ltd Wilkinson House Jordan Hill Road OXFORD OX2 8DR NAG Ltd Response Centre United Kingdom email: support@nag.co.uk Tel: +44 (0)1865 511245 Tel: +44 (0)1865 311744 Fax: +44 (0)1865 310139 Fax: +44 (0)1865 310139 NAG Inc 1400 Opus Place, Suite 200 Downers Grove IL 60515-5702 NAG Inc Response Center USA email: infodesk@nag.com Tel: +1 630 971 2337 Tel: +1 630 971 2345 Fax: +1 630 971 2706 Fax: +1 630 971 2346 NAG GmbH Schleissheimerstrasse 5 D-85748 Garching Deutschland Tel: +49 89 3207395 Fax: +49 89 3207396 NAG Users Association PO Box 426 OXFORD OX2 8SD United Kingdom email: nagua@nag.co.uk Tel: +44 (0)1865 31110