|Library for 2D pencil decomposition and distributed Fast Fourier Transform|
This 2DECOMP&FFT library is written in Fortran. A Fortran 95 compatible compiler is the minimum requirement. In addition, the library relies on allocatable enhancement features (defined in ISO TR 15581, and officially part of Fortran 2003) and Cray pointers (for the shared-memory implementation only). These features are almost universally supported by modern Fortran compilers. The portability of 2DECOMP&FFT has been shown to be very good.
For FFT computations, the library can interface with all popular FFT libraries which, of course, need to be supplied. Some advanced software options may rely on other 3rd-party libraries.
Using Binary Package
A binary package may be available on your system (for example as part of the Open Petascale Libaries installation) for testing purpose. Suppose a binary package is installed at $DECOMP2D_DIR and a Fortran compiler is defined by $F90, to compile an application test.f90 using the library, do:
$F90 -I$DECOMP2D_DIR/include test.f90 -L$DECOMP2D_DIR/lib -l2decomp_fft -o test.exe
Please note that when creating real-world applications using 2DECOMP&FFT, a pre-built binary release is unlikely to suit your needs. This is because the library has many software options that are either hardware or application dependent. Users are always recommended to build the library from source.
Building the Library from Source Using GNU Make
To build the library from source, unpack the source package and follow the instructions here. There are Makefiles located in the $DECOMP2D_DIR/src directory. Normally there is no need to modify the main Makefile. Create a Makefile.inc instead from which software options and platform dependent information can be set. A couple of sample Makefile.inc files prepared for popular hardware are provided as templates.
- Software Options
One may choose to turn on/off certain software features in $DECOMP2D_DIR/src/Makefile.inc by appending flags to the OPTIONS variable. All optional features are implemented as code branches using pre-processing flags within source code. Note that some options are not available on all systems and some are dependent on external libraries which need to be installed separately. The available options are:
If the FFT part of the library is going to be used, users can also specify a suitable FFT engine by setting the FFT variable. 2DECOMP&FFT can interface with the following FFT libraries: FFTW (version 3.x), ACML, ESSL, MKL, FFTPACK, FFTE and even an experimental GPU engine using CUFFT. Please note that not all options are available on all systems. For example you won't find an MKL installation on a AMD Opteron-based supercomputer. For details, check the FFT page.
- System Dependent Information
Users are also responsible for providing a set of system dependent information for their specific hardware platforms, including compiler flags and external library locations. The following variables need to be set:
Some of the software features above require the use of external libraries. Their paths may need to be supplied at either compile or link time.
Several sample Makefile.inc are distributed with the package, supporting major platforms such as Cray XT/XE, IBM BlueGene, etc.
Once all set-up is done, type make from $DECOMP2D_DIR/src to build 2DECOMP&FFT. If successful, a library file will be created within $DECOMP2D_DIR/lib and a number of Fortran module files will be created within $DECOMP2D_DIR/include to be used by applications. To validate the library, try to build and run a few sample applications distributed with the library. Finally, please consider sending in any successful configurations to help support users with similar hardware.
First of all, read this page for technical background before trying this software feature.
To use the shared-memory implementation, download the FreeIPC package, extract it into the $DECOMP2D_DIR/src directory, turn on the -DSHM flag in $DECOMP2D_DIR/src/Makefile.inc and build 2DECOMP&FFT as normal. FreeIPC, provided by Dr. Ian Bush of NAG, allows easy access to System V Inter-Process Communication API from Fortran applications. This package has to be downloaded separately because it uses a different software license.
A number of sample applications are distributed to help validate and demonstrate the proper use of 2DECOMP&FFT (here is a list of sample applications). Each sample code is located in a separate sub-directory under $DECOMP2D_DIR/examples and is designed to test one aspect of the library. Each sample code has its own Makefile, which includes $DECOMP2D_DIR/src/Makefile.inc so options used to compile the base library are inherited to avoid potential conflicting settings. Several sample applications contain code branches defined by pre-processing directives to test those optional library features. If a sample application relies on external libraries, paths to such libraries may need to be set in the Makefile. A README file is available for each sample application. When everything is set up properly, simply type make to build.