Platforms:IBM-POWER

From ALPS
Jump to: navigation, search

The intent of this page is to document the success or the failure you experienced when building ALPS on the IBM power platform.

The current status of sources in the SVN/CVS head is that they compile and build on all power platforms, provided that you use boost_1_33_1 and xlC 8. There are two problems one can encounter:

  • on AIX there is a linker problem which prevents the QMC codes from calculating the spin stiffness.
  • xlC versions before 08.00.0000.0013 crash due to a compiler bug when compiling the alps library. This can be circumvented by compiling without optimisations. Optimisation should then be switched on again when compiling a specific application.

For older xlC versions such as 6 or 7, patches to the boost library are necessary. These patches are available here.

How to fully characterize your machine

  • xlc -qversion
  • uname -a
  • ld -V
  • boost version
  • approximate ALPS version

ALPS on IBM Power machines running AIX

zahir.idris.fr

  • IBM XL C/C++ Enterprise Edition V8.0 for AIX Version: 08.00.0000.0013 (to obtain it, type module load xlc/8.0.0.13)
  • AIX zahir001 3 5 00104B5A4C00
  • ld: LD 1.65 (04/06/2005)
  • boost 1.33.1
  • ALPS CVS sources obtained on 25.05.2007 / or ALPS release 1.3b5
  • important the file alps/src/alps/lattice/graphproperties.h has been patched by changing line 85 to #if !BOOST_WORKAROUND(__IBMCPP__, <= 800). This workaround is necessary due linker problems on the AIX platform. This workaround will disable calculations of the stiffness in some QMC codes.


configure command:

configure LDFLAGS="-L/usr/local/pub/LAPACK/LAPACK-3.0/lib -llapack -lxlf90_r -lessl" 
--with-compiler=ibm64 --with-boost=$HOME/boost_1_33_1 
--with-lapack="-L/usr/local/pub/LAPACK/LAPACK-3.0/lib -llapack -lxlf90_r -lessl" -with-blas=-lessl

With this version of xlc, no more crashes appear (the previous xlc version 08.00.0000.0012 had some problems). Here are a few tricks which were useful with older versions of xlc:


Problem with xlc version 08.00.0000.00012  : compiler crashes upon compiling bondterm.C

workaround

  • add the option --disable-optimization when configuring.
  • It might be possible to change by hand some of the Makefiles for selected applications to switch on the optimization again during the build process. To be tested.

Problem: no rtti object error message when executing dirloop_sse. Is this related to the explicit CXXFLAGS we set ? -qrtti=all helps here


Problem with xlc version 7  : compiler crashes at various places

  • Use ALPS version 1.3a1 (available here) and Boost 1.33.1. Patches to the boost library are necessary. These patches are available here.
  • Then configure with
configure 
CPPFLAGS='-q64 -qlanglvl=extended -O2 -DUSE_DATE_TIME_PRE_1_33_FACET_IO' 
CXXFLAGS='-q64 -qlanglvl=extended -O2 -DUSE_DATE_TIME_PRE_1_33_FACET_IO'
--with-compiler=ibm64 --with-boost=$HOME/boost_1_33_1 
--with-mpi-libs='-binitfini:poe_remote_main -lmpi_r -lvtd_r'
  • Note that Lapack is not recognized here with this configuration. Probably it would be with the LDFLAGS setting mentionned above (not tested).

psi.rzg.mpg.de

  • IBM XL C/C++ Enterprise Edition V8.0 for AIX Version: 08.00.0000.0010
  • AIX psi24 3 5 0010594A4C00
  • ld: LD 1.65 (04/06/2005)
  • boost 1.33.1 (boost 1.34.0 produced error in compiling evaluator.C)
  • ALPS SVN Head sources obtained on June 9, 2007

configure command for the 64 bit version:

   setenv OBJECT_MODE 64
   ./configure
    --prefix=where_you_want 
    --disable-optimization
    --with-compiler=ibm64 
    --with-boost=$HOME/libraries/boost_1_33_1 
    --with-lapack-dir="/afs/rzg/rs_aix53/lib/" 
    --with-lapack="-llapack-essl -lxlf90_r"
    --with-essl="-lesslsmp" 

hanni1.hlrn.de

  • IBM XL C/C++ Enterprise Edition V8.0 for AIX Version: 08.00.0000.0013
  • AIX hanni1-en0 2 5 003CD56E4C00
  • ld: version unknown (executable: 06/25/2005)
  • boost 1.33.1
  • ALPS libraries and applications checked out from CVS on 24.05.2007

First attempt

configure command:

 configure --with-compiler=ibm64 --with-boost=$HOME/boost_1_33_1 --with-lapack="-L/aws/numerics/lapack/ -llapack -lessl -lxlf90_r" --with-blas=-lessl

Problem: dsyev() (without underscore) not found

workaround (until configure script is updated):

  • create empty directory $HOME/lib
  • set environment variable CXXFLAGS to "-L/aws/numerics/lapack/ -llapack -lessl -lxlf90_r"

failure during:

 /bin/sh ../libtool --tag=CXX --mode=link xlC_r   -L/aws/numerics/lapack/ -llapack -lessl -lxlf90_r      -o compactrun  compactrun.o ../lib/alps/libalps.la ../lib/comm-sgl/libcomm-sgl.la   -lexpat -lpthread -lm

message: ld: 0711-212 SEVERE ERROR: Symbol .std::vector<boost::detail::adj_list_ge ... a few KByte of text

Second try

  • use configuration from the first attempt
  • edit the file alps/src/alps/lattice/graphproperties.h and change line 85 to #if !BOOST_WORKAROUND(__IBMCPP__, <= 800) as described above for zahir.idris.fr

Now the libraries compile! Applications configuration is trivial (configure uses the configuration from the library installation).

Two applications don't build: dmrg/dmrg, qmc/looper (lengthy error messages)

Then tested the following tutorials:

  • sparsediag/parm1: terminates with error message Illegal term in expression
  • sparsediag/parm2: works
  • sparsediag/parm3: seems to work
  • fulldiag/parm1: works
  • fulldiag/parm2: works
  • fulldiag/parm3: works
  • fulldiag/parmV15: terminates with error message bad allocation
  • quantum2: in all 3 examples, dirloop_sse terminates with error message non rtti object

euler.rz.tu-bs.de

  • IBM XL C/C++ Enterprise Edition V8.0 for AIX Version: 08.00.0000.0000
  • AIX euler0-i 3 5 00CE074E4C00
  • ld: LD 1.65 (04/06/2005)

configure command:

 configure --with-compiler=ibm64 --with-boost=$HOME/boost_1_33_1 --with-blas=-lessl

failure during:

 xlC_r -DHAVE_CONFIG_H -I../../src -I/home/honecker/alps/src -DNDEBUG -I/home/honecker/boost_1_33_1 -q64 -O2 -qrtti -DBOOST_DISABLE_ASSERTS -w -c -M /home/honecker/alps/src/alps/model/bondterm.C -o bondterm.o

message: xlC_r: 1501-230 Internal compiler error; please contact your Service Representative

Machines running Linux on Power

blanc.cscs.ch (OK)

  • IBM XL C/C++ Advanced Edition V8.0 for Linux Version: 08.00.0000.0003
  • Linux blanc160 2.6.5-7.244-pseries64 #1 SMP Mon Dec 12 18:32:25 UTC 2005 ppc64 ppc64 ppc64 GNU/Linux
  • GNU ld version 2.15.90.0.1.1 20040303 (SuSE Linux) Supported emulations: elf32ppclinux elf32ppc elf32ppcsim elf64ppc
  • boost version 1.33.1
  • ALPS CVS sources obtained on 25.05.2007

ALPS Library

the configure command reads

export CXX mpCC
 configure --prefix=$HOME/ALPS/Installations/CVS-ALPS-xlC_linux/ --with-boost=$HOME/libraries/boost --with-compiler=ibm64 --without-expat

on this machine the configure script finds the LAPACK libraries itself.

On this machine the build and installation process finish without any problem.

ALPS Applications

using the --disable-optimization option when configuring the alps library ensures that the compilation process of the applications finishes without problems (dirloop_sse not built yet). The performance remains to be tested.

Fulldiag: compiling with optimization enabled takes ages. I stopped the compilation process of factory.C after one hour, but it might actually finish.

BlueGene/L Machines

Note that these machines need a cross-compiled version for the runs, and a frontend version of ALPS for the data processing if needed.

bg1.epfl.ch (OK)

  • IBM XL C/C++ Advanced Edition V8.0 for Blue Gene Version: 08.00.0000.0001
  • Linux bg-fe01 2.6.5-7.283-pseries64 #1 SMP Wed Nov 29 16:55:53 UTC 2006 ppc64 ppc64 ppc64 GNU/Linux
  • GNU ld version 2.15.90.0.1.1 20040303 (SuSE Linux) Supported emulations: elf32ppclinux elf32ppc elf32ppcsim elf64ppc
  • boost 1.33.1
  • ALPS SVN Head sources obtained on June 9, 2007

the configure command for the cross-compiled version of ALPS reads:

  setenv CC blrts_xlc
  setenv CXX blrts_xlC

  configure 
  --prefix=$HOME/ALPS/Installations/SVN-Alps-xlC 
  --with-boost=$HOME/libraries/boost 
  --with-compiler=ibm32
  --disable-optimization 
  --with-essl="-L/home/laeuchli/libraries/ibm/ -llapack_440n -lesslbg -lmass -L/opt/ibmcmp/xlf/bg/10.1/blrts_lib/ -lxlf90" 
  --with-lapack="-L/home/laeuchli/libraries/ibm/ -llapack_440n -lesslbg -lmass -L/opt/ibmcmp/xlf/bg/10.1/blrts_lib/ -lxlf90" 
  --with-mpi-incdir=/bgl/BlueLight/ppcfloor/bglsys/include/ 
  --with-mpi-libdir=/bgl/BlueLight/ppcfloor/bglsys/lib/ 
  --with-mpi-libs="-lmpich.rts  -lmsglayer.rts -lrts.rts -ldevices.rts"

the build and install process works without any problem.

for the applications it is sufficient to configure the light way:

  setenv CC blrts_xlc
  setenv CXX blrts_xlC
  configure
  --prefix=$HOME/ALPS/Installations/SVN-Alps-xlC 
  --with-lp_solve=$HOME/libraries/lp_solve

this now only builds applications without any optimization, so that they can easily be tested. For production one should patch the Makefiles in the directory of the specific application to compile at least with -O2.

BlueGene/P Machines

Note that these machines need a cross-compiled version for the runs, and a frontend version of ALPS for the data processing if needed.

genius.rzg.mpg.de (under evaluation)

  • IBM XL C/C++ Advanced Edition for Blue Gene/P, V9.0 Version: 09.00.0000.0000
  • Linux genius1 2.6.16.46-0.12-ppc64 #1 SMP Thu May 17 14:00:09 UTC 2007 ppc64 ppc64 ppc64 GNU/Linux
  • GNU ld version 2.16.91.0.5 20051219 (SUSE Linux) Supported emulations: elf32ppclinux elf32ppc elf32ppcsim elf64alpha alpha armelf_linux armelf hppalinux elf_i386 i386linux elf64_ia64 m68kelf m68klinux elf64ppc elf_s390 elf64_s390 elf32_sparc sparclinux elf64_sparc sun4 elf_x86_64 ppcmacos
  • boost 1.35.0
  • ALPS SVN Head sources obtained on June 27, 2008 (aka 1.3.2)

the configure command for the cross-compiled version of ALPS reads:

  setenv CC mpixlc
  setenv CXX mpixlcxx

  configure 
  --prefix=/u/amlh/BlueGene/ALPS/Installations/alps-1.3.2-bgp-bgxlc
  --with-boost=$HOME/libraries/boost 
  --with-compiler=ibm32

the build and install process works without any problem (apart from being slow!). Lapack support is not yet included, however MPI support is ok.

for the applications it is sufficient to configure the light way:

  setenv CC mpixlc
  setenv CXX mpixlcxx
  configure
  --prefix=/u/amlh/BlueGene/ALPS/Installations/alps-1.3.2-bgp-bgxlc 
  --with-lp_solve=$HOME/libraries/lp_solve


babel.idris.fr (under evaluation)

  • IBM XL C/C++ Advanced Edition for Blue Gene/P, V9.0 Version: 09.00.0000.0004
  • Linux babel1 2.6.16.54-0.2.5-ppc64 #1 SMP Mon Jan 21 13:29:51 UTC 2008 ppc64 ppc64 ppc64 GNU/Linux
  • GNU ld version 2.16.91.0.5 20051219 (SUSE Linux) Supported emulations: elf32ppclinux elf32ppc elf32ppcsim elf64alpha alpha armelf_linux armelf hppalinux elf_i386 i386linux elf64_ia64 m68kelf m68klinux elf64ppc elf_s390 elf64_s390 elf32_sparc sparclinux elf64_sparc sun4 elf_x86_64 ppcmacos
  • boost 1.35.0
  • ALPS release 1.3.3

For this specific version of ALPS release, 2 patches were in order:

  • Add the following line in the header of src/alps/expression/factor.h
#include <alps/expression/evaluator.h>
  • Replace line 105 of src/alps/stringvalue.h by
#if BOOST_WORKAROUND(__IBMCPP__, <= 900)

the configure command for the cross-compiled version of ALPS looks like this:

  setenv CC mpixlc_r
  setenv CXX mpixlcxx_r

  configure 
  --prefix=$HOME/alps/
  --with-boost=$HOME/libraries/boost 
  --with-compiler=ibm32

the build and install process works without any problem. Lapack support is not yet included, however MPI support is ok.

Applications not tested yet.


© 2007-2008 by Andreas Läuchli and Andreas Honecker