[Hpc-forum] SIESTA lefordítása
Rőczei Gábor
roczei at niif.hu
2012. Okt. 1., H, 12:22:58 CEST
Kedves Péter,
/opt/sgi/mpt/mpt-2.04/bin/mpirun helyett mpirun-ra van szükség.
Erről van szó:
roczei at service0:~> type mpirun
mpirun is /usr/local/bin/mpirun
roczei at service0:~>
Ez már értelmezni fogja az SGE által kiválasztott gépeket. Érdemes megnézni egy "less" paranccsal a /usr/local/bin/mpirun tartalmát és akkor egyértelmű lesz, hogy mit csinál ez a script.
Job script:
#!/bin/sh
#$ -N ts4
#$ -pe mpi 4
mpirun -np $NSLOTS /home/sule/bin/SIESTA/siesta-3.1/Obj/siesta <gr.fdf> out_sl
Gábor
On 2012.10.01., at 12:00, Süle Péter wrote:
>
> Kedves Gábor !
>
> Köszönöm segítséget, ez a SGI-MPT/MKL verzió működik.
> Viszont azt tapasztalom, hogy nem jól skálázódik, azaz
> már a 4 procis is lassabban fut mint a soros job (N=1).
> Lehet, hogy nem jól konfigurálom a szkriptet ?
>
> sule at service0:/scratch/sule/SIESTA/gr/test/t4> more s.sh
> #!/bin/sh
> #$ -N ts4
> /opt/sgi/mpt/mpt-2.04/bin/mpirun -np $NSLOTS /home/sule/bin/SIESTA/siesta-3.1/Obj/siesta <gr.fdf> out_sl
>
> Üdv. Péter
>
>
> On Sat, 29 Sep 2012, Rőczei Gábor wrote:
>
>> Kedves Péter,
>>
>> Úgy néz ki hogy sikerült lefordítanom a számodra.
>>
>> Ezek a weboldalak is segítettek:
>>
>> http://hpcwiki.it.okstate.edu/index.php/Building_Siesta
>> http://software.intel.com/sites/products/documentation/doclib/mkl_sa/11/mkl_userguide_lnx/GUID-A2BEE926-ACA7-40B1-94D7-F728D145E1C0.htm
>>
>> Lépések:
>>
>> 1)
>>
>> cd Obj
>> sh ../Src/obj_setup.sh
>>
>> 2)
>>
>> ../Src/configure --with-siesta-blas --with-siesta-lapack --enable-mpi
>>
>> 3)
>>
>> Src/arch.make módosítása erre:
>>
>> # This is a dummy arch.make
>> # You should not compile Siesta in this directory.
>> # See Obj/README
>> #
>>
>> .SUFFIXES:
>> .SUFFIXES: .f .F .o .a .f90 .F90
>>
>> SIESTA_ARCH=x86_64-linux-Intel_MPI
>>
>> FC=mpif90
>> RANLIB=ranlib
>>
>> SYS=nag
>>
>> MKLPATH=/opt/intel/mkl/lib/intel64/
>>
>> SP_KIND=4
>> DP_KIND=8
>> KINDS=$(SP_KIND) $(DP_KIND)
>>
>> FFLAGS=-I/opt/intel/mkl/include/ -I/opt/intel/mkl/include/intel64/lp64/
>> FPPFLAGS= -DFC_HAVE_FLUSH -DFC_HAVE_ABORT
>>
>> COMP_LIBS=dc_lapack.a
>>
>> #SIESTA needs an F90 interface to MPI
>> #This will give you SIESTA's own implementation
>> #If your compiler vendor offers an alternative, you may change
>> #to it here.
>> MPI_INTERFACE=/opt/sgi/mpt/mpt-2.04/lib/libmpi.so
>> MPI_INCLUDE=/opt/sgi/mpt/mpt-2.04/include
>> MPI_LIBS =-L/opt/sgi/mpt/mpt-2.04/lib -lmpi
>> DEFS_MPI=-DMPI
>> DEFS= $(DEFS_MPI)
>>
>> LIBS=-Wl,--start-group $(MKLPATH)/libmkl_blas95_lp64.a \
>> $(MKLPATH)/libmkl_lapack95_lp64.a $(MKLPATH)/libmkl_scalapack_lp64.a \
>> $(MKLPATH)/libmkl_blacs_sgimpt_lp64.a $(MKLPATH)/libmkl_intel_lp64.a \
>> $(MKLPATH)/libmkl_sequential.a $(MKLPATH)/libmkl_core.a -Wl,--end-group \
>> -lpthread -lm
>>
>> #Dependency rules are created by autoconf according to whether
>> #discrete preprocessing is necessary or not.
>> .F.o:
>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $(DEFS) $<
>> .F90.o:
>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $(DEFS) $<
>> .f.o:
>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $<
>> .f90.o:
>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<
>>
>> 4)
>>
>> Makefile módosítás (nem túl szép, de nincs időm most szépen megcsinálni):
>>
>> roczei at service0:~/siesta-3.1/Obj> diff -u Makefile.old Makefile
>> --- Makefile.old 2012-09-29 08:58:56.868220812 +0200
>> +++ Makefile 2012-09-29 08:59:21.191647037 +0200
>> @@ -245,7 +245,11 @@
>> $(COMP_LIBS) $(ALL_OBJS)
>> $(FC) -o siesta \
>> $(LDFLAGS) $(ALL_OBJS) $(FDF) $(WXML) $(XMLPARSER) $(MPI_INTERFACE)\
>> - $(COMP_LIBS) $(FoX_LIBS) $(LIBS)
>> + $(COMP_LIBS) $(FoX_LIBS) $(LIBS) -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_blas95_lp64.a \
>> +/opt/intel/mkl/lib/intel64/libmkl_lapack95_lp64.a /opt/intel/mkl/lib/intel64/libmkl_scalapack_lp64.a \
>> +/opt/intel/mkl/lib/intel64/libmkl_blacs_sgimpt_lp64.a /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a \
>> +/opt/intel/mkl/lib/intel64/libmkl_sequential.a /opt/intel/mkl/lib/intel64/libmkl_core.a -Wl,--end-group \
>> +-lpthread -lm
>> #
>> #------------------------------------------------------------------------
>> #----------------------------------------------------------------------------
>> roczei at service0:~/siesta-3.1/Obj>
>>
>>
>> 5)
>>
>> make
>>
>> 6)
>>
>> Teszt:
>>
>> sule at service0:~/bin/SIESTA/siesta-3.1/Obj> /opt/sgi/mpt/mpt-2.04/bin/mpirun -np 10 ./siesta
>> Siesta Version: siesta-3.1
>> Architecture : x86_64-unknown-linux-gnu--Intel
>> Compiler flags: mpif90 -g
>> PARALLEL version
>>
>> * Running on 10 nodes in parallel
>>>> Start of run: 29-SEP-2012 9:24:59
>>
>> ***********************
>> * WELCOME TO SIESTA *
>> ***********************
>>
>> reinit: Reading from standard input
>> ************************** Dump of input data file ****************************
>>
>> Gábor
>>
>> On 2012.09.28., at 16:33, Süle Péter wrote:
>>
>>>
>>> Köszi. Sajnos az eredmény ua.
>>>
>>> MPI: This application was not compiled with a mpi.h or mpif.h header file from
>>> SGI's MPT MPI implementation.
>>> MPI: This application was not compiled with a mpi.h or mpif.h header file from
>>> SGI's MPT MPI implementation.
>>> MPI: This application was not compiled with a mpi.h or mpif.h h
>>>
>>> Üdv. Péter
>>>
>>> On Fri, 28 Sep 2012, Rőczei Gábor wrote:
>>>
>>>> Kedves Péter,
>>>>
>>>> Ezen leírás (http://www.mail-archive.com/siesta-l@uam.es/msg04389.html) alapján készítettem Neked egy arch.make fájlt:
>>>>
>>>> roczei at service0:~/siesta-3.1/Src> cat arch.make
>>>> # This is a dummy arch.make
>>>> # You should not compile Siesta in this directory.
>>>> # See Obj/README
>>>> #
>>>>
>>>> .SUFFIXES:
>>>> .SUFFIXES: .f .F .o .a .f90 .F90
>>>>
>>>> SIESTA_ARCH=x86_64-linux-Intel_MPI
>>>>
>>>> FC=mpif90
>>>> RANLIB=ranlib
>>>>
>>>> SYS=nag
>>>>
>>>> MKLPATH=/opt/intel/mkl/lib/intel64/
>>>>
>>>> SP_KIND=4
>>>> DP_KIND=8
>>>> KINDS=$(SP_KIND) $(DP_KIND)
>>>>
>>>> FFLAGS=-I/opt/intel/mkl/include/ -I/opt/intel/mkl/include/intel64/lp64/
>>>> FPPFLAGS= -DFC_HAVE_FLUSH -DFC_HAVE_ABORT
>>>>
>>>> COMP_LIBS=dc_lapack.a
>>>>
>>>> #SIESTA needs an F90 interface to MPI
>>>> #This will give you SIESTA's own implementation
>>>> #If your compiler vendor offers an alternative, you may change
>>>> #to it here.
>>>> MPI_INTERFACE=
>>>> MPI_INCLUDE=/opt/sgi/mpt/mpt-2.04/include
>>>> MPI_LIBS =-L/opt/sgi/mpt/mpt-2.04/lib -lmpi
>>>> DEFS_MPI=-DMPI
>>>> DEFS= $(DEFS_MPI)
>>>>
>>>> LIBS=-Wl,--start-group $(MKLPATH)/libmkl_blas95_lp64.a \
>>>> $(MKLPATH)/libmkl_lapack95_lp64.a $(MKLPATH)/libmkl_scalapack_lp64.a \
>>>> $(MKLPATH)/libmkl_blacs_intelmpi_lp64.a $(MKLPATH)/libmkl_intel_lp64.a \
>>>> $(MKLPATH)/libmkl_sequential.a $(MKLPATH)/libmkl_core.a -Wl,--end-group \
>>>> -lpthread -lm
>>>>
>>>> #Dependency rules are created by autoconf according to whether
>>>> #discrete preprocessing is necessary or not.
>>>> .F.o:
>>>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $(DEFS) $<
>>>> .F90.o:
>>>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $(DEFS) $<
>>>> .f.o:
>>>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $<
>>>> .f90.o:
>>>> $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<
>>>>
>>>> roczei at service0:~/siesta-3.1/Src>
>>>>
>>>> Ma több időt nem tudok ezzel foglalkozni. Légyszives próbáld meg ezzel.
>>>>
>>>> Hozzáadtam a CC-hez a hpc-forum-ot is, hátha valaki más is használ Siesta-t és tudja hogyan kell lefordítani.
>>>>
>>>> Gábor
>>>>
>>>>
>>>> On 2012.09.28., at 14:59, Süle Péter wrote:
>>>>
>>>>>
>>>>> Szia !
>>>>>
>>>>> Közben megpróbáltam sgi/mpt-2.04/bin/mpif90-el fordítani,
>>>>> de így is bajok vannak:
>>>>> Simán lefordul, linkelődik, de futáskor:
>>>>>
>>>>> /scratch/sule/SIESTA/gr/test/t4/mpirun -np 4 /home/sule/bin/SIESTA/siesta-3.1/Obj/siesta <gr.fdf
>>>>>
>>>>> MPI: This application was not compiled with a mpi.h or mpif.h header file from
>>>>> SGI's MPT MPI implementation.
>>>>> MPI: Global rank 0 is aborting with error code 0.
>>>>> Process ID: 10995, Host: service0, Program: /mnt/nfs1/home/sule/bin/SIESTA/siesta-3.1/Obj/siesta
>>>>>
>>>>> MPI: --------stack traceback-------
>>>>> MPI: This application was not compiled with a mpi.h or mpif.h header file from
>>>>> SGI's MPT MPI implementation.
>>>>> MPI: This application was not compiled with a mpi.h or mpif.h
>>>>>
>>>>> a következőt használtam:
>>>>> Compiler flags: /opt/sgi/mpt/mpt-2.04/bin/mpif90 -g -O2 -I/opt/sgi/mpt/mpt-2.04/include
>>>>>
>>>>> Elvileg meg kellene találnia az mpi.h és mpif.h file-t
>>>>> az /opt/sgi/mpt/mpt-2.04/include helyen. De valami miatt
>>>>> nem teszi. Ez nekem rendszer hiba gyanús, vagy más
>>>>> nem kód specifikus probléma.
>>>>> Ja és volt: mpi-selector --set mpt-2.04
>>>>>
>>>>> hely:
>>>>> /home/sule/bin/SIESTA/siesta-3.1/Obj/siesta
További információk a(z) Hpc-forum levelezőlistáról