Question: MUMPS best practice?
- Ewout Lagerweij
- Topic Author
- Offline
- New Member
-
Less
More
- Posts: 19
- Thank you received: 1
11 years 9 months ago #7028
by Ewout Lagerweij
Question: MUMPS best practice? was created by Ewout Lagerweij
I want to solve a large linear elastic simulation with approximately 10e6 degrees of freedom.
The simulation contains second order elements and approx 100 beams to distribute the loads.
I am using code-aster-11.3-mpi from Caelinux 2013 on one machine with 8 cores (~4GHz) and 32 Gb of ram.
However I am a bit puzzled on the various options described in the documentation on this topic.
I would like some recommendations of best practices for such simulation.
Some questions, which arise are:
What configuration is needed in astk i.e. what values do i set for mpi_nbcpu and
mpi_nbnoeud?
And what values do i need to set in the .comm file besides setting the solveur parameter to MUMPS.
The simulation contains second order elements and approx 100 beams to distribute the loads.
I am using code-aster-11.3-mpi from Caelinux 2013 on one machine with 8 cores (~4GHz) and 32 Gb of ram.
However I am a bit puzzled on the various options described in the documentation on this topic.
I would like some recommendations of best practices for such simulation.
Some questions, which arise are:
What configuration is needed in astk i.e. what values do i set for mpi_nbcpu and
mpi_nbnoeud?
And what values do i need to set in the .comm file besides setting the solveur parameter to MUMPS.
- Joël Cugnoni
-
- Offline
- Moderator
-
11 years 9 months ago #7037
by Joël Cugnoni
Joël Cugnoni - a.k.a admin
www.caelinux.com
Replied by Joël Cugnoni on topic Re: Question: MUMPS best practice?
Hello,
To use Mumps (and thus mpi parallelism) with this config, you should:
- in astk, set nbcpu=1 (to disable openmp parallelism)
Mpi_nbcpu=8 (use 8 cpus for mpi parallelism)
mpi_nbnoeud=1 (only use 1pc, not in a cluster)
-then for mumps solver options, the default should work, but if you experience problems, try to increase pct_pivot to 100. If 32gb is not enough to avoid swapping, you can also force Mumps to use "out of core" storage (store factor matrices to disk to save ram).
If you are running a non linear solution, a faster (but maybe less robust) solver could be Petsc with mumps single precision preconditioner (someting like precond=SP in eficas).
Hope this helps
Joel
To use Mumps (and thus mpi parallelism) with this config, you should:
- in astk, set nbcpu=1 (to disable openmp parallelism)
Mpi_nbcpu=8 (use 8 cpus for mpi parallelism)
mpi_nbnoeud=1 (only use 1pc, not in a cluster)
-then for mumps solver options, the default should work, but if you experience problems, try to increase pct_pivot to 100. If 32gb is not enough to avoid swapping, you can also force Mumps to use "out of core" storage (store factor matrices to disk to save ram).
If you are running a non linear solution, a faster (but maybe less robust) solver could be Petsc with mumps single precision preconditioner (someting like precond=SP in eficas).
Hope this helps
Joel
Joël Cugnoni - a.k.a admin
www.caelinux.com
- Ewout Lagerweij
- Topic Author
- Offline
- New Member
-
Less
More
- Posts: 19
- Thank you received: 1
11 years 9 months ago - 11 years 9 months ago #7039
by Ewout Lagerweij
Replied by Ewout Lagerweij on topic Re: Question: MUMPS best practice?
I have tried to run my simulation with the standard parameters for mumps on Caelinux 2011 and i get the following error.
This error occurs after an hour of solving time.
When i google for this error, i can only find this topic (www.code-aster.org/forum2/viewtopic.php?id=18526), but it seems to deal with an other problem.
(I also described my problem on the Code_Aster forum (www.code-aster.org/forum2/viewtopic.php?id=18772))
What can be the cause of this error?
This error occurs after an hour of solving time.
When i google for this error, i can only find this topic (www.code-aster.org/forum2/viewtopic.php?id=18526), but it seems to deal with an other problem.
(I also described my problem on the Code_Aster forum (www.code-aster.org/forum2/viewtopic.php?id=18772))
What can be the cause of this error?
[METIS Fatal Error] ***Memory allocation failed for CompressGraph: gdata. Requested size: -1030832780 bytes
[caelinux:13863] *** Process received signal ***
[caelinux:13863] Signal: Aborted (6)
[caelinux:13863] Signal code: (-6)
[caelinux:13863] [ 0] /lib/libpthread.so.0(+0xf8f0) [0x7f9d21aef8f0]
[caelinux:13863] [ 1] /lib/libc.so.6(gsignal+0x35) [0x7f9d20a7aa75]
[caelinux:13863] [ 2] /lib/libc.so.6(abort+0x180) [0x7f9d20a7e5c0]
[caelinux:13863] [ 3] ./asteru() [0x1de48a3]
[caelinux:13863] [ 4] ./asteru(__idxmalloc+0x46) [0x1de4c36]
[caelinux:13863] [ 5] ./asteru(__CompressGraph+0x408) [0x1dfc428]
[caelinux:13863] [ 6] ./asteru(METIS_NodeND+0x2b1) [0x1def811]
[caelinux:13863] [ 7] ./asteru(dmumps_195_+0x22e9) [0x19d9168]
[caelinux:13863] [ 8] ./asteru(dmumps_26_+0x1bc4) [0x1a2b7b5]
[caelinux:13863] [ 9] ./asteru(dmumps_+0x1544) [0x19b62d9]
[caelinux:13863] [10] ./asteru(amumpd_+0x85b) [0x633ceb]
[caelinux:13863] [11] ./asteru(amumph_+0x12c5) [0x575e45]
[caelinux:13863] [12] ./asteru(tldlg3_+0x93d) [0x968b3d]
[caelinux:13863] [13] ./asteru(preres_+0x815) [0x8020d5]
[caelinux:13863] [14] ./asteru(meacmv_+0x15a4) [0x151b134]
[caelinux:13863] [15] ./asteru(mereso_+0x51f) [0xeb347f]
[caelinux:13863] [16] ./asteru(mestat_+0xaf6) [0x7f42a6]
[caelinux:13863] [17] ./asteru(op0046_+0x6dd) [0x6b951d]
[caelinux:13863] [18] ./asteru(ex0000_+0x41c) [0x5a242c]
[caelinux:13863] [19] ./asteru(execop_+0x47d) [0x554edd]
[caelinux:13863] [20] ./asteru(expass_+0x266) [0x522806]
[caelinux:13863] [21] ./asteru() [0x51d057]
[caelinux:13863] [22] /usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x51a0) [0x7f9d250eb030]
[caelinux:13863] [23] /usr/lib/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5a98) [0x7f9d250eb928]
[caelinux:13863] *** End of error message ***
EXECUTION_CODE_ASTER_EXIT_13716-caelinux=134
Last edit: 11 years 9 months ago by Ewout Lagerweij.
Moderators: catux
Time to create page: 0.140 seconds