Running Hybrid MPI/OpenMP Simulations: Difference between revisions
| Line 3: | Line 3: | ||
==OpenMP setting== |
==OpenMP setting== |
||
One of the main settings that we need be careful about is the "processor affinity". By default, multi-threading affinity is not set in mvapich2. That means all your OpenMP threading will be bound to a single processor if you |
One of the main settings that we need be careful about is the "processor affinity". By default, multi-threading affinity is not set in mvapich2. That means all your OpenMP threading will be bound to a single processor if you have not changed this setting (if you have compiled it through mpicc, mpic++ or mpif90). This is because mvapich2 gives priority to MPI threads and does not allow OpenMP to use more than one CPU (even if there are available cpus for OpenMP threads). This setting can be changed through MV2_ENABLE_AFFINITY variable. In your terminal or PBS file type in the following command to enable OpenMP threads to run on different cores: |
||
*MV2_ENABLE_AFFINITY=0 |
*MV2_ENABLE_AFFINITY=0 |
||
Revision as of 05:25, 4 November 2015
Running Hybrid MPI/OpenMP Simulations
This tutorial explains how to run a multi-threaded hybrid MPI/OpenMP simulation using mvapich2 library. mvapich2 is currently installed on both MC2 and WCR clusters. In this page we assume the code is already written in the hybrid fashion, e.g. both MPI and OpenMP headers are included in the code and you are able to compile and run your code using either MPI or OpenMP.
OpenMP setting
One of the main settings that we need be careful about is the "processor affinity". By default, multi-threading affinity is not set in mvapich2. That means all your OpenMP threading will be bound to a single processor if you have not changed this setting (if you have compiled it through mpicc, mpic++ or mpif90). This is because mvapich2 gives priority to MPI threads and does not allow OpenMP to use more than one CPU (even if there are available cpus for OpenMP threads). This setting can be changed through MV2_ENABLE_AFFINITY variable. In your terminal or PBS file type in the following command to enable OpenMP threads to run on different cores:
- MV2_ENABLE_AFFINITY=0