Shared Memory Jobs

Shared Memory Parallelization relies on threading techniques such as OpenMP, where indiviual threads, each running on (preferably) their own CPU core, access the same RAM area.

In SLURM, the option --cpus-per-task or, short, -c is key.

As the threads are initiated by the execution of a program, this is best viewed as one single task. Hence, keep the option -n (or --ntasks in its long form) at its default value of 1.

Code

Consider the following C++ example source code file simpson.cpp. It numerically calculates $\int_1^{100.000}{x^2}\,dx$ using the Simpson's Rule, where each simpson step can be calculated independent of the others. The result, however, is the sum of all simpson steps and needs to be available to each thread in order to execute the addition. The calculation is parallelized in a shared memory fashion using OpenMP.

#include <iostream>
#include <math.h>
using namespace std;

// Simpson rule
double simpson(double dx, double y1, double y2, double my) {
  return 1./6. * dx * (y1 + 4. * my + y2);
}

// Our f(x) function. In this case x^2
double f_x(double x) {
  return pow(x, 2.);
}

// Main program
int main() {
  uint64_t steps = 4e9;              // number of integration steps
  double a = 1.;                     // Integral starting from a
  double b = 1e5;                    // Integral ending at b
  double dx = (b - a) / steps;       // step width

  double result = 0.;                // result

  // The result has to be reduced by all threads
  #pragma omp parallel for reduction (+: result)
  for (uint64_t i=0; i<steps; i++) {
    double x1 = a + i * dx;          // this will be private for each thread
    result += simpson(dx, f_x(x1), f_x(x1 + dx), f_x((x1 + x1 + dx) / 2.));
  }

  cout << "result is: " << result << endl;
}

Compile the code with the GCC compiler and use the OpenMP library:

$ g++ -fopenmp -o simpson simpson.cpp

Submit job

To run the resulting binary on the cluster in a shared memory way, write a job script openmp.sh:

#!/bin/bash

#SBATCH -J MyOpenMPJob
#SBATCH -c 16

export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}

srun ./simpson

The environment variable OMP_NUM_THREADS controls how many thread the program will spawn. Here we set it to the the number of CPUs per task that we request for the job. During job execution that requested number of CPUs is also stored in an environment variable SLURM_CPUS_PER_TASK set by SLURM at job start.

Submit the job like any other:

$ sbatch openmp.sh