Skip to end of metadata
Go to start of metadata

Compiling

Compilation of CUDA or OpenACC jobs must be performed on the GPU nodes. Therefore, you must run an interactive job to compile. Crane has 2 shared GPU partitions: gpu_k20 and gpu_m2070. An example command to compile in the GPU gpu_k20 partition could be:

$ srun --partition=gpu_k20 --gres=gpu --mem-per-cpu=1024 --ntasks-per-node=6 --nodes=1 --pty $SHELL

The above command will start a shell on a GPU node with 6 cores and 6GB of ram in order to compile a GPU job.  The above command could also be useful if you want to run a test GPU job interactively.

You may request multiple gpus by changing the --gres value to --gres=gpu:2. Note that this value is per node. For example, --nodes=2 --gres=gpu:2 will request 2 nodes with 2 GPUs each, for a total of 4 GPUs.

Submitting Jobs

CUDA and OpenACC submissions require running on GPU nodes.

cuda.submit

#!/bin/sh
#SBATCH --time=03:15:00
#SBATCH --mem-per-cpu=1024
#SBATCH --job-name=cuda
#SBATCH --partition=gpu_k20
#SBATCH --gres=gpu
#SBATCH --error=/work/[groupname]/[username]/job.%J.err
#SBATCH --output=/work/[groupname]/[username]/job.%J.out

 

module load cuda/6.5

./cuda-app.exe

 

OpenACC submissions require loading the PGI compiler (which is currently required to compile as well).

openacc.submit

#!/bin/sh
#SBATCH --time=03:15:00
#SBATCH --mem-per-cpu=1024
#SBATCH --job-name=cuda-acc
#SBATCH --partition=gpu_k20
#SBATCH --gres=gpu
#SBATCH --error=/work/[groupname]/[username]/job.%J.err
#SBATCH --output=/work/[groupname]/[username]/job.%J.out

 

module load cuda/6.5 compiler/pgi/16

./acc-app.exe

  • No labels