Tips for allocating Job Resources
Cherry-creek currently two types of compute nodes:
| cp |
48 "penguin" cherry-creek 2 (Penguin Computing Relion Servers) nodes (2 Intel Xeon E5-2640v3 8 core, 128Gb ram) |
| ci |
24 "waterfall" cherry-creek 2 (Intel Servers) nodes (2 Intel Xeon E5-2697v2 12core, 192Gb ram) Please do not specifically request "ci" nodes unless your jobs need the additional memory. |
The "ci" nodes are typically very busy with long running jobs. If you
request that your
jobs run on them, they may wait a significant amount of time before the "ci"
modes will become available.
Example resource specifications
The following are commonly used resource allocation parameters:
| walltime=# |
this is the TOTAL wall time that the job will be allowed to run (format: HH:MM:SS (hours, minutes, seconds)). There are limits placed on different job queues. The normal queue (workq) is limited to 744 hours (approx. 30 days), and the test queue (small) is limited to 15 minutes. |
| select=# |
this tells PBS/Pro that the following settings are to be applied to this number of separate nodes (known as a "chunk"). |
| ncpus=# | specifies the number of # cpu cores to be assigned to this chunk. |
| mpiprocs=# |
specifies the number of # MPI processes assigned to this chunk. (Normally this is the same as the ncpus value). |
| mem=# | specifies the amount of memory (in mb, gb, etc) to be assigned to this chunk. |
| cput=# |
specifies the amount of cpu time for the chunk (format: HH:MM:SS (hours, minutes, seconds)). A reasonable value for cpu time is the number of mpiprocs*walltime. |
| Qlist=string | Specifies the type of node to run on cp,ci are the current choices. |
For example, your MPI job requires 2 chunks of four cpu cores.
It also requires 30Gb of memory and you want it to run on a ci node.
The resource request would look like:
#PBS -q workq
#PBS -l walltime=24:00:00
#PBS -l select=2:ncpus=4:mpiprocs=4:mem=30gb:cput=96:00:00:Qlist=ci
Also, the system requires a minimum of 4Gb of ram for the operating system on each node,
so do not allocate more than 124Gb on the cp nodes, or 188Gb on the ci nodes.
