availableCores {parallelly} | R Documentation |
The current/main R session counts as one, meaning the minimum number of cores available is always at least one.
availableCores( constraints = NULL, methods = getOption2("parallelly.availableCores.methods", c("system", "nproc", "mc.cores", "_R_CHECK_LIMIT_CORES_", "PBS", "SGE", "Slurm", "LSF", "fallback", "custom")), na.rm = TRUE, logical = getOption2("parallelly.availableCores.logical", TRUE), default = c(current = 1L), which = c("min", "max", "all") )
constraints |
An optional character specifying under what
constraints ("purposes") we are requesting the values.
For instance, on systems where multicore processing is not supported
(i.e. Windows), using |
methods |
A character vector specifying how to infer the number of available cores. |
na.rm |
If TRUE, only non-missing settings are considered/returned. |
logical |
Passed to
|
default |
The default number of cores to return if no non-missing settings are available. |
which |
A character specifying which settings to return.
If |
The following settings ("methods") for inferring the number of cores are supported:
"system"
-
Query detectCores(logical = logical)
.
"nproc"
-
On Unix, query system command nproc
.
"mc.cores"
-
If available, returns the value of option
mc.cores
.
Note that mc.cores is defined as the number of
additional R processes that can be used in addition to the
main R process. This means that with mc.cores = 0
all
calculations should be done in the main R process, i.e. we have
exactly one core available for our calculations.
The mc.cores option defaults to environment variable
MC_CORES (and is set accordingly when the parallel
package is loaded). The mc.cores option is used by for
instance mclapply()
of the parallel
package.
"PBS"
-
Query TORQUE/PBS environment variables PBS_NUM_PPN and NCPUS.
Depending on PBS system configuration, these resource
parameter may or may not default to one.
An example of a job submission that results in this is
qsub -l nodes=1:ppn=2
, which requests one node with two cores.
"SGE"
-
Query Sun/Oracle Grid Engine (SGE) environment variable
NSLOTS.
An example of a job submission that results in this is
qsub -pe smp 2
(or qsub -pe by_node 2
), which
requests two cores on a single machine.
"Slurm"
-
Query Simple Linux Utility for Resource Management (Slurm)
environment variable SLURM_CPUS_PER_TASK.
This may or may not be set. It can be set when submitting a job,
e.g. sbatch --cpus-per-task=2 hello.sh
or by adding
#SBATCH --cpus-per-task=2
to the ‘hello.sh’ script.
If SLURM_CPUS_PER_TASK is not set, then it will fall back to
use SLURM_CPUS_ON_NODE if the job is a single-node job
(SLURM_JOB_NUM_NODES is 1), e.g. sbatch --ntasks=2 hello.sh
.
"LSF"
-
Query Platform Load Sharing Facility (LSF) environment variable
LSB_DJOB_NUMPROC.
Jobs with multiple (CPU) slots can be submitted on LSF using
bsub -n 2 -R "span[hosts=1]" < hello.sh
.
"custom"
-
If option parallelly.availableCores.custom is set and a function,
then this function will be called (without arguments) and it's value
will be coerced to an integer, which will be interpreted as a number
of available cores. If the value is NA, then it will be ignored.
For any other value of a methods
element, the R option with the
same name is queried. If that is not set, the system environment
variable is queried. If neither is set, a missing value is returned.
Return a positive (>= 1) integer.
If which = "all"
, then more than one value may be returned.
Together with na.rm = FALSE
missing values may also be returned.
Note that availableCores()
may return a single core. Because of this,
using something like:
ncores <- availableCores() - 1
may return zero, which is often not intended. Instead, use:
ncores <- max(1, availableCores() - 1)
It is possible to override the maximum number of cores on the machine
as reported by availableCores(methods = "system")
. This can be
done by first specifying
options(parallelly.availableCores.methods = "mc.cores")
and
then the number of cores to use, e.g. options(mc.cores = 8)
.
To get the set of available workers regardless of machine,
see availableWorkers()
.
message(paste("Number of cores available:", availableCores())) ## Not run: options(mc.cores = 2L) message(paste("Number of cores available:", availableCores())) ## End(Not run) ## Not run: ## IMPORTANT: availableCores() may return 1L options(mc.cores = 1L) ncores <- max(1, availableCores() - 1) message(paste("Number of cores to use:", ncores)) ## End(Not run) ## Not run: ## Use 75% of the cores on the system but never more than four options(parallelly.availableCores.custom = function() { ncores <- max(parallel::detectCores(), 1L, na.rm = TRUE) min(0.75 * ncores, 4L) }) message(paste("Number of cores available:", availableCores())) ## What is available minus one core but at least one options(parallelly.availableCores.custom = function() { max(1L, parallelly::availableCores() - 1L) }) message(paste("Number of cores available:", availableCores())) ## End(Not run)