hooks

Hooks for adding tags, filtering and setting job resources in ReFrame tests

assign_tasks_per_compute_unit(test, compute_unit, num_per=1)

Assign one task per compute unit. More than 1 task per compute unit can be assigned with num_per for compute units that support it. Automatically sets num_tasks, num_tasks_per_node, num_cpus_per_task, and num_gpus_per_node, based on the current scale and the current partition’s num_cpus, max_avail_gpus_per_node and num_nodes. For GPU tests, one task per GPU is set, and num_cpus_per_task is based on the ratio of CPU-cores/GPUs. For CPU tests, one task per CPU is set, and num_cpus_per_task is set to 1. Total task count is determined based on the number of nodes to be used in the test. Behaviour of this function is (usually) sensible for MPI tests.

WARNING: when using COMPUTE_UNITS.HWTHREAD and invoking a hook for process binding, please verify that process binding happens correctly.

Arguments: - test: the ReFrame test to which this hook should apply - compute_unit: a device as listed in eessi.testsuite.constants.COMPUTE_UNITS

Examples: On a single node with 2 sockets, 64 cores and 128 hyperthreads: - assign_tasks_per_compute_unit(test, COMPUTE_UNITS.HWTHREAD) will launch 128 tasks with 1 thread per task - assign_tasks_per_compute_unit(test, COMPUTE_UNITS.CPU) will launch 64 tasks with 2 threads per task - assign_tasks_per_compute_unit(test, COMPUTE_UNITS.CPU_SOCKET) will launch 2 tasks with 64 threads per task

extract_memory_usage(test)

Extract the memory in MiB from the job output file as written by hook measure_memory_usage() To Use this hook, add the following method to your test class:

@performance_function('MiB', perf_key='memory') def extract_memory_usage(self): return hooks.extract_memory_usage(self)

filter_supported_scales(test)

Filter tests scales based on which scales are supported by each partition in the ReFrame configuration. Filtering is done using features, i.e. the current test scale is requested as a feature. Any partition that does not include this feature in the ReFrame configuration file will effectively be filtered out.

filter_valid_systems_by_device_type(test, required_device_type)

Filter valid_systems by required device type and by whether the module supports CUDA, unless valid_systems is specified with --setvar valid_systems=.

Any invalid combination (e.g. a non-CUDA module with a required_device_type GPU) will cause the valid_systems to be set to an empty string, and consequently the test.valid_systems to an invalid system name (eessi.testsuite.constants.INVALID_SYSTEM).

measure_memory_usage(test)

Write the memory usage into the job output file if we are in a Slurm job and if cgroups is enabled in Slurm First try to obtain the memory with cgroups v2, if that fails try with cgroups v1 (v2 takes precedence) Intended to be used in tandem with hook extract_memory_usage() To use this hook, add the following method to your test class:

@run_after('init') def measure_memory_usage(self): "Measure memory usage" hooks.measure_memory_usage(self)

req_memory_per_node(test, app_mem_req)

This hook will request a specific amount of memory per node to the batch scheduler. First, it computes which fraction of CPUs is requested from a node, and how much the corresponding (proportional) amount of memory would be. Then, the hook compares this to how much memory the application claims to need per node (app_mem_req). It then passes the maximum of these two numbers to the batch scheduler as a memory request.

Note: using this hook requires that the ReFrame configuration defines system.partition.extras[EXTRAS.MEM_PER_NODE] That field should be defined in GiB

Arguments: - test: the ReFrame test to which this hook should apply - app_mem_req: the amount of memory this application needs (per node) in MiB

Example 1: - A system with 128 cores and 64,000 MiB per node. - The test is launched on 64 cores - The app_mem_req is 40,000 (MiB) In this case, the test requests 50% of the CPUs. Thus, the proportional amount of memory is 32,000 MiB. The app_mem_req is higher. Thus, 40,000 MiB (per node) is requested from the batch scheduler.

Example 2: - A system with 128 cores per node, 128,000 MiB mem per node. - The test is launched on 64 cores - the app_mem_req is 40,000 (MiB) In this case, the test requests 50% of the CPUs. Thus, the proportional amount of memory is 64,000 MiB. This is higher than the app_mem_req. Thus, 64,000 MiB (per node) is requested from the batch scheduler.

set_compact_process_binding(test)

This hook sets a binding policy for process binding. More specifically, it will bind each process to subsequent domains of test.num_cpus_per_task cores.

Arguments: - test: the ReFrame test to which this hook should apply

A few examples: - Pure MPI (test.num_cpus_per_task = 1) will result in binding 1 process to each core. this will happen in a compact way, i.e. rank 0 to core 0, rank 1 to core 1, etc - Hybrid MPI-OpenMP, e.g. test.num_cpus_per_task = 4 will result in binding 1 process to subsequent sets of 4 cores. I.e. rank 0 to core 0-3, rank 1 to core 4-7, rank 2 to core 8-11, etc

It is hard to do this in a portable way. Currently supported for process binding are: - Intel MPI (through I_MPI_PIN_DOMAIN) - OpenMPI (through OMPI_MCA_rmaps_base_mapping_policy) - srun (LIMITED SUPPORT: through SLURM_CPU_BIND, but only effective if task/affinity plugin is enabled)

set_compact_thread_binding(test)

This hook sets a binding policy for thread binding. It sets a number of environment variables to try and set a sensible binding for OPENMP tasks.

Thread binding is supported for: - GNU OpenMP (through OMP_NUM_THREADS, OMP_PLACES and OMP_PROC_BIND) - Intel OpenMP (through KMP_AFFINITY)

set_modules(test)

Skip current test if module_name is not among a list of modules, specified with --setvar modules=.

set_omp_num_threads(test)

Set number of OpenMP threads equal to number of CPUs per task

set_tag_scale(test)

Set resources and tag based on current scale