'SLURM Require GPU Resource Request

Is there a way to require jobs to have the --gres=gpu:X on sbatch and srun job submissions? Either have the jobs require that flag or set a default number of gpus per job like DefMemPerNode.

My slurm.conf is currently set to

SchedulerType=sched/backfill
SelectType=select/cons_res
SelectTypeParameters=CR_Core

We want users to always define gpu reservation.

If there's no way through slurm configuration, is there a way to force gpu usage through slurm only and prevent access to gpus without submitting a job?



Solution 1:[1]

You can modify the job submission on the fly using the Job Submit plugin interface. Slurm can run custom Lua script at submission time and that script can modify the resource requests as well as other job parameters. See an example here.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 damienfrancois