[Slurm] How to use

Ref) https://www.hpc2n.umu.se/batchsystem/examples_scripts

Ref) http://slurm.schedmd.com/sbatch.html

Step 1. Make a script for a single job.

You can make a script file for a single job.

Below is an example script file (test.sh) 

test.sh

      #!/bin/sh

      #SBATCH --time=30-00:00:00    //minimum acceptable walltime (runtime), format: day-hours:minutes:seconds

      #SBATCH --job-name Landscape_10  //name of job shown in squeue

      #SBATCH --ntasks=1                     //CPU resources

      #SBATCH --ntasks-per-node=1        //CPU resources

      #SBATCH --cpus-per-task=1            //CPU resources

      #SBATCH -o ./%J.output.txt             //for output file

      #SBATCH -e ./%J.error.txt               //for error file

      ./Landscape 10                            //running program and argument

Almost everything will be the same for the other jobs except the walltime, name of jobs, and running command.

Step 2. Submit your job (script file)

Just type below command in your command line

username@server:~/$ sbatch ./test.sh

Step 3. Check your job state

username@server:~/$ squeue -u username  //put your username in "username" position

Or you can use pipe and grep

username@server:~/$ qstat |grep username //put your username in "username" position

Step 4. (Optional) Manipulate many jobs

I usually use a C file for making script files and submit file (run.sh).

Below is the code for making those files.

int main(int argc, char* argv[])

      {

          FILE* fp;

          FILE* fp1;

          char file[64];

          fp = fopen( "run.sh" , "w");

     

          for(int t=10; t<=20; t+=1)

          {

              sprintf(file,"script/t%d.sh", t);

              fp1 = fopen(file, "w");

              fprintf(fp, "sbatch %s\n", file);

     

              fprintf(fp1, "#!/bin/sh\n");

              fprintf(fp1, "# Set your minimum acceptable walltime, format: day-hours:minutes:seconds\n");

             fprintf(fp1, "#SBATCH --time=30-00:00:00\n");

              fprintf(fp1, "# Set name of job shown in squeue\n");

              fprintf(fp1, "#SBATCH --job-name Landscape_%d\n", t);

              fprintf(fp1, "# Request CPU resources\n");

              fprintf(fp1, "#SBATCH --ntasks=1\n");

              fprintf(fp1, "#SBATCH --ntasks-per-node=1\n");

              fprintf(fp1, "#SBATCH --cpus-per-task=1\n");

              fprintf(fp1, "#SBATCH -o ./%J.output.txt\n");

              fprintf(fp1, "#SBATCH -e ./%J.error.txt\n");

              fprintf(fp1, "./Landscape %d\n", t);

              fclose(fp1);

          }

          fclose(fp);

          return 0;

     }

When I implement the above program, the submit file "run.sh" 

run.sh

      sbatch script/t10.sh

      sbatch script/t11.sh

      sbatch script/t12.sh

      sbatch script/t13.sh

      sbatch script/t14.sh

      sbatch script/t15.sh

      sbatch script/t16.sh

      sbatch script/t17.sh

      sbatch script/t18.sh

      sbatch script/t19.sh

      sbatch script/t20.sh

and script files are created on the script folder (if you directly use above C program, you have to make script folder in advance).

Then, you can submit the jobs using just one command line as follows:

username@server:~/$ sh run.sh

If you prefer python language, then you can make your own program. 

It doesn't matter what language you used. Most important thing is that you can make your submit file and script files at the same time using just one program. Then, just submit all jobs using one command line.