Skip to content

Slurm report

Automatic usage report

You can generate accounting statistics from Slurm using the slurmreport script, located in /ceph/hpc/bin/.

Usage:

/ceph/hpc/bin/slurmreport [-m] [-c | -w] [-a accountlist] [-s startdate -e enddate] [-r report-directory]

where:

-m: Send report by E-mail to root@localhost

-a: Select an account (or list of accounts)

-s -e: Select Start and End dates of the report

-c: Select current month from the 1st until today

-w: Select the last week

-r: Copy the report to a specified directory as well
./slurmreport -s 2021-08-18 -e 2021-08-01

Date format could be either YYYY-MM-DD or MMDD

If you run the script without specifying additional features, the report will be generated for the last month and placed into your home directory.

Manual usage report

Create an accounting report manually using sreport command:

Usage report in CPU Hours

Report cluster account utilization by filing specified fields for the desired period of time start/end:

sreport cluster AccountUtilizationByUser -T billing user=$USER -t Hours start=2021-01-01 end=2021-05-25 format=Proper%20,Login%20,Accounts,Used

Example:

sreport cluster AccountUtilizationByUser -T billing user=user -t Hours start=2021-01-01 end=2021-10-25 format=Proper%20,Login%20,Accounts,Used
--------------------------------------------------------------------------------
Cluster/Account/User Utilization 2021-01-01T00:00:00 - 2021-10-07T14:59:59 (24156000 secs)

Usage reported in CPU Hours
--------------------------------------------------------------------------------
         Proper Name                Login         Account        Used
-------------------- -------------------- --------------- -----------
          User User                 user        vega-users     5009241

Used=5009241 is in Thread Hours.

To get result in Core Hours divide the number by 2.

Usage report in TRES Hours

Fill specified fields for the desired period of time start/end:

Create report for your project account:

ACC=$(sacctmgr show assoc cluster=vega user=$USER format=account%40 -n) ; ACC=$(tr -d ' ' <<< "$ACC") ; sreport --tres billing cluster AccountUtilizationByAccount account=$ACC -t Hours start=2021-01-01 end=2022-01-01 format=Proper%20,Login%20,Accounts%30,Used

Create report for your user account only

sreport -T billing cluster UserUtilizationByAccount -t Hours start=2021-01-01 end=2022-01-01 user=$USER format=Proper%20,Login%15,Account%40,Used%20

Usage report in CPU Hours for account

Fill specified fields for the desired period of time start/end:

Create report for your project account:

sreport -t hourper --tres=billing cluster AccountUtilizationByUser Account=eurohpc-2010paXXX-users Start=2021-07-01 End=`date -d tomorrow +%Y-%m-%d` format=Accounts%25,Login%20,Used

Example:

$ sreport -t hourper --tres=billing cluster AccountUtilizationByUser Account=eurohpc-2010paXXXX-users Start=2021-07-01 End=`date -d tomorrow +%Y-%m-%d` format=Accounts%25,Login%20,Used
--------------------------------------------------------------------------------
Cluster/Account/User Utilization 2021-07-01T00:00:00 - 2022-03-24T13:59:59 (23036400 secs)
Usage reported in TRES Hours/Percentage of Total
--------------------------------------------------------------------------------
                  Account                Login                Used
------------------------- -------------------- -------------------
 eurohpc-2010paXXXX-users                                 1919017(0.07%)
 eurohpc-2010paXXXX-users             user1                  5316(0.00%)
 eurohpc-2010paXXXX-users             user2                791721(0.03%)
 eurohpc-2010paXXXX-users             user3                  4064(0.00%)
 eurohpc-2010paXXXX-users             user4                851589(0.03%)
 eurohpc-2010paXXXX-users             user5                266326(0.01%)

Used=1919017 is in Thread Hours.

To get result in Core Hours divide the number by 2.

This example shows the total usage of all resources on (cpu and gpu partition) for the project (1919017) and is divided by users.

User1, user2, user3, user4, and user5 can view their own usage by partition. For usage on cpu partition is used the following command:

sreport job SizesByAccount -T billing User=user1 -t Hours Start=2021-07-01 End=`date -d tomorrow +%Y-%m-%d` account=eurohpc-2010pa5872-users grouping=9999999 Partition=cpu format=Account%25

--------------------------------------------------------------------------------
Job Sizes 2021-07-01T00:00:00 - 2022-03-24T13:59:59 (23036400 secs)
TRES type is billing
Time reported in Hours
--------------------------------------------------------------------------------
                  Account 0-9999998 TRE >= 9999999 TR % of cluster
------------------------- ------------- ------------- ------------
 eurohpc-2010pa5872-users             1             0      100.00%

Output is shows that user1 had used 1 Thread Hours, or 0.5 Core Hours.

For usage on gpu partition is used following command:

$ sreport job SizesByAccount -T billing User=user1 -t Hours Start=2021-07-01 End=`date -d tomorrow +%Y-%m-%d` account=eurohpc-2010pa5872-users grouping=9999999 Partition=gpu format=Account%25
--------------------------------------------------------------------------------
Job Sizes 2021-07-01T00:00:00 - 2022-03-24T13:59:59 (23036400 secs)
TRES type is billing
Time reported in Hours
--------------------------------------------------------------------------------
                  Account 0-9999998 TRE >= 9999999 TR % of cluster
------------------------- ------------- ------------- ------------
 eurohpc-2010pa5872-users          5315             0      100.00%

Output is shows that user1 had used 5315 Thread Hours, or 2657.5 Core Hours.

Summarizing: user1 had usage of 1 Thread Hours on cpu partition and 5135 Thread Hours on gpu partition, whats gives 5316 total Thread Hours.