Migrating from PBS to IBM LoadLeveller
To help users migrate from the PBS job submission system (used by IU's AVIDD cluster) to IBM's LoadLeveller (LL) job submission system (used by Big Red and Libra clusters), the following tables show commands, environment variables and resource specifications of two batch systems side by side.
If you are a LoadLeveller user, looking for equivalent options on PBS, then please refer to the Migrating from IBM LoadLeveller to PBS page.
If you're not familiar with either LL or PBS, and are new to Big Red, then please refer to the Working on Big Red primer.
Commonly used commands
| In PBS, if you use.. | .. then, in LL, use | |
|---|---|---|
| Job submission | qsub <scriptfile> | llsubmit <scriptfile> |
| Job deletion | qdel <job_id> | llcancel <job_id> |
| Job status (for user) | qstat -u <username> | llq -u <username> |
| Extended Job status | qstat -f <job_id> | llq -l <job_id> |
| Hold a job temporarily | qhold <job_id> | llhold <job_id> |
| Resume job on hold | qrls <job_id> | llhold -r <job_id> |
| List of usable queues | qstat -Q | llclass |
| GUI for batch system | xpbs | xloadl |
Environment variables
| In PBS, if you use.. | .. then, in LL, use | |
|---|---|---|
| Job ID | $PBS_JOBID | $LOADL_STEP_ID |
| Submission Directory | $PBS_O_WORKDIR | $LOADL_STEP_INITDIR |
| Processor list | $PBS_NODEFILE | $LOADL_PROCESSOR_LIST (See note below) |
Important Note about $LOADL_PROCESSOR_LIST:
For jobs using more than 128 tasks (processors/cores),
the value of $LOADL_PROCESSOR_LIST will be set to NULL. If your job uses
more than 128 tasks, then you should run:
/opt/ibmll/LoadL/full/bin/ll_get_machine_list > /tmp/machinelist.$LOADL_STEP_IDand then use the temporary file (/tmp/machinelist.$LOADL_STEP_ID) created to get the list of machines assigned to your job.
Resource specifications
| In PBS, if you use.. | .. then, in LL, use | |
|---|---|---|
| Queue | #PBS -q <queue> | #@ class=<queue> |
| Nodes | #PBS -l nodes=<#> | #@ node=<#> |
| Processors | #PBS -l ppn=<#> | #@ tasks_per_node=<#> |
| Wall clock limit | #PBS -l walltime=<hh:mm:ss> | #@ wall_clock_limit=<hh:mm:ss> |
| Standard output file | #PBS -o <file> | #@ output=<file> |
| Standard error | #PBS -e <file> | #@ error=<file> |
| Copy Environment | #PBS -V | #@ environment=COPY_ALL |
| Notification event | #PBS -m abe | #@ notification=start|error| complete|never|always |
| Email address | #PBS -M <email> | #@ notify_user=<email> |
| Job name | #PBS -N <name> | #@ job_name=<name> |
| Job restart | #PBS -r <y|n> | #@ restart=<yes|no> |
| Job type | N/A | #@ job_type=<type> |
| Initial directory | N/A | #@ initialdir=<directory> |
| Node usage | N/A? | #@ node_usage=not_shared |
| Memory requirement | N/A | #@ requirements=(Memory >= NumKiloBytes) |
For more information, check out the LLGuide document for BigRed and Libra, and the PBS Guide for AVIDD.
Common Moab scheduler commands
The BigRed, AVIDD and Libra clusters all use the Moab job scheduler. Frequently used Moab scheduler commands include:
| Command | LoadLeveler & PBS |
|---|---|
| Show currently running/queued jobs | showq | less |
| Check a job's status | checkjob <jobid> or checkjob -v <jobid> (for details) |
| Show when your job might start | showstart <job_id> |
| Show fairshare information (priority, etc.) | diagnose -f | less |
| Check a nodes status | checknode <nodename> |
| Show current reservations | showres |
For more information on Moab, refer to the Moab Workload Manager User's Manual.
