Managing Computational Resources

Resources

Hosts

class qmpy.Host(*args, **kwargs)

Host model - stores all host information for a cluster.

Relationships:

account

allocation

Attributes:

name: Primary key.

binaries: dict of label:path pairs for vasp binaries.

check_queue: Path to showq command

checked_time: datetime object for the last time the queue was

checked.

hostname: Full host name.

ip_address: Full ip address.

nodes: Total number of nodes.

ppn: Number of processors per node.

running: dict of PBS_ID:state pairs.

sub_script: Path to qsub command

sub_text: Path to queue file template.

utilization: Number of active cores (based on showq).

walltime: Maximum walltime on the machine.

state: State code. 1=Up, 0=Full (auto-resets to 1 when jobs are

collected), -1=Down.

exception DoesNotExist

exception MultipleObjectsReturned

activate()

Allow jobs to be run on this system. Remember to save() to enact change

check_host()

Pings the host to see if it is online. Returns False if it is offline.

check_running()

Uses the hosts data and one of the associated accounts to check the PBS queue on the Host. If it has been checked in the last 2 minutes, it will return the previously returned result.

static create()

Classmethod to create a Host model. Script will ask you questions about the host to add, and will return the created Host.

deactivate()

Prevent new jobs from being started on this system. Remember to save() changes

get_project(maxuse=400)

Out of the active projects able to run on this host,

select one at random

Output:

Project, Active project able to run on this host

Accounts

class qmpy.Account(*args, **kwargs)

Base class for a User account on a Host.

Attributes:

host

id

job

run_path

state

user

username

exception DoesNotExist

exception MultipleObjectsReturned

Users

class qmpy.User(*args, **kwargs)

User model - stores an oqmd users information.

Relationships:

Account via account_set

Allocation via allocation_set

Project via project_set

Attributes:

id

username

first_name

last_name

date_joined

is_active

is_staff

is_superuser

last_login

email

exception DoesNotExist

exception MultipleObjectsReturned

Allocation

class qmpy.Allocation(*args, **kwargs)

Base class for an Allocation on a computing resources.

Attributes:

host

job

key

name

project

state

users

exception DoesNotExist

exception MultipleObjectsReturned

Project

class qmpy.Project(*args, **kwargs)

Base class for a project within qmpy.

Attributes:

allocations

entry

name

priority

state

task

users

exception DoesNotExist

exception MultipleObjectsReturned

static create()

Create a new project. Prompts user on std-in for name, users, and allocations of this project.

Queue models

Task

class qmpy.Task(*args, **kwargs)

Model for a :Task: to be done.

A :Task: consists of a module, which is the name of a computing script, and a set of keyword arguments, specified as a dictionary as the kwargs attribute of the task. In order for a Task for be completed, it must also be assigned one or more :Project:s.

Relationships:

Entry via entry

Job via job_set

Project via project_set

Attributes:

id

created: datetime object for when the task was created.

finished: datetime object for when the task was completed.

module: The name of a function in scripts

kwargs: dict of keyword:value pairs to pass to the calculation

module.

priority: Priority of the task. Lower values are more urgent.

state: State code, given by the table below.

Task codes:

Code	Description
-2	being held
-1	encountered error
0	ready to run
1	jobs running
2	completed

exception DoesNotExist

exception MultipleObjectsReturned

complete()

Sets the Task state to 2 and populates the finished field.

property errors

List of errors encountered by related calculations.

get_jobs(project=None, allocation=None, account=None, host=None)

Check the calculation module specified by the Task, and returns a list of Job objects accordingly.

Calls the task’s entry’s “do” method with the Task.module as the first argument, and passing Task.kwargs as keyword arguments.

Returns:

List of Job objects. When nothing is left to do for the task, returns empty.

Raises:

ResourceUnavailableError:

Raise if for the specified project, allocation, account and/or host there are no available cores.

property jobs

List of jobs related to the task.

property projects

List of related projects.

save(*args, **kwargs)

Save the current instance. Override this in a subclass if you want to control the saving process.

The ‘force_insert’ and ‘force_update’ parameters can be used to insist that the “save” must be an SQL insert or update (or equivalent for non-SQL backends), respectively. Normally, they should not be set.

Job

class qmpy.Job(*args, **kwargs)

Base class for job submitted to a compute cluster.

Relationships:

Task via task

Account via account. The account the calculation is

performed on.

Allocation via allocation. The allocation on which the

calculation is being performed.

Entry via entry

Attributes:

id

created: datetime object for when the task was created.

finished: datetime object for when the task was completed.

ncpus: # of processors assigned.

path: Origination path of the calculation.

run_path: Path of the calculation on the compute resource.

qid: PBS queue ID number.

walltime: Max walltime (in seconds).

state: State code, defined as in the table below.

Job codes

Code	Description
-1	encountered error
0	ready to submit
1	currently running
2	completed

exception DoesNotExist

exception MultipleObjectsReturned