libmuscle.native_instantiator.slurm module

class libmuscle.native_instantiator.slurm.NCEVisitor

Bases: NodeVisitor

Processes a parsed NCE and produces a list of cpu counts per node.

Nodes cores expressions are used by SLURM to describe cores on a collection of nodes. See parse_slurm_nodes_cores() below.

generic_visit(node: Node, visited_children: Sequence[Any]) → Sequence[Any]

Default visitor method

Parameters:

• node – The node we’re visiting

• visited_children – The results of visiting the children of that node, in a list

I’m not sure there’s an implementation of this that makes sense across all (or even most) use cases, so we leave it to subclasses to implement for now.

visit_int(node: Node, _: Sequence[Any]) → int

Returns the value as an int

visit_nce(node: Node, visited_children: Tuple[List[int], Sequence[Tuple[Any, List[int]]]]) → List[int]

Return a list of nodes corresponding to the NRE.

visit_nce_run(node: Node, visited_children: Tuple[int, Sequence[Tuple[Any, int, Any]]]) → List[int]

Return a list of core counts produced by this run.

visit_run_length(node: Node, visited_children: Tuple[str, int]) → int

Return the number of repetitions.

class libmuscle.native_instantiator.slurm.NREVisitor

Bases: NodeVisitor

Processes a parsed NRE and produces a list of nodes.

Node range expressions are used by SLURM to describe collections of nodes. See parse_slurm_nodelist() below.

generic_visit(node: Node, visited_children: Sequence[Any]) → Sequence[Any]

Default visitor method

Parameters:

• node – The node we’re visiting

• visited_children – The results of visiting the children of that node, in a list

I’m not sure there’s an implementation of this that makes sense across all (or even most) use cases, so we leave it to subclasses to implement for now.

visit_identifier(node: Node, _: Sequence[Any]) → str

visit_index_range(node: Node, visited_children: Tuple[Tuple[int, int], Sequence[Tuple[Any, Tuple[int, int]]]]) → List[str]

Return a list of indexes corresponding to the range.

visit_index_set(node: Node, visited_children: Tuple[List[str], Sequence[Tuple[Any, List[str]]]]) → List[str]

Return a list of indexes corresponding to the set.

visit_int(node: Node, _: Sequence[Any]) → Tuple[int, int]

Returns the value and a field width of -1.

visit_integer(node: Node, visited_children: Sequence[Tuple[int, int]]) → Tuple[int, int]

Returns the value of the int, and a field width or -1.

visit_nre(node: Node, visited_children: Tuple[List[str], Sequence[Tuple[Any, List[str]]]]) → List[str]

Return a list of nodes corresponding to the NRE.

visit_nre_part(node: Node, visited_children: Tuple[str, Sequence[Tuple[Any, List[str], Any]]]) → Tuple[str, List[str]]

Return the identifier part and a list of indexes for the set.

visit_nre_parts(node: Node, visited_children: Sequence[Tuple[str, List[str]]]) → List[str]

Return a list of node ids for the part.

visit_padded_int(node: Node, _: Sequence[Any]) → Tuple[int, int]

Returns the value of the int and the field width.

class libmuscle.native_instantiator.slurm.SlurmInfo

Bases: object

Detects and holds information about the present SLURM scheduler.

agent_launch_command(agent_cmd: List[str], nnodes: int) → List[str]

Return a command for launching one agent on each node.

Parameters:

agent_cmd – A command that will start the agent.

get_logical_cpus_per_node() → List[int]

Return the number of logical CPU cores per node.

This returns a list with the number of cores of each node in the result of get_nodes(), which gets read from SLURM_JOB_CPUS_PER_NODE.

get_nodes() → List[str]

Get a list of node names from SLURM_JOB_NODELIST.

This inspects SLURM_JOB_NODELIST or SLURM_NODELIST and returns an expanded list of node names.

If SLURM_JOB_NODELIST is “node[020-023]” then this returns [“node020”, “node021”, “node022”, “node023”].

in_slurm_allocation() → bool

Check whether we’re in a SLURM allocation.

Returns true iff SLURM was detected.

class libmuscle.native_instantiator.slurm.SlurmQuirks

Bases: object

Collects features of the present SLURM.

cpu_bind: str

CPU binding argument, –cpu-bind or –cpu_bind.

overlap: bool

True iff –overlap must be specified for srun.

libmuscle.native_instantiator.slurm.parse_slurm_nodelist(s: str) → List[str]

Parse a SLURM node range expression and produce node names.

Exactly what the syntax is for a “node range expression” isn’t entirely clear. Some examples are given throughout the documentation:

linux[00-17] lx[10-20] tux[2,1-2] tux[1-2,2] tux[1-3] linux[0-64,128] alpha,beta,gamma lx[15,18,32-33] linux[0000-1023] rack[0-63]_blade[0-41]

unit[0-31]rack is invalid

If a range uses leading zeros, then so should the generated indexes. See _node_range_expression_grammar above for my best guess at the correct grammar.

This function takes a string containing an NRE and returns the corresponding list of node names.

libmuscle.native_instantiator.slurm.parse_slurm_nodes_cores(s: str) → List[int]

Parse a SLURM nodes cores expression and produce node names.

The sbatch documentation page describes the format under SLURM_JOB_CPUS_PER_NODE as CPU_count[(xnumber_of_nodes)][,CPU_count [(xnumber_of_nodes)] …]. and gives the example of ‘72(x2),36’ describing a set of three nodes, the first two with 72 cores and the third with 36.

See _nodes_cores_expression_grammar above for the corresponding grammar.

This function takes a string containing an NCE and returns the corresponding list of node names.

libmuscle.native_instantiator.slurm.slurm() → SlurmInfo

SlurmInfo singleton