Notes on Torque - Revision history

Administrator at 18:08, 2 November 2015

2015-11-02T18:08:01Z

← Older revision		Revision as of 13:08, 2 November 2015
Line 10:		Line 10:

	<ul>		<ul>
	<li>qb is a succinct queue monitoring script that tallies each user's running and queued jobs, total cores used and free, and down nodes (if any). The script was originally written by Daniel Sinkovits and should be available to copy from another user's bin to your own. Maintenance includes keeping the username list up-to-date and adjusting the variable <tt>node_total</tt> (~~currenlty~~ 37) if a node goes down permanently.		<li>qb is a succinct queue monitoring script that tallies each user's running and queued jobs, total cores used and free, and down nodes (if any). The script was originally written by Daniel Sinkovits and should be available to copy from another user's $HOME/bin folder to your own. Maintenance includes keeping the username list up-to-date and adjusting the variable <tt>node_total</tt> (currently 37) if a node goes down permanently.
	<li>If qb indicates a node is down, check ~~the~~ ~~ganglia of~~ Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with		<li>If qb indicates a node is down, check Ganglia on Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with
	<pre>		<pre>
	qstat -f \| grep -B 3 [node_ID]		qstat -f \| grep -B 3 [node_ID]
Line 27:		Line 27:
	</pre>		</pre>
	where <tt>[destination]</tt> is the new queue (either 'fast' or 'default' for our system) and <tt>[job_id]</tt> is the job ID.		where <tt>[destination]</tt> is the new queue (either 'fast' or 'default' for our system) and <tt>[job_id]</tt> is the job ID.
	<li>Delete sequence of jobs via		<li>Delete a sequence of jobs via
	<pre>		<pre>
	qdel `seq [min_job_id] [max_job_id]`		qdel `seq [min_job_id] [max_job_id]`

Bernie: /* Special usage notes */

2015-07-08T15:07:41Z

Special usage notes

← Older revision		Revision as of 10:07, 8 July 2015
Line 10:		Line 10:

	<ul>		<ul>
	<li>qb is a succinct queue monitoring script that tallies each user's running and queued jobs, total cores used and free, and down nodes (if any). The script was originally written by Daniel Sinkovits and should be available to copy from ~~any other~~ user's bin to your own. Maintenance includes keeping the username list up-to-date and adjusting the variable <tt>node_total</tt> (currenlty 37) if a node goes down permanently.		<li>qb is a succinct queue monitoring script that tallies each user's running and queued jobs, total cores used and free, and down nodes (if any). The script was originally written by Daniel Sinkovits and should be available to copy from another user's bin to your own. Maintenance includes keeping the username list up-to-date and adjusting the variable <tt>node_total</tt> (currenlty 37) if a node goes down permanently.
	<li>If qb indicates a node is down, check the ganglia of Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with		<li>If qb indicates a node is down, check the ganglia of Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with
	<pre>		<pre>

Bernie: /* Special usage notes */

2015-07-08T15:07:17Z

Special usage notes

← Older revision		Revision as of 10:07, 8 July 2015
Line 10:		Line 10:

	<ul>		<ul>
	<li>qb is a succinct queue monitoring script that tallies each user's running and queued jobs, total cores used and free, and down nodes (if any). The script was originally written by Daniel Sinkovits and should be available to copy from any other user's ~~directory~~ to your own. Maintenance includes keeping the username list up-to-date and adjusting the variable <tt>node_total</tt> (currenlty 37) if a node goes down permanently.		<li>qb is a succinct queue monitoring script that tallies each user's running and queued jobs, total cores used and free, and down nodes (if any). The script was originally written by Daniel Sinkovits and should be available to copy from any other user's bin to your own. Maintenance includes keeping the username list up-to-date and adjusting the variable <tt>node_total</tt> (currenlty 37) if a node goes down permanently.
	<li>If qb indicates a node is down, check the ganglia of Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with		<li>If qb indicates a node is down, check the ganglia of Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with
	<pre>		<pre>

Bernie: /* Special usage notes */

2015-07-08T15:06:52Z

Special usage notes

← Older revision		Revision as of 10:06, 8 July 2015
Line 10:		Line 10:

	<ul>		<ul>
			<li>qb is a succinct queue monitoring script that tallies each user's running and queued jobs, total cores used and free, and down nodes (if any). The script was originally written by Daniel Sinkovits and should be available to copy from any other user's directory to your own. Maintenance includes keeping the username list up-to-date and adjusting the variable <tt>node_total</tt> (currenlty 37) if a node goes down permanently.
	<li>If qb indicates a node is down, check the ganglia of Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with		<li>If qb indicates a node is down, check the ganglia of Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with
	<pre>		<pre>

Bernie at 20:47, 25 June 2015

2015-06-25T20:47:41Z

← Older revision		Revision as of 15:47, 25 June 2015
Line 1:		Line 1:
	=== Overview ===		=== Overview ===

			The CSML uses Torque with Maui for most of its job scheduling. Please see the Cambridge Dept. of Chemistry's [http://www.ch.cam.ac.uk/computing/maui-and-torque-introduction user guide] for information on submitting and checking jobs, and the associated [http://www.ch.cam.ac.uk/computing/maui-administration admin guide] for more job scheduling tools.


	=== General usage ===		=== General usage ===

Bernie: /* Special usage notes */

2015-01-28T20:31:42Z

Special usage notes

← Older revision		Revision as of 15:31, 28 January 2015
Line 28:		Line 28:
	<li>Delete sequence of jobs via		<li>Delete sequence of jobs via
	<pre>		<pre>
	qdel `seq [~~job_id1~~] [~~job_id2~~]`		qdel `seq [min_job_id] [max_job_id]`
	</pre>		</pre>
	where <tt>[~~job_id1~~]</tt> is the first job ID of the sequence of jobs you want to delete and <tt>[~~job_id2~~]</tt> is the last one.		where <tt>[min_job_id]</tt> is the first job ID of the sequence of jobs you want to delete and <tt>[max_job_id]</tt> is the last one.
	</ul>		</ul>

Bernie: /* Special usage notes */

2015-01-28T20:30:41Z

Special usage notes

← Older revision		Revision as of 15:30, 28 January 2015
Line 19:		Line 19:
	qalter -l cput=[new_cput] [job_id]		qalter -l cput=[new_cput] [job_id]
	</pre>		</pre>
	where <tt>[new_cput]</tt> is the desired amount of cpu time and <tt>[job_id]</tt> is the job's ID number. Multiple jobs can be changed simultaneously by using <tt>`seq [min_job_id] [max_job_id]`</tt> in the place of <tt>job_id</tt>. This will change all jobs <tt>[min_job_id]</tt> through <tt>[max_job_id]</tt>. Lowering the cpu time requirement of a job can decrease its wait time in the queue, as the scheduler is more likely to be able to use if for backfilling. On the other hand, increasing the cpu time requirement can be used to ensure that a job is able to finish properly (and can be done even while the job is running), but requires root permission.		where <tt>[new_cput]</tt> is the desired amount of cpu time and <tt>[job_id]</tt> is the job's ID number. Multiple jobs can be changed simultaneously by using <tt>`seq [min_job_id] [max_job_id]`</tt> in the place of <tt>job_id</tt>. This will change all jobs <tt>[min_job_id]</tt> through <tt>[max_job_id]</tt>. Lowering the cpu time requirement of a job can decrease its wait time in the queue, as the scheduler is more likely to be able to use it for backfilling. On the other hand, increasing the cpu time requirement can be used to ensure that a job is able to finish properly (and can be done even while the job is running), but requires root permission.
	</li>		</li>
	<li>Move a queued (i.e., waiting) job to a different queue via		<li>Move a queued (i.e., waiting) job to a different queue via

Bernie: /* Special usage notes */

2014-06-13T02:30:48Z

Special usage notes

← Older revision		Revision as of 21:30, 12 June 2014
Line 10:		Line 10:

	<ul>		<ul>
	<li>If qb indicates a node is down, check the ganglia of ~~[http://minotaur.research.northwestern.edu/ganglia/index.php~~ Minotaur] or ~~[https://hydra.research.northwestern.edu/ganglia/~~ Hydra] to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with		<li>If qb indicates a node is down, check the ganglia of Minotaur or Hydra to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with
	<pre>		<pre>
	qstat -f \| grep -B 3 [node_ID]		qstat -f \| grep -B 3 [node_ID]

Bernie: /* Special usage notes */

2014-06-13T02:23:27Z

Special usage notes

← Older revision		Revision as of 21:23, 12 June 2014
Line 10:		Line 10:

	<ul>		<ul>
			<li>If qb indicates a node is down, check the ganglia of [http://minotaur.research.northwestern.edu/ganglia/index.php Minotaur] or [https://hydra.research.northwestern.edu/ganglia/ Hydra] to find which node it is (down nodes are in red at the top of the list). Grep the output of qstat to find the jobs running on any down nodes with
			<pre>
			qstat -f \| grep -B 3 [node_ID]
			</pre>
			where <tt>[node_ID]</tt> is the name of the down node, e.g., <tt>h036</tt> on Hydra. Make sure the owners of all jobs on the node have a chance to take note of which of their jobs went down and then restart the down nodes using Microway control.
	<li>Change the total cpu time allotted to a job via		<li>Change the total cpu time allotted to a job via
	<pre>		<pre>

Wzhwei at 22:28, 9 June 2014

2014-06-09T22:28:19Z

← Older revision		Revision as of 17:28, 9 June 2014
Line 21:		Line 21:
	</pre>		</pre>
	where <tt>[destination]</tt> is the new queue (either 'fast' or 'default' for our system) and <tt>[job_id]</tt> is the job ID.		where <tt>[destination]</tt> is the new queue (either 'fast' or 'default' for our system) and <tt>[job_id]</tt> is the job ID.
	<li>Delete sequence of jobs		<li>Delete sequence of jobs via
	<pre>		<pre>
	qdel `seq [job_id1] [job_id2]`		qdel `seq [job_id1] [job_id2]`