CoEPP RC
 

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
cloud:advanced_staging [2013/02/28 16:33]
neilds [Another way to stage input and output of many runs]
cloud:advanced_staging [2013/07/30 13:40]
rwilson [Advanced Staging]
Line 1: Line 1:
 ===== Advanced Staging ===== ===== Advanced Staging =====
 +
 +<​code>​
 +This page is obsolete. ​ You don't need to do any staging to use the cloud.
 +</​code>​
  
 The simple introduction to staging showed how to stage a file to a worker node, generate a single output file and stage the output file back to the original directory. The simple introduction to staging showed how to stage a file to a worker node, generate a single output file and stage the output file back to the original directory.
Line 137: Line 141:
 The above method to stage the //output// of many similar jobs does work, but the '​unpicking'​ of the ''​PBS_JOBID''​ environment variable is a little fiddly, and that approach doesn'​t lend itself to staging //input// files. ​ So what to do? The above method to stage the //output// of many similar jobs does work, but the '​unpicking'​ of the ''​PBS_JOBID''​ environment variable is a little fiddly, and that approach doesn'​t lend itself to staging //input// files. ​ So what to do?
  
-We can use the **-t** option to ''​qsub''​. ​ If we look at the options to ''​qsub''​ (''​man qsub''​),​ we see:+We can use the **''​-t''​** option to ''​qsub''​. ​ If we look at the options to ''​qsub''​ (''​man qsub''​),​ we see:
 <​code>​ <​code>​
 -t array_request -t array_request
Line 191: Line 195:
   1542[].c3torque ​           array_test ​      ​rwilson ​               0 R short          ​   1542[].c3torque ​           array_test ​      ​rwilson ​               0 R short          ​
  
-Note that we don't see a set of five job numbers. ​ Instead we see a single number with a **[]** suffix, showing that this is an //array job//.+Note that we don't see a set of five job numbers. ​ Instead we see a single number with a **''​[]''​** suffix, showing that this is an //array job//.
  
 Once ''​qstat''​ shows that the array job has finished, we see the output directories:​ Once ''​qstat''​ shows that the array job has finished, we see the output directories:​
cloud/advanced_staging.txt · Last modified: 2013/07/30 13:41 by rwilson
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki