Things we could do

  1. Make the examples (eg. in A Walk in the Cloud) more closely match a realistic workflow: compilation of the script/tool we want to use, testing on the interactive nodes, followed by finally getting the thing to run in the cloud.
  2. Lazy staging: copy-on-write unionfs mount over a caching read-only network filesystem of some kind. Remote files only retrieved when first accessed, otherwise reads/writes local. Thus big compiles can be executed without having to copy the entire directory tree if not all the source files are required. Intermediate files are written to the local filesystem (presumably a temporary directory), and the final output can be staged out as normal.
  3. Maybe have some examples demonstrating the use of scp or rsync for in-script staging, as in, if that seems useful
  4. Compilation: ross suggested some examples of compiling things in the cloud. In fact, the torque docs suggest using distcc for distributed compilation, but not sure if this could be installed on our worker nodes without it conflicting/slowing down torque jobs by using up resources too eagerly.
  5. Mailing list: “cxsupport”. It would be nice to have a public forum of some kind for users to submit questions/problems and have them answered, with the answers preserved for posterity in the archives, as well as hopefully written up on the wiki as example use cases for the future.
  6. FAQ - will probably grow out of “cxsupport” questions (we want to move it to its own page from cloud once it gets to any real size)
  7. In general we want pre-made recipes for known use-cases so that our users have to do as little thinking as possible!
cloud/ideas.txt · Last modified: 2013/03/01 17:24 by neilds
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki