Sydney T3

• Authentication is provided by the CoEPP central authentication system. This consists of an LDAP directory and a Kerberos KDC.

• In order to access the Sydney Tier 3 you must be part of the ui_syd UNIX group.

• The following picture provides an ilustration of the proposed model for local usage:

1. Users should preferencially login-in through sui.coepp.org.au. - sui.coepp.org.au is an alias to sydui1.syd.coepp.org.au, which is the preferable frontend interface for the Sydney cluster. 2. sydui1.syd.coepp.org.au can be used to execute interactive applications. - If sydui1.syd.coepp.org.au is overloaded, users can also directly login to sydui[2,3,4].syd.coepp.org.au. 3. sydui1.syd.coepp.org.au can also be used to submit long batch jobs. - Sydney cluster provides 40 physical cores for batch executions accessible through four different queues (syd_extralong, syd_long, syd_medium and syd_short). - If no queue is specified, jobs will execute in syd_medium (default queue). 4. The '\imports\sydpp1' and '\imports\sydpp2' filesystems are exported through all the nodes.

• For information how users can access and manage spefic software in the Tier-3, please check Tier-3 Software Management

-bash-4.1$ qstat -q
server: sydtorque.syd.coepp.org.au

Queue            Memory CPU Time Walltime Node  Run Que Lm  State
---------------- ------ -------- -------- ----  --- --- --  -----
syd_extralong      --      --    336:00:0   --    0   0 --   E R
syd_short          --      --    02:00:00   --    0   0 --   E R
syd_medium         --      --    10:00:00   --    4   0 --   E R
syd_long           --      --    96:00:00   --    0   0 --   E R
                                               ----- -----
                                                   4     0

$ qmgr -c 'p s' | grep default_queue
set server default_queue = syd_medium

• Sydney users can access CERN.CH AFS systems from all sydui machines.

 
$ df
(...)
AFS                                      2147483647           0 2147483647   0% /afs

$ cd /afs/cern.ch/; ls -la
(...)

• To access to private AFS areas, you will have to be authenticated against AFS filesystem at CERN. That is done via Kerberos.

• The sequence of commands which allows you to authenticate is the following:
  1. Get a kerberos ticket: kinit <afsuser>@CERN.CH (it will ask for your AFS CERN / LXPLUS password)
  2. List your kerberos ticket: klist
  3. Translate the ticket to an afs token: aklog

$ kinit goncalo@CERN.CH
Password for goncalo@CERN.CH:

$ klist
Ticket cache: FILE:/tmp/krb5cc_1051_sucfqR
Default principal: goncalo@CERN.CH
Valid starting     Expires            Service principal
12/19/14 04:45:55  12/20/14 04:45:55  krbtgt/CERN.CH@CERN.CH
    renew until 12/26/14 04:45:55

$ aklog

$ ls /afs/cern.ch/user/g/goncalo
(...)

• The sydpp data filesystems (/home, /export/sydpp1 and /export/sydpp2) are not available in the School of Physics desktops since December 2014.

• The best way to access to data on those filesystems is to login in syduis machines. The performance will always be better using this setup.

• However, if for sporadic activities you need a direct access to sydpp filesystems in your labtop, you can use SSHFS.

• The SSHFS is a filesystem client based on the SSH File Transfer Protocol. Since most SSH servers already support this protocol it is very easy to set up: i.e. on the server side there's nothing to do. On the client side mounting the filesystem is as easy as logging into the server with ssh.

• Most of the standard linux distribution already offer an SSHFS RPM or DEB.
  • On your personnel laptop: just run 'yum install fuse-sshfs' or 'apt-get install fuse-sshfs' (the epel repo must be enabled)

• To run it, you do not need root permissions.

• sshfs will work from your personnel laptops, and from the school of physics desktops.

• Check that i am not a privileged user

$ whoami
goncalo

• Check my partitions

$ df
Filesystem                   1K-blocks      Used Available Use% Mounted on
/dev/mapper/fedora_ozzy-root 100660656   6691476  88832796   8% /
devtmpfs                       4045160         0   4045160   0% /dev
tmpfs                          4052620       204   4052416   1% /dev/shm
tmpfs                          4052620      9196   4043424   1% /run
tmpfs                          4052620         0   4052620   0% /sys/fs/cgroup
tmpfs                          4052620        12   4052608   1% /tmp
/dev/sda6                       991512    114448    809480  13% /boot
/dev/sda2                       262144    114444    147700  44% /boot/efi
/dev/mapper/fedora_ozzy-home 288210280 195558820  77988192  72% /home

• Create a directory, and invoke sshfs to mount on that directory. User is authenticated via ssh password:

$ mkdir ~goncalo/SSHFS

$ sshfs goncalo@sydui4.syd.coepp.org.au:/home/goncalo ~goncalo/SSHFS
goncalo@sydui4.syd.coepp.org.au's password:

$ df
Filesystem                                      1K-blocks        Used  Available Use% Mounted on
/dev/mapper/fedora_ozzy-root                    100660656     6691476   88832796   8% /
devtmpfs                                          4045160           0    4045160   0% /dev
tmpfs                                             4052620         204    4052416   1% /dev/shm
tmpfs                                             4052620        9196    4043424   1% /run
tmpfs                                             4052620           0    4052620   0% /sys/fs/cgroup
tmpfs                                             4052620          12    4052608   1% /tmp
/dev/sda6                                          991512      114448     809480  13% /boot
/dev/sda2                                          262144      114444     147700  44% /boot/efi
/dev/mapper/fedora_ozzy-home                    288210280   195558848   77988164  72% /home
goncalo@sydui4.syd.coepp.org.au:/home/goncalo 19527641056 13986740864 5540900192  72% /home/goncalo/SSHFS <---

• Perform operations on that remote dir

$ cd /home/goncalo/SSHFS

$ ll
total 65540
drwxr-xr-x 1 1051 1011       52 Dec  8 01:35 BELLE
-rw-r--r-- 1 1051 1011 67108864 Dec 11 00:56 testfile

$ echo "HELLO SSHFS" > greeting.txt

$ ll
total 65544
drwxr-xr-x 1 1051  1011       52 Dec  8 01:35 BELLE
-rw-r--r-- 1 1051 10000       12 Dec 18 05:47 greeting.txt
-rw-r--r-- 1 1051  1011 67108864 Dec 11 00:56 testfile

• Once finished, just unmount the dir

$ cd; fusermount -u ~goncalo/SSHFS

$ df
Filesystem                   1K-blocks      Used Available Use% Mounted on
/dev/mapper/fedora_ozzy-root 100660656   6691480  88832792   8% /
devtmpfs                       4045160         0   4045160   0% /dev
tmpfs                          4052620       204   4052416   1% /dev/shm
tmpfs                          4052620      9196   4043424   1% /run
tmpfs                          4052620         0   4052620   0% /sys/fs/cgroup
tmpfs                          4052620        16   4052604   1% /tmp
/dev/sda6                       991512    114448    809480  13% /boot
/dev/sda2                       262144    114444    147700  44% /boot/efi
/dev/mapper/fedora_ozzy-home 288210280 195558856  77988156  72% /home

• Running X applications and forwarding remote X windows is enabled through XPRA

• Long term data storage and archiving at Sydney