CoEPP RC
 

Sydney T3

User Access and Authentication

  • Authentication is provided by the CoEPP central authentication system. This consists of an LDAP directory and a Kerberos KDC.
  • In order to access the Sydney Tier 3 you must be part of the ui_syd UNIX group.

Resources

Topology

  • The following picture provides an ilustration of the proposed model for local usage:
1. Users should preferencially login-in through sui.coepp.org.au.
- sui.coepp.org.au is an alias to sydui1.syd.coepp.org.au, which is the preferable frontend interface for the Sydney cluster.

2. sydui1.syd.coepp.org.au can be used to execute interactive applications.
- If sydui1.syd.coepp.org.au is overloaded, users can also directly login to sydui[2,3,4].syd.coepp.org.au.

3. sydui1.syd.coepp.org.au can also be used to submit long batch jobs.
- Sydney cluster provides 40 physical cores for batch executions accessible through four different queues (syd_extralong, syd_long, syd_medium and syd_short).
- If no queue is specified, jobs will execute in syd_medium (default queue).

4. The '\imports\sydpp1' and '\imports\sydpp2' filesystems are exported through all the nodes.

Servers and Hardware

Server Role CPUs Mem
sydui1.syd.coepp.org.au
(alias: sui.coepp.org.au)
User Interface to the grid, main interactive node and Sydney Tier 3 batch system node 12 CPUs 24 GB RAM
sydui2.syd.coepp.org.au User Interface to the grid, shared interactive node 2 and batch node 2 12 CPUs 24 GB RAM
sydui3.syd.coepp.org.au User Interface to the grid, shared interactive node 3 and batch node 3 12 CPUs 24 GB RAM
sydui4.syd.coepp.org.au User Interface to the grid, shared interactive node 4 and batch node 4 12 CPUs 24 GB RAM
Hostname sydui01.syd.coepp.org.au
sydui02.syd.coepp.org.au
sydui03.syd.coepp.org.au
sydui04.syd.coepp.org.au
Vendor Dell
Model 4 x PowerEdge R410
OS Scientific Linux 6
Photo
Hostname sydpp.syd.coepp.org.au (NFS Server)
Vendor Dell
Model PowerEdge R510
PowerVault MD1200 Arrays
OS Scientific Linux 6
Photo

Software Management

Batch System

Queues and Times

-bash-4.1$ qstat -q
server: sydtorque.syd.coepp.org.au

Queue            Memory CPU Time Walltime Node  Run Que Lm  State
---------------- ------ -------- -------- ----  --- --- --  -----
syd_extralong      --      --    336:00:0   --    0   0 --   E R
syd_short          --      --    02:00:00   --    0   0 --   E R
syd_medium         --      --    10:00:00   --    4   0 --   E R
syd_long           --      --    96:00:00   --    0   0 --   E R
                                               ----- -----
                                                   4     0

Default Queue

$ qmgr -c 'p s' | grep default_queue
set server default_queue = syd_medium

AFS

Introduction

  • Sydney users can access CERN.CH AFS systems from all sydui machines.
 
$ df
(...)
AFS                                      2147483647           0 2147483647   0% /afs

$ cd /afs/cern.ch/; ls -la
(...)

Kerberos Tickets and AFS Tokens

  • To access to private AFS areas, you will have to be authenticated against AFS filesystem at CERN. That is done via Kerberos.
  • The sequence of commands which allows you to authenticate is the following:
    1. Get a kerberos ticket: kinit <afsuser>@CERN.CH (it will ask for your AFS CERN / LXPLUS password)
    2. List your kerberos ticket: klist
    3. Translate the ticket to an afs token: aklog
$ kinit goncalo@CERN.CH
Password for goncalo@CERN.CH:

$ klist
Ticket cache: FILE:/tmp/krb5cc_1051_sucfqR
Default principal: goncalo@CERN.CH
Valid starting     Expires            Service principal
12/19/14 04:45:55  12/20/14 04:45:55  krbtgt/CERN.CH@CERN.CH
    renew until 12/26/14 04:45:55

$ aklog

$ ls /afs/cern.ch/user/g/goncalo
(...)

SSHFS

Introduction

  • The sydpp data filesystems (/home, /export/sydpp1 and /export/sydpp2) are not available in the School of Physics desktops since December 2014.
  • The best way to access to data on those filesystems is to login in syduis machines. The performance will always be better using this setup.
  • However, if for sporadic activities you need a direct access to sydpp filesystems in your labtop, you can use SSHFS.
  • The SSHFS is a filesystem client based on the SSH File Transfer Protocol. Since most SSH servers already support this protocol it is very easy to set up: i.e. on the server side there's nothing to do. On the client side mounting the filesystem is as easy as logging into the server with ssh.
  • Most of the standard linux distribution already offer an SSHFS RPM or DEB.
    • On your personnel laptop: just run 'yum install fuse-sshfs' or 'apt-get install fuse-sshfs' (the epel repo must be enabled)
  • To run it, you do not need root permissions.
  • sshfs will work from your personnel laptops, and from the school of physics desktops.

How to run SSHFS

  • Check that i am not a privileged user
$ whoami
goncalo
  • Check my partitions
$ df
Filesystem                   1K-blocks      Used Available Use% Mounted on
/dev/mapper/fedora_ozzy-root 100660656   6691476  88832796   8% /
devtmpfs                       4045160         0   4045160   0% /dev
tmpfs                          4052620       204   4052416   1% /dev/shm
tmpfs                          4052620      9196   4043424   1% /run
tmpfs                          4052620         0   4052620   0% /sys/fs/cgroup
tmpfs                          4052620        12   4052608   1% /tmp
/dev/sda6                       991512    114448    809480  13% /boot
/dev/sda2                       262144    114444    147700  44% /boot/efi
/dev/mapper/fedora_ozzy-home 288210280 195558820  77988192  72% /home
  • Create a directory, and invoke sshfs to mount on that directory. User is authenticated via ssh password:
$ mkdir ~goncalo/SSHFS

$ sshfs goncalo@sydui4.syd.coepp.org.au:/home/goncalo ~goncalo/SSHFS
goncalo@sydui4.syd.coepp.org.au's password:

$ df
Filesystem                                      1K-blocks        Used  Available Use% Mounted on
/dev/mapper/fedora_ozzy-root                    100660656     6691476   88832796   8% /
devtmpfs                                          4045160           0    4045160   0% /dev
tmpfs                                             4052620         204    4052416   1% /dev/shm
tmpfs                                             4052620        9196    4043424   1% /run
tmpfs                                             4052620           0    4052620   0% /sys/fs/cgroup
tmpfs                                             4052620          12    4052608   1% /tmp
/dev/sda6                                          991512      114448     809480  13% /boot
/dev/sda2                                          262144      114444     147700  44% /boot/efi
/dev/mapper/fedora_ozzy-home                    288210280   195558848   77988164  72% /home
goncalo@sydui4.syd.coepp.org.au:/home/goncalo 19527641056 13986740864 5540900192  72% /home/goncalo/SSHFS <---
  • Perform operations on that remote dir
$ cd /home/goncalo/SSHFS

$ ll
total 65540
drwxr-xr-x 1 1051 1011       52 Dec  8 01:35 BELLE
-rw-r--r-- 1 1051 1011 67108864 Dec 11 00:56 testfile

$ echo "HELLO SSHFS" > greeting.txt

$ ll
total 65544
drwxr-xr-x 1 1051  1011       52 Dec  8 01:35 BELLE
-rw-r--r-- 1 1051 10000       12 Dec 18 05:47 greeting.txt
-rw-r--r-- 1 1051  1011 67108864 Dec 11 00:56 testfile
  • Once finished, just unmount the dir
$ cd; fusermount -u ~goncalo/SSHFS

$ df
Filesystem                   1K-blocks      Used Available Use% Mounted on
/dev/mapper/fedora_ozzy-root 100660656   6691480  88832792   8% /
devtmpfs                       4045160         0   4045160   0% /dev
tmpfs                          4052620       204   4052416   1% /dev/shm
tmpfs                          4052620      9196   4043424   1% /run
tmpfs                          4052620         0   4052620   0% /sys/fs/cgroup
tmpfs                          4052620        16   4052604   1% /tmp
/dev/sda6                       991512    114448    809480  13% /boot
/dev/sda2                       262144    114444    147700  44% /boot/efi
/dev/mapper/fedora_ozzy-home 288210280 195558856  77988156  72% /home

Running X application

  • Running X applications and forwarding remote X windows is enabled through XPRA

Long term data storage and archiving

tier3/syd.txt · Last modified: 2015/08/13 12:34 by goncalo
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki