How to use the Remeis cluster - Revision history

Weber at 14:17, 14 April 2025

2025-04-14T14:17:41Z

Vashukla: /* The Data-Storage System */

2023-06-23T20:46:56Z

The Data-Storage System

Zainab: /* How To use the Remeis Cluster */

2021-09-16T12:59:57Z

How To use the Remeis Cluster

Dauser at 08:48, 20 February 2019

2019-02-20T08:48:33Z

Obst at 08:22, 20 February 2019

2019-02-20T08:22:27Z

Dauser: /* How To use the Remeis Cluster */

2019-02-19T12:06:00Z

How To use the Remeis Cluster

Thalhammer: Fixed link

2018-06-21T11:01:55Z

Fixed link

Thalhammer: Fixed link

2018-06-21T11:00:11Z

Fixed link

Thalhammer: Fixed link

2018-06-21T10:42:04Z

Fixed link

Lorenz: /* ssh-keys */

2018-04-12T10:01:58Z

ssh-keys

@@ Line 5: / Line 5: @@
 === Performing Computations  ===
-* The servers ''crux'', ''grus'', and ''phoenix'' should not be used for computations at all.
+* You should never log in on servers directly and especially not run any computations on servers without using slurm.
 * Computations and extractions on the Cluster should be submitted via '''slurm'''. Information on how to use it can be found [[Slurm|here]].
 * Also interactive sessions can be started via '''slurm'''.
@@ Line 13: / Line 13: @@
 * '''/userdata:''' Extracted data like images or spectra or other space consuming stuff or stuff which can re recreated/downloaded again should be put there. /userdata is a RAID6 system, i.e. two disks can fail simultaneously w/o data loss --> your data is safe on /userdata! However, there is NO backup of userdata, as /userdata is too big to be backed up simply. And there is no quota on /userdata.
-* '''/home:''' This is the place for self-developed software, scripts, documents, TeX files, diploma/phd/... theses, papers, spreadsheets, etc. /home is also a RAID system, i.e. more than one disk may fail w/o data loss. Furthermore, /home is backed up every night to TWO different disks (in two machines) AND once a week on an external disk that is stored outside the observatory  for the worst case (e.g., fire). It therefore does not make sense to create your own private backups on local scratch disks etc. /home, however, is quite small (1 TB for all the institute); we therefore have a quota of about 10GB per user. It is no problem to get more space, if you can't move stuff around anymore, just drop an Email to the admins. On the other hand, you don't have to fill your quota completely! To check your quota you can use the ''quota'' command on crux, like
+* '''/home:''' This is the place for self-developed software, scripts, documents, TeX files, diploma/phd/... theses, papers, spreadsheets, etc. /home is also a RAID system, i.e. more than one disk may fail w/o data loss. Furthermore, /home is backed up every night to TWO different disks (in two machines) AND once a week on an external disk that is stored outside the observatory  for the worst case (e.g., fire). It therefore does not make sense to create your own private backups on local scratch disks etc. /home, however, is quite small (5 TB for all the institute); we therefore have a quota of about 10GB per user. It is no problem to get more space, if you can't move stuff around anymore, just drop an Email to the admins. On the other hand, you don't have to fill your quota completely! To check your quota you can use the ''quota'' command on libra, like
-   ssh crux quota -s -f /home
+   ssh libra quota -s -f /exports/disk1
-* '''/scratch:''' For temporary or otherwise not very important data. You should not store anything important on /scratch!! The scratch disks are NOT backedup and the scratch disks are not fail safe, i.e. if a scratch disk is broken (which happens regularly), your data is GONE.
+* '''/scratch:''' For temporary or otherwise not very important data. You should not store anything important on /scratch!! The scratch disks are NOT backedup and the scratch disks are not fail safe, i.e. if a scratch disk is broken (which happens regularly), your data is GONE. scratch might be emptied at any time without warning.
-* '''Satellite Data:''' We already have archives for most satellite raw data (applies mostly to the X-ray people). So please do not download any raw data yourself, but talk to the person in charge of a specific mission. You can find out who is responsible by executing <code>showSatelliteContacts</code> in the shell.
+* '''Satellite Data:''' We already have archives for most satellite raw data (applies mostly to the X-ray people). So please do not download any raw data yourself, but talk to Ingo.
 ===  Logging into Remeis ===
@@ Line 29: / Line 29: @@
-===== Using the NX-Client =====
+===== Using x2go =====
-If you need a graphics interface, it might be worth to use the <code>nxclient</code> to connect to the Remeis Cluster. The corresponding <code>nxserver</code> is installed on (almost) all machines. In order to use it you have to [http://www.nomachine.com/download.php download] it and follow the Connection-Wizard to connect on a host. Always use the same machine, as only a limited number of total connections is allowed on each machine. This works for Linux, Mac OSX, and even Windows.
+For remote login with a graphical interface you can use [[X2Go Remote Desktop]]
 ===== Hosts =====
-Host names are generally the constellation names, e.g., <code>norma</code> or <code>leo</code>. All available hosts can be found in the shell variable <code>$RemeisHosts</code>. The servers (<code>crux</code>, <code>grus</code>, and <code>phoenix</code>) should not be used.
+Host names are generally the constellation names, e.g., <code>norma</code> or <code>mensa</code>.
 ==== Connecting without password and short hostnames ====
@@ Line 51: / Line 51: @@
 In order to define a short-cut for a host, you have to add the following information to the ''~/.ssh/config''
     # the short-name you want to call the  host (most useful is the general name of the machine
-    Host=pulsar
+    Host=norma
     # now define user and host such that ''<User>@<Hostname>'' is the correct adress
-    Hostname=pulsar.sternwarte.uni-erlangen.de
+    Hostname=norma.sternwarte.uni-erlangen.de
     User=YourUsername
 ===== ssh-keys =====
-If you setup the ssh-keys properly, you can login via ssh without entering a password. This isn't only nice to have, but if you use [[torque:start|Torque]], this is mandatory. A detailed description can be found [[SSH-Keys|here]].
+If you setup the ssh-keys properly, you can login via ssh without entering a password (although you should definitely protect your ssh key with a password). A detailed description can be found [[SSH-Keys|here]].
 === Setting the Default Options/Software ===
@@ Line 68: / Line 68: @@
     # activate the X-ray software (HEADAS, satellite extraction scripts, ...)
-    source $SOFTDIR/softwarescript_Xray.csh
+    source /data/software/cshrc.in

@@ Line 11: / Line 11: @@
 ===  The Data-Storage System ===
-* '''/userdata:''' Extracted data like images or spectra or other space consuming stuff or stuff which can re recreated/downloaded again should be put there. /userdata is a RAID6 system, i.e. two disks can fail simultaneously w/o data loss --> your data is safe on /userdata! However, there is NO backup of userdata, as /userdata is too big to be backuped simply. And there is no quota on /userdata.
+* '''/userdata:''' Extracted data like images or spectra or other space consuming stuff or stuff which can re recreated/downloaded again should be put there. /userdata is a RAID6 system, i.e. two disks can fail simultaneously w/o data loss --> your data is safe on /userdata! However, there is NO backup of userdata, as /userdata is too big to be backed up simply. And there is no quota on /userdata.
-* '''/home:''' This is the place for self-developped software, scripts, documents, TeX files, diploma/phd/... theses, papers, spreadsheets, etc. /home is also a RAID system, i.e. more than one disk may fail w/o data loss. Furthermore, /home is backedup every night to TWO different disks (in two machines) AND once a week on an external disk that is stored outside the observatory  for the worst case (fire etc). It therefore does not make sense to create your own private backups on local scratch disks etc. /home, however, is quite small (1 TB for all the institute); we therefore have a quota of about 10GB per user. It is no problem to get more space, if you can't move stuff around anymore, just drop an Email to the admins. On the other hand, you don't have to fill your quota completely! To check your quota you can use the ''quota'' command on crux, like
+* '''/home:''' This is the place for self-developed software, scripts, documents, TeX files, diploma/phd/... theses, papers, spreadsheets, etc. /home is also a RAID system, i.e. more than one disk may fail w/o data loss. Furthermore, /home is backed up every night to TWO different disks (in two machines) AND once a week on an external disk that is stored outside the observatory  for the worst case (e.g., fire). It therefore does not make sense to create your own private backups on local scratch disks etc. /home, however, is quite small (1 TB for all the institute); we therefore have a quota of about 10GB per user. It is no problem to get more space, if you can't move stuff around anymore, just drop an Email to the admins. On the other hand, you don't have to fill your quota completely! To check your quota you can use the ''quota'' command on crux, like
    ssh crux quota -s -f /home
@@ Line 19: / Line 19: @@
 * '''Satellite Data:''' We already have archives for most satellite raw data (applies mostly to the X-ray people). So please do not download any raw data yourself, but talk to the person in charge of a specific mission. You can find out who is responsible by executing <code>showSatelliteContacts</code> in the shell.
 ===  Logging into Remeis ===

@@ Line 1: / Line 1: @@
 == How To use the Remeis Cluster ==
-The Cluster of the Remeis obsveratory is designed to make all ressources available for easy usage. Nevertheless, there are a few important things one has to know in order to use the cluster properly. In the following the most important information will be listed. Other questions or problems with the system or software sould be directed to [mailto:admin@sternwarte.uni-erlangen.de admin@sternwarte.uni-erlangen.de].
+The Cluster of the Remeis observatory is designed to make all resources available for easy usage. Nevertheless, there are a few important things one has to know in order to use the cluster properly. In the following the most important information will be listed. Other questions or problems with the system or software sould be directed to [mailto:admin@sternwarte.uni-erlangen.de admin@sternwarte.uni-erlangen.de].
 === Performing Computations  ===

@@ Line 6: / Line 6: @@
 * The servers ''crux'', ''grus'', and ''phoenix'' should not be used for computations at all.
-* Computations and extractions on the Cluster should be submitted via '''torque'''. Information on how to use it can be found [[Torque|here]].
+* Computations and extractions on the Cluster should be submitted via '''slurm'''. Information on how to use it can be found [[Slurm|here]].
-* Also [[Torque#Interactive_Jobs|interactive sessions]] can be started via '''torque'''.
+* Also interactive sessions can be started via '''slurm'''.
 ===  The Data-Storage System ===

← Older revision		Revision as of 08:22, 20 February 2019
Line 76:		Line 76:
	More information on the TC shell and how to configure it can be found [[TcShell\|here]].		More information on the TC shell and how to configure it can be found [[TcShell\|here]].

−	[[Category:Current Members]]	+	[[Category:Current Members]][[Category:Software]]

← Older revision		Revision as of 11:01, 21 June 2018
Line 74:		Line 74:


−	More information on the TC shell and how to configure it can be found [[~~csh:start~~\|here]].	+	More information on the TC shell and how to configure it can be found [[TcShell\|here]].

	[[Category:Current Members]]		[[Category:Current Members]]