How To

Frequent questions

 

 

Request for an account

You can send your request for an account on this page.
Your request will be examined and validated by GenOuest. Accounts are presently free for the academic community but things will change in the next months since the operation and maintenance of our infrastructure has a non negligeable cost that we can not support easily.

Your account will allow you to access to the different services offered by GenOuest : computing, Galaxy portal, collaborative tools, project management, instant communication, data sharing. For more information on collaborative tools see the CeSGO page.

Please note that the my.genouest service can also help you in the password retrieval process if you forget it.

Lost password

If you lose your password, just point your browser to my.genouest.org, enter your login and click on “RESET”. You will receive the instructions by mail.

Connection to the cluster

Once you have an account on the platform, you can connect via SSH to the genossh.genouest.org server.

genossh.genouest.org is a front-end of the cluster from which you can submit jobs with the Sun Grid Engine tool (SGE) job manager. You first need to connect to the front-end server via SSH from your computer.

You can connect to genossh.genouest.org from anywhere, but only with a properly configured SSH Key.

Connecting to genossh.genouest.org from a windows computer

On Windows, Putty can be used to load SSH keys and connect via SSH to the cluster. Have a look at this video tutorial explaining the whole procedure (creating a SSH key and then connecting to the cluster):

Connecting to genossh.genouest.org from a linux computer

You need first to generate an SSH key. To do so, launch this command on your computer:

ssh-keygen -t rsa -b 4096

The command will ask for a password: it will protect your SSH key, and you will need it everytime you will use it to connect to the cluster (depending on your configuration, a program named ssh-agent can remember this password after you entered it the first time you connect).

The ssh-keygen program will create two files in your home directory:

$HOME/.ssh/id_rsa
$HOME/.ssh/id_rsa.pub

id_rsa is your private key: keep this file secret.

id_rsa.pub is your public key. You need to open this file and copy-paste its content to http://my.genouest.org (“Public SSH key” form on the right side, once your are logged in).

Add your key in your ssh agent:

 ssh-add $HOME/.ssh/id_rsa

You should then be able to connect to the cluster with this command:

ssh <your-login>@genossh.genouest.org

[up]

Data storage

Once logged, you have access to three volumes, available on all computing nodes.

  • your home directory (/home/genouest/<your-group>/<your-login>). We have a total of around 100Tb of storage capacity shared between all the home directories, and each user have a quota of 100Gb. You can check your disk usage with the command “quota -s”. A snapshot mechanism is available on this volumes, if you erased a file by mistake, your can rescue it by looking into the ~/.snapshots directory.
  • a project directory (/groups/<your-group>) that you share with your team. We have a total of around 200Tb of storage capacity shared between all these group directories. Each project have a specific quota, and a single person in your team is responsible to grant you access to this volume.  You can check your disk usage with the command “df -h /groups/<your-group>”.
  • a high performance storage space (/omaha-beach/<your-login>).We have a total of around 80Tb for /omaha-beach, and each user have a quota of 120Gb. You can check your disk usage with the command “pan_quota /omaha-beach/”.

Quotas are intentionally restrictive, if you need them to be increased, please contact support@genouest.org.

As a general rule, user should not write during the jobs in the /home or /groups directory, nor do heavy read operations on these volume. They are used to keep your data safe. During jobs, one should use the /omaha-beach directory. This directory is hosted by a high performance system and designed for temporary data. It supports heavy read and write operations.

Please note that none of your data is backed up. If you would like us to backup your data for specific reasons, you can contact us and we will help you to find a solution.

We strongly advise you to anticipate your storage needs: if you plan to generate a big amount of data, please contact us before to check that we have the possibility to host this data. It is preferable to anticipate this when applying for grants that imply data generation and analysis.

Before generating data on the cluster, please do not forget to check the remaining available space. To do so, you may use the quota commands above, or use the df command for global disk usage:

df -h

[up]

Snapshots

If you erase some files by mistake, you can recover the files by looking in the .snapshots directory. The snapshots are taken each hour and are kept for 5 weeks.
To access the snapshot files of your account just go to the .snapshots directory.

cd .snapshots

There, you will see all the directories in which the files are stored. The directories are easily recognizable by their name: hourly, daily, weekly.

Important

The snapshot mechanism is only available on /home and /groups directory.

Please note that snapshots are *not* backups. They provide protection against user error, but not against mechanical failure.

Please consider an external backup solution if your data is valuable.

[up]

Biological databanks

Some databanks are available in the /db directory. They are automatically updated and indexed on a regular basis using BioMAJ. You can consult the list of available banks on our BioMAJ instance.

Software

Pre-installed software are available in /softs/local (see software manager for a list of installed software). To use a software, you have to load its environment. For example to load python 2.7 you can launch this command (the dot at the beginning is important):

.  /softs/local/env/envpython-2.7.sh

This will automatically configure the PATH, libraries etc… in your shell environment. Any subsequent python command you will launch will use this 2.7 version.

To get a list of all environments available, just list the content of /softs/local/env/env*.

Note: DO NOT USE the python/perl/…. of the node directly, always load a specific version from /softs/local.

[up]

Conda

Conda is also available to install software on the cluster.

Conda environments behave much the same as python virtualenv, but use conda packages instead of python modules. A list of available conda packages is here : https://anaconda.org/anaconda/repo.

Conda allows you to install the software you need in your own storage volumes (/home, /groups or /omaha-beach). The software needs to be available as Conda packages.

By default, the channels defaults, bioconda, conda-forge and r are available on the cluster. The Bioconda channel in particular is tailored for bioinformatics tools. You may add channel you need. Please keep in mind that private channels might present security risks (software will not be vetted). If possible, please keep to the standard channels.

https://conda.io/docs/user-guide/tasks/manage-channels.html

To use Conda, first source it the usual way (on a node):

. /local/env/envconda.sh

With Conda, you can create as many environments as you want, each one containing a list of packages you need. You need to activate an environment to have access to software installed in it. You can activate only one environment at a time.

To create a new environment containing biopython, deeptools (v2.3.4), bowtie and blast, run:

conda create -p ~/my_env biopython deeptools=2.3.4 bowtie blast

To activate it:

source activate ~/my_env

To deactivate it:

source deactivate

Feel free to test this new way to install software, and to give us feedback whether you are happy or not of it.

[up]

Virtualenv

Several versions of Python are available on the cluster. Each one comes with a specific set of modules preinstalled.
If you need to install a module, or to have a different module version, you can use Virtualenv.
Virtualenvs are a way to create a custom Python environment, completely isolated from the Python installation on the cluster. You can install all the Python modules you want in this isolated environment.

To use it, first create a new virtualenv like this:

 . /local/env/envpython-3.6.3.sh
virtualenv ~/my_new_env

This will create the directory ~/my_new_env. This directory will contain a minimal copy of Python 3.6.3 (the one you sourced just before), without any module installed in it, and completely isolated from the global 3.6.3 python version installed by GenOuest. You can source another version of Python (e.g. 2.7) if you prefer.

To use this virtualenv, you need to activate it:

. ~/my_new_env/bin/activate

Once activate, your prompt will show that you activated a virtualenv:

(my_new_env)[login@cl1n025 ~]$

You can then install all the Python modules you need in this virtualenv:

pip install biopython
pip install pyaml...

Now when you run python, you will be using the virtualenv’s Python version, containing only the modules you installed in it.

Once you have finished working with the virtualenv, you can stop using it and switch back to the normal environment like this:

deactivate

You can create as many virtualenv as you want, each one being a directory that you can safely remove when you don’t need it anymore.

[up]

Launching jobs on the cluster

It is forbidden to execute computations directly on the frontals (genocluster2.irisa.fr or genossh.genouest.org). You MUST first connect to a node (using qrsh) or submit a job to a node (using qsub).

When you submit a job, it is dispatched on one of the computing nodes of the cluster.

Those nodes have different characteristics (cpu, ram).We have servers from 32G up to 512G RAM on the nodes, with at 8 to 40 cores each. Launch the following command to display the list of available nodes and their characteristics and load:

qhost

You can submit a job with the “qsub” command, and you can monitor your jobs with the “qstat” command.

You can (must) select the appropriate number of resources needed for your program.
If your job needs more than 1 CPU, you can request for multiple CPU slots:

 qsub -pe make 8 job_script.sh

In this example, we request 8 free CPU on a node to execute the bash script “job_script.sh”.

If you use more than selected RAM, your process will be killed.

If your job needs 15Gb of RAM for example, you can request a node with 15Gb of RAM available:

 qsub -l h_vmem=15G job_script

For interactive operations, it is possible to use the “qrsh” command to login onto a node.

To kill a job, simply execute:

qdel XXX  (XXX is the job identifier)

A memento on usages with the cluster is available.
You can find a quick tutorial on SGE on this web site.

[up]

X FORWARDING

If you want to run a software that requires access to an X11 server, you can enable X forwarding by following these steps:

First, connect to the cluster with the -XC options (X is to enable X forwarding, C is to enable compression):

ssh -XC <your-login>@genossh.genouest.org

Then you need to connect to a compute node using the qlogin command:

qlogin

You will be connected to a compute node with everything setup to use X forwarding. You might be asked to enter your account password or ssh key passphrase.

The qlogin commands takes the same options as qrsh, so you can chose a specific node or ask for specific memory or cpu the same way as qrsh.

[up]

GO-Docker

GO-Docker is a SGE-like batch scheduling system. It provides a command-line and a web interface to submit shell scripts in Docker containers. The web interface provides an easy way to submit jobs and get an overview of their status.

A REST interface also eases its integration to other tools for automation.

Basically, one select the CPU/memory requirements, and the container image needed for the computation. Some default images are provided, the genouest one is a clone of the current computation nodes based on CentOS.

It also provides interactive sessions to the tasks, in fact a SSH access the container.

Access

https://godocker.genouest.org

Tutorials

https://bitbucket.org/osallou/go-docker/wiki/Usertutorial

Jobs metrics, once job is over can be queried at:

https://prometheus.genouest.org/graph

(see more doc at https://godocker.atlassian.net/wiki/display/GOD/GODOCKER and https://bitbucket.org/osallou/go-docker/wiki/Home)

Screencasts:

[up]

Singularity

Singularity is a new technology allowing to use containers in a High-Performance Computing environment.

Just as Docker, it allows you to launch applications inside containers, completely isolated from the rest of the system. However, unlike Docker, you don’t have access to the root account inside the container. This makes it possible to use it on a standard cluster like the GenOuest one.

Singularity is installed on the newest computing nodes of the cluster. To use it, you need to source it:

. /local/env/envsingularity-2.4.5.sh

Then you can launch any singularity container, for example:

singularity run shub://GodloveD/lolcow

Singularity is compatible with Docker images, you can run one like this:

singularity shell docker://biocontainers/bowtie2

If you want to have access to some specific directories from the cluster, you can use the -B option like this:

singularity shell -B /db:/db -B /omaha-beach:/omaha-beach docker://biocontainers/bowtie2

See the official website for more information on how to use Singularity.

[up]

Genostack

The complete documentation is available on these pages

Cloud storage

Storage in the cloud is accessible at https://genostack-data.genouest.org

This storage is a cold storage facility, you may compare it to a Dropbox storage. This means that you cannot read/write files directly but you need to pull/push your files to access them. However, you can remotely access them via your browser and share them with temporary url with other users. Files can also be annoted with additional meta-data.

This service is hosted in our openstack cloud (openstack swift), and data can be accessed from your cloud virtual machines, our cluster or any external location. Storage is linked to an openstack project. By default, you have a cloud project matching your genouest user identifier, and your identifiers are your genouest account ones.

You can upload/download files from a Web user interface, a python client, or via API to manage access via your scripts/services.

Command-line example:

swift --os-auth-url https://genostack-api-keystone.genouest.org/v3 --auth-version 3 --os-project-name my_genouest_user_identifier --os-project-domain-name Users --os-username my_genouest_user_identifier --os-user-domain-name Users --os-password my_genouest_user_password list

Default quota is 100Gb but can be extended on demand.

[up]

Support

For support, questions, information, please send a mail to the GenOuest Team.

For users having a GenOuest account, we also offer an additional way to get in touch with us through instant messaging with Rocket.Chat (https://instant.cesgo.org/channel/support-genouest)

 

[up]