Welcome to the Visions

This Wiki should help you getting started with the workflow of our group.

It is somewhat sorted by relevance. Your first step should be getting access to the communication channels. After that acquire accounts as most links in this document need internal accounts.

Communication

Chat

A lot of Vision communication, be it professional or personal (especially during the pandemic) happen in a slak-like chat https://chat.bioai.eu/visions.

There are several topic specific channels, also many are invite only (NDA's and stuff). Your supervisor should add you to relvant channels

Mumble

There is also a [Mumble] voice chat server. URL: jitsi.bioai.eu Port: 64738

Accounts

Typically, you will need the following accounts:

KIP-Account (see KIP Institute Login)
Flagship/ex-BrainScaleS-Account (which provides Access to the gitviz-Repository)
EBRAINS Account

KIP

The KIP login process is used for both, physical (i.e. keys) and virtual (i.e. user login stuff) access to KIP facilities. All F9 group services use some kind of KIP account/authentication. The login form can be found here: https://www.kip.uni-heidelberg.de/service/verwaltung/form_forms (needs KIP login).

Rules (as a hint for your supervisor) regarding the "LOGIN Computing" part:

students should not use their URZ account name but rather a readable abbreviation of their real name, e.g. mmustermann
state that the student should be added to f9 mailing list (IT) ... (it is an *alias* mailing list for kip_vision)
same for kip visions wiki (IT)
chip design permission (ask MD) if needed
everybody starts out as a member of F9_guests (field "3"); somewhat later (e.g. 6 months) the person may be also added to F9 (switch primary group, keep F9_guests as a secondary); this is triggered by a supervisor.
- (this affects LDAP/POSIX group membership rights)

You will have to collect all signatures and give your 50€ key deposit to Mrs. Hafranke or Mrs. Potthoff. Suggested waypoints:

1st floor: Signature of JS and FK
Ground floor: deposit for key in room 00.304
2nd floor: Signature of H. Jacobsen or alternatively E. Schmetzer
3rd floor: First go to EDV (they will cut off the lower part of the form). The procedure for getting a key has changed. FK triggers and you can pick the key at Mr. Jacobsens Office.

Flagship/ex-BrainScaleS

Flagship/ex-BrainScaleS accounts are managed by Björn Kindler. You can contact him directly bjoern.kindler@kip.uni-heidelberg.de. This account is also needed for Redmine/GitViz access.

EBRAINS

When you got your KIP mail address, please also:

register here for an HBP / EBRAINS account: https://iam.ebrains.eu/register
and click the "request to join" button for the T6.1 unit (our task in HBP SGA3): https://wiki.ebrains.eu/bin/view/Identity/#/units/all:projects:hbp:consortium:SGA3:WP6:T6_01 When granted this gives you access to e.g. the WP6 internal Collab.

Aftermath

When you have all necessary accounts, do the following steps

inform cluster commissioner, i.e. CM, ECM, they will create wang home and slurm user
Sign form to allow open sourcing of contributions.
If you do chip or FPGA development, you need ASIC permissions. Ask your supervisor to write an email to MD (+ get an introduction by your supervisor).
Follow this instruction to create a ssh key and register it in OpenProject and Gerrit

Meetings

All regular meetings can be found here

Workplace

Who should I call for help? (FAQ)

The power is not working/toilet is overflowing/... → 54-5111 Störungsstelle („Alles was zur Grundversorgung gehört: Strom/Gas/Wasser“; „der Hausmeister würde ohnehin nur dort anrufen“)
Office doors don't open (e.g. empty lock battery), and other "Haustechnik" stuff ⇒ haustechnik@kip.uni-heidelberg.de

Computers

We use a Debian Jessie-based default installation. The configuration is automatically managed. In case of package requests, please ask StB. Bugs and requests should be posted here.

The spack package manager is used to provide modules, if you run into issues try to find your question here.

If the GUI session is blocked by a previous user, go to a non GUI session (Ctrl+Alt+F1/2/3) log in and start GUI with startx

The list of visionary desktop computers can be found here Visions_Privat:ComputerUsage. Available unused machines can be found in this list.

Space

There are some places in the "Werkstatt" building (room 501) and in the container building. In case of a transient shortage of spaces, internshippers and bachelors are expected to "fill up" (i.e. they do not have a static assignment to a specific place) all available places.

Network

If you want to use your own laptop inside the KIP network, you first need to register your laptop's MAC address. AG, JS are allowed to edit the list.

From inside the KIP network, outgoing traffic has to go through the EDV:KIPProxy. You can create a proxy auto-config file, e.g. /etc/proxy.pac and configure your browser and system to use it. This way you do not need to manually switch proxy settings when switching between KIP and other networks.

function FindProxyForURL(url, host)
{
  if (isInNet(myIpAddress(), "129.206.180.0", "255.255.255.0")) {
    if (isInNet(host, "129.206.0.0", "255.255.0.0")) {
      return "DIRECT";
    }
    return "PROXY proxy.kip.uni-heidelberg.de:8080" ;
  } else {
    return "DIRECT";
  }
}

Computing

Know your Tools

Classes teach you all about advanced topics within CS, from operating systems to machine learning, but there’s one critical subject that’s rarely covered, and is instead left to students to figure out on their own: proficiency with their tools. We’ll teach you how to master the command-line, use a powerful text editor, use fancy features of version control systems, and much more!

https://missing.csail.mit.edu/

Cluster Access

The F9 cluster is part of the BrainScaleS and HBP hardware systems. In times of idle nodes (i.e. the associated neuromorphic hardware parts are idle too), conventional software simulations can be run on the system. Please note, the cluster's main objective is controlling neuromorphic hardware and not number crunching. Having a KIP-Account gives you a home folder (distributed filesystem, AFS) on all machines running the default installation. However, this is not sufficient for cluster usage. You need a "wang" home and cluster access permissions. Both are managed by ECM and CM. In case of a missing cluster_home you will see an error message when you lock into a compute server.

The frontend/login node is hel and you can access it via ssh -X hel

If you are working from outside the institute, you can access the frontend via ssh -p <port> <kipuser>@brainscales-r.kip.uni-heidelberg.de The following ports give access to:

11022: hel
12022: comicsans (only a selection of users has access, "currently heavy WIP")

Mosh ports are 50000 + X, e.g. 61022 for hel (mosh brainscales-r.kip.uni-heidelberg.de --ssh="ssh -p 11022" -p 61022). If the mosh port (e.g. 61022) is already being used by someone else, try the next one (or something in the range of [50000 + X, 50000 + 1000 + X)). mosh does not work from within eduroam.

Server Usage

The machine mentioned above is not the compute node itselve, but is only the frontend to access the compute cluster. Large jobs (i.e. CPU/IO hogs or long-running things) on the frontend node will be killed by the administrators. So for heavy work (read everything after the bug/syntax fixing) please dispatch execution to the cluster:

A quick introduction to the job scheduling system slurm used on our cluster can be found here

X on cluster nodes

Although not recommended, some old tools require X support. Follow the instructions at [1] if you encounter such an old tool.

Data Management

There are differen NFS data storage mount points:

KIP/F9 Data Storage
Mount Point	Storage Backend	Redundancy	Backup Strategy	Usable Size	User Quota	Typical Application
`/afs/kip/user/USERNAME`	HDD	RAID	yes (1 version)		10G	Distributed home directory
`/scratch`	SSD	—	—			Scratch/temp data; might be deleted at any time
`/wang`	HDD	RAID6 (2R)	—	13T	0.3T	General purpose
`/ley`	HDD	RAID6 (2R)	yes (ADSM; 1 version)	7T	0.1T	Important stuff (not too large!)
`/loh`	4x Archive HDD	RAID5 (1R)	—	16T	1T	Archives of machines, homes, etc.
`/fasthome`	NVM	—	—			Software projects (lots of compiling)

FPGA & ASIC

There are several login nodes for ASIC work, e.g., vmimas, vtitan, vrhea. Servers, software and libraries are managed by MD.

Software Development

Most (all?) software developers work remotely on server machines. Tools like screen or tmux can keep your session open between reconnects.

Git

As a general rule, everything should be tracked in a version control system, in our case, that is git. Period. If you hear git for the first time I highly recommend spending an hour going through a git tutorial of your choice. Here are some examples:

The Simple Guide
Understanding Git Conceptually Highly recommended as additional reading because it focuses on the concepts behind git
TODO add one or two more?

The F9 gitolite server is hosted on our OpenProject server: https://openproject.bioai.eu

Whether your own work should be tracked in the group gitviz or not, should be decided together with your supervisor. But mostly, you will need to checkout some repo for read access. If your own/private work is not for the trash, you should request a repository (or contribute to an existing one) on gitviz.

Accessing external repositories using the git or ssh protocol requires the user to call socksify or tsocks:

 socksify git clone git://external.server/repo.git # on wheezy default installation
 tsocks git clone git://external.server/repo.git # on 'new' jessie default installation

License

Keep your code contributions (L)GPL-clean because we might want to publish it on a public web site. If you copy code from somewhere, verify license compatibility and mention the source as a code comment!

Code Review (Gerrit)

For core software components (and other repositories involving multiple developers), we use gerrit as a code review tool. The server F9's Gerrit Server and a small tutorial can be found here [2].

Gerrit uses BrainScaleS or Flagship accounts for authentication. Use

   export MY_BRAINSCALES_OR_FLAGSHIP_ACCOUNT=myBssOrFlagshipUsername
   ldapsearch -h visions-ldap -x -W -D uid=${MY_BRAINSCALES_OR_FLAGSHIP_ACCOUNT},ou=brainscales,ou=people,dc=kip.uni-heidelberg,dc=de -b ou=brainscales,ou=people,dc=kip.uni-heidelberg,dc=de uid=${MY_BRAINSCALES_OR_FLAGSHIP_ACCOUNT}

to register your account (or to verify your credentials).

Continuous Integration (Jenkins)

We encourage continuous and automatic testing, see F9's Jenkins Server. Contact YS (or ECM) for details.

Jenkins uses BrainScaleS or Flagship accounts for authentication, see Gerrit section.

Bug Reports and Project Management (OpenProject)

Bugs should be posted immediately in the OpenProject project associated with the module that produced the error. The title should be descriptive. As a general rule, the traceback is necessary for the developer to find the actual bug, but the more relevant information are given the easier the fix. Ideally, you create a minimal example that reproduces the problem and upload the script, including the module's loaded. F9's OpenProject Server

How to: Building Software

log onto frontend (exp: hel)

  > ssh hel

Create and switch to project directory (choose path according to personal preference):

  > mkdir -p code/project_name
  > cd code/project_name

Load the waf build tool module. More information on waf

  > module load waf

Normally ssh will ask for the password of your private key every time you connect to another machine (e.g. during git pull). If you want to avoid this you can start ssh-agent, which will load/unlock your private key by asking you once and keep running in the background.

  > eval $(ssh-agent)

Add your personal ssh key agent to the agent (link -> create key?)

  > ssh-add

Choose the wanted project (exp: sthal)

  > waf setup --project sthal
  # ——— or specify multiple projects: ———
  > waf setup --project sthal --project cake

Configure and compile on the cluster. The default job gets 2 CPUs and 2GB of RAM. If your code runs in parallel or needs more memory, please specify this (e.g. 8 CPUs (-c8) or 20GB RAM (--mem=20G). We use singularity container to manage external dependencies, see Singularity

  > srun -p compile -c8 --pty singularity exec --app visionary-wafer /containers/stable/latest waf install --targets=*

Set environment variable e.g. LD_LIBRARY_PATH or PATH to the compiled software

  > module load localdir

In order to pull and install the complete software stack such that the BrainScaleS Guidebook examples are executable follow (assuming that we build on the cluster)

  > module load waf
  > waf setup --project=pyhmf --repo-db-url=https://github.com/electronicvisions/projects.git
  > singularity exec --app visionary-wafer /containers/stable/latest waf configure
  > srun -p compile -c8 --pty singularity exec --app visionary-wafer /containers/stable/latest waf install --targets=* 
  > module load localdir # load the compiled software stack

After installation changesets can be reviewed and uploaded to the Gerrit server.

How to: Building Papers

We also perform (LaTeX) builds of the latest publications in our containerized software environment. E.g.

   > singularity exec --overlay /my/local/path/to/containers/overlays/2020-05-28_buster_texlive.img /my/local/path/to/containers/stable/latest make

Please look for the latest overlay to use it. (Back story: We separated our LaTeX installation from the stable containers and use an filesystem overlay to add the LaTeX packages.)

Other information:

we try to generate figures from sources during the paper build (e.g. PDF from SVG or such stuff)
however, we do store published PDFs (e.g. the pdf that was sent into review, …)

Style

We have style guides for C++ and Python. [[3]]

To format your code properly, run git-clang-format and inspect/add the resulting changes before committing. To apply clang format on the current commit

  > singularity exec --app visionary-defaults /containers/stable/latest git-clang-format HEAD^

There is also a convenient vim plugin [[4]]

B.Sc. / M.Sc.

Thesis

When thinking about how to structure your thesis, consider these internal recommendations.

B.Sc. colloquium / Theses hand-in dates

Find a suitable date for your colloquium and other things (KM & JS & you). A lot of people have full schedules, so do it well in advance!
Set all the dates, inform the examination office (Prüfungssek.) and make sure that Felicitas Kleveta is up2date, too.

Thesis Publication

It is strongly suggested to upload your Thesis to the KIP publications archive https://www.kip.uni-heidelberg.de/Veroeffentlichungen/new.php
A printed version of your thesis is required for the "Lehrstuhl" archive; ask Felicitas Kleveta for details.

Results Guidelines

https://brainscales-r.kip.uni-heidelberg.de/projects/symap2ic/wiki/visionarythesisguidelines

Funding acknowledgement

for work, where the researcher itself was funded from HBP: "This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement Nos. 720270, 785907 and 945539 (Human Brain Project, HBP)"
for work, where the BrainScaleS systems were used (e.g. Bachelor and Master Thesis), please add an acknowledgement for the systems: "The work carried out in this ... Thesis used systems, which received funding from the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreements Nos. 720270, 785907 and 945539 (Human Brain Project, HBP)
for the full picture: "HBP official" funding acknowledgement info

(Explanation: 720270 is HBP SGA1 (1.4.2016-31.3.2018), 785907 is HBP SGA2 (1.4.2018-31.3.2020) and 945539 is HBP SGA3 (1.4.2020 - 31.3.2023). For most-current hardware use it is sufficient to acknowledge SGA2 and SGA3.)

Administrative Stuff

Travel

before travel: fill out travel request form (Dienstreiseantrag) and hand over to FK
after travel: fill out reimbursement form (Dienstreiseabrechnungsformular) (provide invoices etc.) and hand over to FK
when getting back the result: check if everything is correct, hand over to OA

The forms can be found here: https://www.kip.uni-heidelberg.de/service/verwaltung/form_forms (needs KIP login).

Vacation

Ask your supervisor, you might be needed urgently during the desired time.
Fill out your personal "Urlaubskarte" and get a signature from JS. If you have no contract, skip this step. Otherwise it is obligatory before you leave!
Hand it over to FK. It will arrive in your post box a few days later with another signature. Now is a good time to note these days as vacation in your timesheet.
Create an issue with start and due date of your absence for our vacation calendar so your colleagues know that you are on vacation.

Internships

Schein

Windows Software

If you need Windows software, e.g. to fill out forms (Word, Excel), you can log in remotely to a Windows machine:

rdesktop kipwin

Username: KIP\$your_accountname

(From the outside you can perform a port-forwarding via ssh -L 3389:kipwin.kip.uni-heidelberg.de:3389 hel (if hel is in your ssh config) and run rdesktop localhost.)

Orders

above 500€ => multiple offers (print out from some comparison website) needed
up to 2.5k€ => get three different offers (ask three shops for offers)
- e.g. write to first three cheapest shops listed on some comparison websites
- KIP "Kundennummer"s:
  - Mindfactory: 10970071

Using The Hardware

The HBP SP9 Guidebook provides introductions to both, the Spikey system and the BrainScaleS system.

Core Hardware Components

For the BrainScaleS system, the NMPM hardware specification provides detailed information; see Jenkins doc job "HBP Spec". TODO: Write something about hardware stuff.

Software Environment

We apply container technology to improve reproducibility. Continuously built container images are deployed to /containers (e.g. /containers/stable/latest).

Core Software Components

This section gives a very brief description of the main software packages developed for the NMPM-1 and Spikey systems. Short introduction on how to get and build repos https://openproject.bioai.eu/projects/waf/wiki/Wiki

Common

PyNN

A python API to specify neural networks, independent of the actual simulator used. Typical use case is to use it as frontend for simulations in nest or neuron. Documentation is available on its webpage (sort of at least) [5], in order to get started it is best to ask someone for a simple working script and trying to reproduce it with minor changes (ideally your supervisor has some propositions for you).

PyHMF

The BrainScaleS-hardware-specific PyNN implementation/backend. Maintainers: ECM

halco

Hardware coordinate system for all modern (BSS-1 and later) systems. Maintainers: ECM, CM

HostARQ

The communication protocol stack for communication between the BrainScaleS-1/2 hardware (FCP FPGAs) and host computers. Maintainers: ECM, CM, JI

Spikey

PyNN.hardware/PyHAL

The Spikey-hardware-specific PyNN implementation/backend. Maintainers: TP (backup: ECM)

SpikeyHAL

Spikey hardware access layer. Maintainers: AG or ECM

BrainScaleS-1

Cake

Calibration framework for the BrainScaleS hardware. Maintainers: SS

Euter/Ester

The C++-layer providing a representation of neuronal network descriptions (generated by PyHMF) -- used for BrainScaleS hardware. Maintainers: ECM

Marocco

The translation layer which converts abstract neuronal networks into a hardware bit stream (i.e. a valid hardware configuration) -- used for BrainScaleS hardware. Maintainers: ECM, SS

StHALbe, hicann-system

StHAL, HALbe and hicann-system are the hardware access layers -- used for BrainScaleS hardware. Maintainers: ECM, CM, SS

ESS

The Executable System Specification is a BrainScaleS hardware simulator. Originally developed as a chip verification software by AG, it evolved into a neuronal network simulator (by BV). Maintainer: BV, OJB, SS

BrainScaleS-2

hxcomm

Communication interface between HICANN-X and Host computer (via HostARQ) Maintainers: PSP, ECM

fisch

FPGA Instruction Set Compiler (adapter between a "pretty" FPGA interface and the real world) Maintainers: PSP, ECM

haldls

Hardware Abstraction Layer for BrainScaleS-2 systems. Maintainers: PSP, ECM, CM

Modeling Software Packages

SBS

Allows for simple creation and sampling of and with Boltzmann machines of PyNN neurons. See [Tutorial[6]] (requires access to gitweb) Maintainer: OJB

Documentation

Central starting points for documentation besides this page are

SP9 specification for BrainScaleS-1
symap2ic Wiki
Publications, especially BSc, MSc, PhD theses

Search

Content is distributed in many different places. Some inspiration is given below:

Search in Project Management: https://openproject.bioai.eu to find content in issues, project wikis, etc.
Search in Code Review: https://gerrit.bioai.eu to find content in changesets
Search in Repositories: https://gerrit.bioai.eu/gitweb?p=PROJECTNAME.git to find repository contents (also git grep or pickaxe); there is no multi-project git search
Search in KIP Wiki: https://wiki.kip.uni-heidelberg.de (in terms of visionary content basically only the Noobies Tutorial is up2date)
Search in Chat: https://chat.bioai.eu to find chat messages
Search in CI: https://jenkins.bioai.eu to find CI jobs
Search in Monitoring: https://grafana.bioai.eu/grafana/ to find Dashboards, https://brainscales-r.kip.uni-heidelberg.de:10443/ to find individual time-series metrics.

Abbreviated Names

 AB:  Andreas Baumbach
 AE:  Arne Emmel
 AFK: Ákos Kungl
 AG:  Andreas Grübl
 AL:  Aron Leibfried
 BC:  Benjamin Cramer
 BK:  Björn Kindler
 CM:  Christian Mauch
 CP:  Christian Pehle
 CVR: Carl Von Randow
 DH:  Dan Husmann
 ECM: Eric Müller
 FE:  Falk Ebert
 FK:  Felicitas Kleveta
 FP:  Felix Passenberg
 HS:  Hartmut Schmidt
 JG:  Julian Goeltz
 JI:  Joscha Ilmberger
 JM:  José Montes
 JJK: Jakob Kaiser
 JS:  Johanwelche user doc wird denn atm nicht von jemandem bearbeitet?nes Schemmel
 JW:  Johannes Weis
 MAP: Mihai Petrovici
 MD:  Markus Dorn
 MG:  Maurice Güttler
 OMA: Oscar Martín-Almendral
 OJB: Oliver Breitwieser
 PD:  Philipp Dauer
 PSP: Philipp Spilger
 SB:  Sebastian Billaudelle
 SS:  Sebastian Schmitt
 TT:  Tobias Thommes
 VK:  Vitali Karasenko
 YS:  Yannik Stradmann

Visions:NoobiesTutorial

Welcome to the Visions

Communication

Chat

Mumble

Accounts

KIP

Flagship/ex-BrainScaleS

EBRAINS

Aftermath

Meetings

Workplace

Who should I call for help? (FAQ)

Computers

Space

Network

Computing

Know your Tools

Cluster Access

Server Usage

X on cluster nodes

Data Management

FPGA & ASIC

Software Development

Git

License

Code Review (Gerrit)

Continuous Integration (Jenkins)

Bug Reports and Project Management (OpenProject)

How to: Building Software

How to: Building Papers

Style

B.Sc. / M.Sc.

Thesis

B.Sc. colloquium / Theses hand-in dates

Thesis Publication

Results Guidelines

Funding acknowledgement

Administrative Stuff

Travel

Vacation

Internships

Windows Software

Orders

Using The Hardware

Core Hardware Components

Software Environment

Core Software Components

Common

PyNN

PyHMF

halco

HostARQ

Spikey

PyNN.hardware/PyHAL

SpikeyHAL

BrainScaleS-1

Cake

Euter/Ester

Marocco

StHALbe, hicann-system

ESS

BrainScaleS-2

hxcomm

fisch

haldls

Modeling Software Packages

SBS

Documentation

Search

Abbreviated Names

Navigation menu

Search