Visions:NoobiesTutorial: Difference between revisions

From KIP Wiki
⧼kip-jumptonavigation⧽⧼kip-jumptosearch⧽
Line 438: Line 438:
JJK: Jakob Kaiser
JJK: Jakob Kaiser
JS: Johannes Schemmel
JS: Johannes Schemmel
JVS: Jan Straub
JW: Johannes Weis
JW: Johannes Weis
MAP: Mihai Petrovici
MAP: Mihai Petrovici

Revision as of 12:19, 11 September 2023

Welcome to the Visions

This Wiki should help you getting started with the workflow of our group.

It is somewhat sorted by relevance. Your first step should be getting access to the communication channels. After that acquire accounts as most links in this document need internal accounts.

Communication

Chat

A lot of Vision communication, be it professional or personal (especially during the pandemic) happens in a slack-like chat https://chat.bioai.eu/visions.

There are several topic specific channels, some of them might be invite only (NDA's and stuff). Your supervisor should add you to relevant channels.

Mumble

There is also a [Mumble] voice chat server. URL: jitsi.bioai.eu Port: 64738

Zoom

There are two permanent Zoom rooms available. The first is used for most meetings. The second is backup if the first is already in use. Feel free to use the rooms for discussions or things like pair programming. Note that these rooms are meant for internal use only. If you want to meet with external people please ask BK for a one time zoom room.

Accounts

Typically, you will need the following accounts:

  • KIP-Account (see KIP Institute Login)
  • Flagship/ex-BrainScaleS-Account (which provides Access to the gitviz-Repository)
  • EBRAINS Account

KIP

The KIP login process is used for both, physical (i.e. keys) and virtual (i.e. user login stuff) access to KIP facilities. All F9 group services use some kind of KIP account/authentication. The login form can be found here: https://www.kip.uni-heidelberg.de/loginform/ (needs KIP login). In addition the PDF forms "Einwilligungserklärung zur Veröffentlichung personenbezogener Daten" and "Allgemeine Verhaltensregeln im KIP" (your supervisor provides those, see https://www.kip.uni-heidelberg.de/internes/formulare) need to be filled out, signed (photo/scan) and sent to FK.

Rules (as a hint for your supervisor) regarding the "LOGIN Computing" part:

  • students should not use their URZ account name but rather a readable abbreviation of their real name, e.g. mmustermann
  • state that the student should be added to f9 mailing list (IT) ... (it is an *alias* mailing list for kip_vision) (not possible on web-based loginform)
  • same for kip visions wiki (IT) (not possible on web-based loginform)
  • chip design permission if needed (⇒ ask MD)
  • everybody starts out as a member of F9_guests (field "3"); somewhat later (e.g. 6 months) the person may be also added to F9 (switch primary group, keep F9_guests as a secondary); this is triggered by a supervisor.
    • (this affects LDAP/POSIX group membership rights)
    • the supervisor has to verify that KIP IT created the account with proper settings :)
  • (for the supervisor) don't forget to add the student into the internal list of students (the link can be found in the header of A-visionary-place)

If you need an building access key you will have to collect all signatures and deposit 50€ via bank transfer (ask FK for the details) (give to Mrs. Hafranke or Mrs. Potthoff). Suggested waypoints:

  • Send the forms from above, fill out web-based login form, ask FK for bank transfer details.
  • Perform bank transfer, send transfer receipt (scan/photo) to FK and Petra Hafranke-Grauer (petra.hafranke-grauer@kip.uni-heidelberg.de); afterwards a key is programmed by J. Weight and H. Jacobsen will handout the keys.
  • 1st floor: Signature of JS and FK
  • Ground floor: deposit for key in room 00.304
  • 2nd floor: Signature of H. Jacobsen or alternatively E. Schmetzer
  • 3rd floor: First go to EDV (they will cut off the lower part of the form). The procedure for getting a key has changed. FK triggers and you can pick the key at Mr. Jacobsen's Office.

Flagship/ex-BrainScaleS

Flagship/ex-BrainScaleS accounts are managed by Björn Kindler. You can contact him directly bjoern.kindler@kip.uni-heidelberg.de. This account is also needed for Open Project/Gerrit access.

EBRAINS

When you got your KIP mail address, please also:

Aftermath (DON'T FORGET TO DO THESE STEPS!!!!!!!!1111einself)

When you have all necessary accounts, do the following steps

  • log into OpenProject
  • inform cluster commissioner, i.e. CM, ECM, they will create wang (see #Data Management) home, add slurm user and set OP group permissions
  • If you do chip or FPGA development, you need ASIC permissions. Ask your supervisor to write an email to MD (+ get an introduction by your supervisor).
  • Follow this instruction to create a ssh key and register it in OpenProject and Gerrit
  • register your account with our own LDAP server for authentication.
   export MY_BRAINSCALES_OR_FLAGSHIP_ACCOUNT=myBssOrFlagshipUsername
   ldapsearch -h visions-ldap -x -W -D uid=${MY_BRAINSCALES_OR_FLAGSHIP_ACCOUNT},ou=brainscales,ou=people,dc=kip.uni-heidelberg,dc=de -b ou=brainscales,ou=people,dc=kip.uni-heidelberg,dc=de uid=${MY_BRAINSCALES_OR_FLAGSHIP_ACCOUNT}
  • Sign form to allow open sourcing of contributions.

Documentation

Central starting points for documentation besides this page are

Search

Content is distributed in many different places. Some inspiration is given below:

Meetings

There are different meetings for various topics, some of which you should attend.

All regular meetings can be found here

Workplace

Computers

For a subset of PCs (mainly in the ENI container) there is a (currently unmanaged) Debian Jessie-based default installation, where you can login via your KIP account.

The list of visionary desktop computers can be found here Visions_Privat:ComputerUsage.

Available unused machines can be found in this list. If need talk to your supervisor about setting up a new installation for you.

If you want to use your own laptop inside the KIP network, you first need to register your laptop's MAC address. AG, JS are allowed to edit the list.

Network

From inside the KIP network, outgoing traffic has to go through the EDV:KIPProxy. You can create a proxy auto-config file, e.g. /etc/proxy.pac and configure your browser and system to use it. This way you do not need to manually switch proxy settings when switching between KIP and other networks. 

function FindProxyForURL(url, host)
{
  if (isInNet(myIpAddress(), "129.206.180.0", "255.255.255.0")) {
    if (isInNet(host, "129.206.0.0", "255.255.0.0")) {
      return "DIRECT";
    }
    return "PROXY proxy.kip.uni-heidelberg.de:8080" ;
  } else {
    return "DIRECT";
  }
}

Home Office

For information regarding home office including using inventory at home consult https://openproject.bioai.eu/projects/symap2ic/wiki/home-office.

Computing

Know your Tools

Classes teach you all about advanced topics within CS, from operating systems to machine learning, but there’s one critical subject that’s rarely covered, and is instead left to students to figure out on their own: proficiency with their tools. We’ll teach you how to master the command-line, use a powerful text editor, use fancy features of version control systems, and much more!

https://missing.csail.mit.edu/

Cluster Access

The F9 cluster is part of the BrainScaleS and HBP hardware systems. In times of idle nodes (i.e. the associated neuromorphic hardware parts are idle too), conventional software simulations can be run on the system. Please note, the cluster's main objective is controlling neuromorphic hardware and not number crunching. Having a KIP-Account gives you a home folder (distributed filesystem, AFS) on all machines running the default installation. However, this is not sufficient for cluster usage. You need a wang (see #Data Management) home and cluster access permissions. Both are managed by ECM and CM. In case of a missing cluster_home you will see an error message when you lock into a compute server.

The frontend/login node is hel and you can access it via ssh -X hel

If you are working from outside the institute, you can access the frontend via ssh -p <port> <kipuser>@brainscales-r.kip.uni-heidelberg.de The following ports give access to:

  • 11022: hel
  • 12022: comicsans (only a selection of users has access, "currently heavy WIP")

Mosh ports are 50000 + X, e.g. 61022 for hel (mosh <kipuser>@brainscales-r.kip.uni-heidelberg.de --ssh="ssh -p 11022" -p 61022). If the mosh port (e.g. 61022) is already being used by someone else, try the next one (or something in the range of [50000 + X, 50000 + 1000 + X)). mosh does not work from within eduroam.

Server Usage

The machine mentioned above is not the compute node itself, but is only the frontend to access the compute cluster. Large jobs (i.e. CPU/IO hogs or long-running things) on the frontend node will be killed by the administrators. So for heavy work (read everything after the bug/syntax fixing) please dispatch execution to the cluster:

A quick introduction to the job scheduling system slurm used on our cluster can be found here

X on cluster nodes

Although not recommended, some old tools require X support. Follow the instructions at [1] if you encounter such an old tool.

Data Management

There are different NFS data storage mount points:

KIP/F9 Data Storage
Mount Point Storage Backend Redundancy Backup Strategy Usable Size User Quota Typical Application
/afs/kip/user/USERNAME HDD RAID yes (1 version) 10G Distributed home directory
/scratch SSD — — Scratch/temp data; might be deleted at any time
/wang HDD RAID6 (2R) — 13T 0.3T General purpose
/ley HDD RAID6 (2R) yes (ADSM; 1 version) 7T 0.1T Important stuff (not too large!)
/loh 4x Archive HDD RAID5 (1R) — 16T 1T Archives of machines, homes, etc.
/fasthome NVM — — Software projects (lots of compiling)

FPGA & ASIC

There are several login nodes for ASIC work, e.g., vmimas, vtitan, vrhea. Servers, software and libraries are managed by MD.

Using The Hardware

Software Development

When working with F9 you will most likely do software development in some shape or form.

Most repositories that are used by more than one person follow a continuous integration (CI) workflow to ensure that things are always in a proper state.

You will need to learn how to use our build tool waf, run jobs on the cluster, how git works, commit and review changes with our code review tool Gerrit and many more things.

Building Software or how I learned to stop worrying and love waf

Quick introduction on how to checkout and install a project using waf build tool

Git

As a general rule, everything should be tracked in a version control system, in our case, that is git. Period. If you are not familiar with git we highly recommend spending an hour going through a git tutorial of your choice. Here are some examples:

Whether your own work should be tracked in the group gitviz or not, should be decided together with your supervisor. But mostly, you will need to checkout some repo for read access. If your own/private work is not for the trash, you should request a repository (or contribute to an existing one) on gitviz.

Code Review (Gerrit)

For core software components (and other repositories involving multiple developers), we use gerrit as a code review tool.

The server F9's Gerrit Server and a small tutorial can be found here.

Continuous Integration (Jenkins)

We encourage continuous and automatic testing, see F9's Jenkins Server. Contact YS (or ECM) for details.

Jenkins uses BrainScaleS or Flagship accounts for authentication, see Account section.

Code Style

We have style guides for C++ and Python. [[2]]

To format your C++ code properly, run git-clang-format and inspect/add the resulting changes before committing. To apply clang format on the current commit 

  > singularity exec --app visionary-defaults /containers/stable/latest git-clang-format HEAD^

There is also a convenient vim plugin [[3]]

Bug Reports and Project Management (OpenProject)

Bugs should be posted immediately in the OpenProject project associated with the module that produced the error. The title should be descriptive. As a general rule, the traceback is necessary for the developer to find the actual bug, but the more relevant information are given the easier the fix. Ideally, you create a minimal example that reproduces the problem and upload the script, including the module's loaded. F9's OpenProject Server

How to: Building Papers

We also perform (LaTeX) builds of the latest publications in our containerized software environment. E.g. 

   > singularity exec --overlay /my/local/path/to/containers/overlays/2020-05-28_buster_texlive.img /my/local/path/to/containers/stable/latest make

Please look for the latest overlay and use it. (Back story: We separated our LaTeX installation from the stable containers and use a filesystem overlay to add the LaTeX packages.)

Other information:

  • we try to generate figures from sources during the paper build (e.g. PDF from SVG or such stuff)
  • however, we do store published PDFs (e.g. the pdf that was sent into review, …)

Accessing external repositories

using the git or ssh protocol requires the user to call socksify or tsocks: 

 socksify git clone git://external.server/repo.git # on wheezy default installation
 tsocks git clone git://external.server/repo.git # on 'new' jessie default installation

License

Keep your code contributions (L)GPL-clean because we might want to publish it on a public web site. If you copy code from somewhere, verify license compatibility and mention the source as a code comment!

B.Sc. / M.Sc.

Other requirements

  • B.Sc.: At least 4 weeks (12 max) of internship are required → report (no slides required) in the group meeting.
  • M.Sc.: After 3 months in the group, a MFS talk is required in the group meeting (approx. 20min). It should cover the student's effort to get to know the field, read literature, efforts to learn about the tools and methods, etc. (BTW. the MFP talk is defined by the "Modelhandbuch".

Thesis

B.Sc. colloquium / Theses hand-in dates

  • Find a suitable date for your colloquium and other things (KM & JS & you). A lot of people have full schedules, so do it well in advance!
  • Set all the dates, inform the examination office (Prüfungssek.) and make sure that Felicitas Kleveta is up2date, too.

Thesis Publication

Results Guidelines

Funding acknowledgement

  • for work, where the researcher itself was funded from HBP: "This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement Nos. 720270, 785907 and 945539 (Human Brain Project, HBP)"
  • for work, where the BrainScaleS systems were used (e.g. Bachelor and Master Thesis), please add an acknowledgement for the systems: "The work carried out in this ... Thesis used systems, which received funding from the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreements Nos. 720270, 785907 and 945539 (Human Brain Project, HBP)
  • for the full picture: "HBP official" funding acknowledgement info

(Explanation: 720270 is HBP SGA1 (1.4.2016-31.3.2018), 785907 is HBP SGA2 (1.4.2018-31.3.2020) and 945539 is HBP SGA3 (1.4.2020 - 31.3.2023). For most-current hardware use it is sufficient to acknowledge SGA2 and SGA3.)

  • for work where STRUCTURES or people involved in STRUCTURES (not only funded but involved) were involved, please add an acknowledgement: "This work is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany Ìs Excellence Strategy EXC 2181/1 - 390900948 (the Heidelberg STRUCTURES Excellence Cluster). "

Administrative Stuff

Who should I call for help? (FAQ)

  • The power is not working/toilet is overflowing/... → 54-5111 Störungsstelle („Alles was zur Grundversorgung gehört: Strom/Gas/Wasser“; „der Hausmeister würde ohnehin nur dort anrufen“)
  • Office doors don't open (e.g. empty lock battery), and other "Haustechnik" stuff ⇒ haustechnik@kip.uni-heidelberg.de

Travel

  • before travel: fill out travel request form (Dienstreiseantrag) and hand over to FK
  • after travel: fill out reimbursement form (Dienstreiseabrechnungsformular) (provide invoices etc.) and hand over to FK
  • when getting back the result: check if everything is correct, hand over to OA

The forms can be found here: https://www.kip.uni-heidelberg.de/service/verwaltung/form_forms (needs KIP login).

Vacation

  • Ask your supervisor, you might be needed urgently during the desired time.
  • Fill out your personal "Urlaubskarte" and get a signature from JS. If you have no contract, skip this step. Otherwise it is obligatory before you leave!
  • Hand it over to FK. It will arrive in your post box a few days later with another signature. Now is a good time to note these days as vacation in your timesheet.
  • It is optional but strongly suggested to allow automatic entry of your vacations into an overview. You can opt in at the top of our work info page.

Illness

Here's the latest (2023-04-06) information on how to report yourself sick by Elke Schmetzer:

The procedure for electronic certificates of incapacity to work has been in place since 01.01.2023.

Please note the following changes: From now on, please report to the email: krankmeldung@kip.uni-heidelberg and to your group leader in case of incapacity for work, stating the expected duration. We also need to know whether you are privately or legally insured. If the incapacity for work lasts longer than 3 days and you have been to the doctor and are on sick leave, we and your group leader need the relevant information as described above. The same applies to subsequent sickness notifications. As soon as you are fit for work again, report back to us as before, by email at krankmeldung@kip.uni-heidelberg.de and to your group leader. Privately insured persons will continue to receive a certificate of incapacity for work in paper form and please hand this in to Ms Kalemou in future.

If you have any further questions please feel free to call Ms. Kalemou or me.

Internships

Schein

Windows Software

If you need Windows software, e.g. to fill out forms (Word, Excel), you can log in remotely to a Windows machine: 

rdesktop kipwin

Username: KIP\$your_accountname

(From the outside you can perform a port-forwarding via ssh -L 3389:kipwin.kip.uni-heidelberg.de:3389 hel (if hel is in your ssh config) and run rdesktop localhost.)

Orders

  • above 500€ => multiple offers (print out from some comparison website) needed
  • up to 2.5k€ => get three different offers (ask three shops for offers)
    • e.g. write to first three cheapest shops listed on some comparison websites
    • KIP "Kundennummer"s:
      • Mindfactory: 10970071

Abbreviated Names

 AB:  Andreas Baumbach
 AE:  Arne Emmel
 AFK: Ãkos Kungl
 AG:  Andreas Grübl
 AL:  Aron Leibfried
 BC:  Benjamin Cramer
 BK:  Björn Kindler
 CM:  Christian Mauch
 CP:  Christian Pehle
 CVR: Carl Von Randow
 DH:  Dan Husmann
 ECM: Eric Müller
 FE:  Falk Ebert
 FK:  Felicitas Kleveta
 FP:  Felix Passenberg
 HS:  Hartmut Schmidt
 JG:  Julian Goeltz
 JI:  Joscha Ilmberger
 JM:  José Montes
 JJK: Jakob Kaiser
 JS:  Johannes Schemmel
 JVS: Jan Straub
 JW:  Johannes Weis
 MAP: Mihai Petrovici
 MC:  Milena Czierlinski
 MD:  Markus Dorn
 MG:  Maurice Güttler
 MS:  Maximilian Stucke
 OMA: Oscar Martín-Almendral
 OJB: Oliver Breitwieser
 PD:  Philipp Dauer
 PSP: Philipp Spilger
 RH:  Robin Heinemann
 SB:  Sebastian Billaudelle
 SBO: Sven Bordukat
 SS:  Sebastian Schmitt
 TT:  Tobias Thommes
 VK:  Vitali Karasenko
 YS:  Yannik Stradmann
Retrieved from "https://wiki.kip.uni-heidelberg.de/KIPwiki/index.php?title=Visions:NoobiesTutorial&oldid=39430"