SLURM Cluster - Dashboard and other tools

This project is part of my work in the Visual Computing Group (see my work experience here).

The lab was not familiar with SLURM and job queues. Part of my responsibilities was to set up the system and ensure that researchers could understand and efficiently use the new cluster setup.

To achieve this, I developed a website, along with all the necessary back-end systems, so that users could visualize the state of the cluster, their jobs, and resource usage. Additionally, none of the members were familiar with Linux DevOps or SLURM. It was my job to understand their requirements and translate them into practical implementations.

Milestones

• Cluster Setup

  • LDAP server and user authentication accross nodes
  • NFS storage accessible accross all nodes, mounted on demand with autofs
  • SLURM cluster setup, user accounts, welcome to the group email, and priority management
  • Metrics collection with Graphana and Telegraf
  • Netboot and node boot configurations
  • Ansible Playbooks
  • Storage benchmarks
  • Modulefiles
  • Storage quotas
  • Enroot integration

• Dashboard

  • Web interface with Vue.js
  • Backend development with Flask

• Others

  • Maintenance tasks and updates

Other Developed Tools

LDAP Triggers

GitHub: https://github.com/marcbenedi/ldap-triggers

LDAP Triggers is an Open Source python package under MIT license which triggers an action every time an LDAP action is done. The current supported actions are creation/deletion of users/groups.

It comes with very clear documentation and it’s very easy to set up.

More information: https://blog.marcb.pro/posts/ldap-triggers/

LDAP Emails

GitHub: https://github.com/marcbenedi/ldap-email

This tool allows sending (templated) emails to users in an LDAP database by username or group.

$ ldapemail send -u user1 -u user2 email-template
$ ldapemail send -g group1 group-template

Gallery

UI/UX is not one of my strengths 😅, but I am still proud of the website, as it is still used in the lab and people find it very convinient.

Parts of the image may be blurried to preserve user and node information.

Don’t be shy, click on the images! 👇