Description of the Lisa system

What is the Lisa system

The Lisa system is a cluster computer consisting of several hundreds of multi-core nodes running the Linux operating system. The system is installed and maintained by SURFsara.

Participants

The following participants are involved in the Lisa system:

  • The University of Amsterdam (UvA)
  • The VU University Amsterdam (VU)
  • The SURF organisation (SURF, which has taken over from NWO)

System configuration

The Lisa system is constantly evolving and growing to satisfy the needs of the participants. At this moment the configuration is as follows:

Number Type Clock Scratch Memory Cache Cores InfiniBand
128 L5520 2.26 GHz 85 GB 24 GB QPI 5.86 GT/s 8 MB 8 -
256 L5520 2.26 GHz 85 GB 24 GB QPI 5.86 GT/s 8 MB 8 Qlogic
32 L5640 2.26 GHz 220 GB 24 GB QPI 5.86 GT/s 12 MB 12 -
64 L5640 2.26 GHz 220 GB 24 GB QPI 5.86 GT/s 12 MB 12 Qlogic
144 E5-2650L 1.80 GHz 750 GB 32 GB QPI 8.00 GT/s 20 MB 16 -
32 E5-2650 v2 2.60 GHz 870 GB 32 GB QPI 8.00 GT/s 20 MB 16 -
Total number of cores 7040
Total amount of memory 17 TB
Total peak performance 82 TFlop/sec
Disk space 100 TB for the home file systems
Operating system Debian Linux AMD64 OS
Qlogic 4x DDR Infiniband network
Bandwidth 1600 MB/sec
Latency <6 µsec

Managing Lisa

Most of the software that is used to manage the Lisa system is Open Source. We are using the following software to manage the system:

  • CFEngine 3, A configuration engine
  • As batch software we use Torque and Maui. For Torque we have devloped a pbs Python interface. So it is now possible to develop batch utilities in Python instead of C.
  • torque_2_deb, SURFsara has developed software to make a debian package from the Torque source
  • Ganglia. For ganglia SURFsara has developed an jobmonarch add-on to monitor OpenPBS/Torque queues. https://ganglia.surfsara.nl
  • SALI, SALI (Sara Automatic Linux Installer) is a tool that allows you to install Linux on multiple machines at once. It support several protocols for downloading by way of aria2 to install a machine. For example, BitTorrent and rsync are supported. SALI originates from SystemImager and still uses the same philosophy. It is a scalable method for performing unattended installation. SALI is mostly used in cluster setups.