• Institute of Fluid Mechanics and Heat Transfer

Search:

Overview

Basic data

Hardware

Firmware

User's manuals for motherboard/BIOS and BMC/IPMI.

Configuration

All customized configuration files are usually marked by group oswat, sometimes also user oswat,
find /etc -user oswat -o -group oswat

BIOS

press DEL to enter BIOS, F11 for boot menu

Admins

• Thomas Loimer
• Rudolf Ladner (ZID)

Firmware update

Baseboard Management Controller (BMC)

Update firmware, because newer version provides iKVM/HTML5 virtual console and Redfish.
Initially, IP Address of BMC was set to static 128.131.183.xxx - needed to go to the server room and access BMC from the console.

On 2024-04-24, download firmware BMC_X10AST2400-32M_20210528_03.93_STD.zip from https://www.supermicro.com/en/support/resources/downloadcenter/firmware/MBD-X10SRW-F/BMC

Unzip, trying to dump current firmware with binary AlUpdate contained in zip-file resulted in core dump.
Log in into GUI of BMC -> Maintenance -> Update Firmware
upload exactly 32 MiB file BMC_X10AST2400-32M_20210528_03.93_STD.bin Uncheck box Preserve configuration; Network settings thankfully remain.

BIOS

Download firmware on 2024-04-24 from https://www.supermicro.com/en/support/resources/downloadcenter/firmware/MBD-X10SRW-F/BIOS, file X10SRW1.605.zip, unzip. BIOS update needs an activation key, get one by using the tool from https://github.com/bwachter/supermicro-ipmi-key.[[BR]] Generate activation key, ./supermicro-ipmi-key 0c:c4:7a:37:57:9d, output: 90d4 cdb7 ab21 0cf7 33d0 96fa. Log in into GUI of BMC -> BIOS Update, Choose file (exactly 16 MiB) X10SRW1.605, uncheck any preserve-boxes (ME region, NVRAM, SMBIOS). Click Start Upgrade.

BIOS Settings

Advanced -> PCIe/PCI/PnP Configuration:

• RSC-R1UW-E8R SLOT1 PCI-E X8 OPROM, RSC-R1UW-2E16 SLOT1 PCI-E X16 OPROM, RSC-R1UW-2E16 SLOT1 PCI-E X16 OPROM: Set type to EFI, was Legacy.
  Onboard LAN OPROM Type: EFI, was Legacy.
  Boot: Set boot type to EFI, was Dual.

Installation of the base system (2024-04-29)

Boot - the only boot media present is PXE booting - choose
Linuxe Network Installs -> Debian -> Text Based Install

Debian Installer

• Language: C - no localization
  Select your location: Europe -> Austria
  keymap to use: American English (since this is the keymap on the virtual iKVM/HTML5 keyboard)
  Configure the network: eno1 (yields a dhcp address during installation)

  • -> hostname: gs2
    -> domain name: fluid.tuwien.ac.at
    

  Choose a mirror of the Debian archive: enter information manually
  

  • -> mirror hostname: gd.tuwien.ac.at
    -> mirror directory: /opsys/linux/debian
    -> proxy information: (blank)
    

  Set up users and passwords:

  • -> root password: ***
    -> Full name for user account: (blank)
    -> username: oswat
    -> Password for the new user: (same as root)
    

  Partition disks:
  

  • -> Guided - use entire disk
    -> Select disk to partition: SCSI5 (0,0,0) (sda) - 500.1 GB ATA Samsung SSD 850
    -> All files in one partition ... then, change partition to
    

    • #1 267.4 MB B f ESP efi_fs EFI System Partition, bootable (choose 265 MiB)
      #2 465 GB f ext4 root_fs /
      #3 34.8 GB f swap
      

  Configuring popularity-contest: Yes
  Software selection: (nothing except)
  

  • [*] SSH server
    [*] standard system utilities
    

  Reboot

Installation of the base system

Network

Remove legacy ifupdown, use systemd-networkd and systemd-resolved.

ssh oswat@dhcp1 # Log in to the temporary address

# some network details are already set
chgrp oswat /etc/hostname

# the static IP address
cat >/etc/systemd/network/20-gs2.network <<EOF
[Match]
Type=ether

[Network]
Description=Static ethernet connection
Address=128.130.169.115/25
Gateway=128.130.169.1
DNS=128.130.4.3
DNS=128.131.4.3
Domains=fluid.tuwien.ac.at
#NTP=tutimeb.tuwien.ac.at tutimec.tuwien.ac.at tutimea.tuwien.ac.at
NTP=128.130.3.131 128.131.2.3 128.130.2.3
EOF

chgrp oswat /etc/systemd/network/20-gs2.network

# remove legacy network stack
apt purge ifupdown && systemctl start systemd-networkd

After that, the shell freezes, kill ssh, log in again,

ssh oswat@gs2
su - # change to root
systemctl enable systemd-networkd
# systemd-resolved is supposed to have DNS caching,
# use it instead of a manual /etc/resolv.conf file
apt install systemd-resolved
# these are packages recommended by systemd-resolved; here they are useful
apt install libnss-myhostname libnss-resolve

# append my public key to .ssh/authorized_keys
# scp ed25519.pub >/root/.ssh/authorized_keys

Package sources

Use the local mirror gd.tuwien.ac.at, as entered during installation.
Do not install recommended packages.

chgrp oswat /etc/apt/sources.list

echo 'APT::Install-Recommends "false";' >/etc/apt/apt.conf.d/90recommended_false
chgrp oswat /etc/apt/apt.conf.d/90recommended_false

sshd does not accept environment

Do not forward the client's locale, since there is only C/POSIX on gs1. # Also, there is no sudo, allow root to login. # echo "PermitRootLogin yes" >/etc/ssh/sshd_config.d/permitrootlogin.conf

apt install patch
patch /etc/ssh/sshd_config <<EOF
--- /etc/ssh/sshd_config.orig   2023-04-12 16:19:45.904116844 +0200
+++ /etc/ssh/sshd_config        2023-03-29 10:20:37.697903087 +0200
@@ -109,7 +109,7 @@
 #Banner none

 # Allow client to pass locale environment variables
-AcceptEnv LANG LC_*
+#AcceptEnv LANG LC_*

 # override default of no subsystems
 Subsystem      sftp    /usr/lib/openssh/sftp-server
EOF

chgrp oswat /etc/ssh/sshd_config

Debian bookworm (12.0) was installed on the 500 GB SSD, one partition (≈100 MB) for the efi (/boot/efi), the reminder of the disk for root (/), ext4 filesystem. The file systems were labelled, root_fs and home_fs.
While in the debian installer, ssh-server was chosen, no localications, only C and C.utf8.
Install grub2 as bootloader; The kernel efi-stub needs a commandline (root=/dev/sda2 initrd=\EFI\debian\initrd.img), no way to set that from the BIOS.
Had to create an administrative user: oswat, the home directory is /opt/oswat!

Partitioning

File systems

apt install btrfs-progs zstd
# create a btrfs-raid1 for data (-d) and medatadata (-m) from /dev/sdb and /dev/sdc
mkfs.btrfs -f -L home_fs -d raid1 -m raid1 /dev/sdb /dev/sdc

cat >>/etc/fstab <<EOF
# / was on /dev/sda2 during installation
LABEL=root_fs   /               ext4    noatime,errors=remount-ro 0       1
# /boot/efi was on /dev/sda1 during installation
UUID=5AA6-2350  /boot/efi       vfat    umask=0077                0       1
LABEL=home_fs   /home           btrfs   compress=zstd             0       2
EOF

Installation sources

User access

sshd customization

Allow all members of E322 to log in

# see https://arthurdejong.org/nss-pam-ldapd/
apt install libnss-ldapd libpam-ldapd nscd

# here is the entire /etc/nslcd.conf file
# some options in the first 20 lines might be set at installation time
cat >/etc/nslcd.conf <<EOF
#
# /etc/nslcd.conf
# nslcd configuration file. See nslcd.conf(5)
# for details.

# The user and group nslcd should run as.
uid nslcd
gid nslcd

# The location at which the LDAP server(s) should be reachable.
uri ldaps://dc.intern.tuwien.ac.at

# The search base that will be used for all queries.
base ou=tu,dc=intern,dc=tuwien,dc=ac,dc=at

# The LDAP protocol version to use.
#ldap_version 3

# The DN to bind with for normal lookups.
binddn cn=E322_LDAP,ou=interactive,ou=exchange,ou=IT-services,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at
bindpw **<confidential>**

# The DN used for password modifications by root.
#rootpwmoddn cn=admin,dc=example,dc=com

# SSL options
#ssl off
tls_reqcert never
tls_cacertfile /etc/ssl/certs/ca-certificates.crt

# The search scope.
#scope sub

# Here ends the shipped configuration file, customisation starts ...

base    passwd  ou=people,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at
base    shadow  ou=people,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at
base    group   ou=groups,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at

# Mappings
filter  passwd  (memberOf:1.2.840.113556.1.4.1941:=cn=E322_ALL,ou=groups,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at)
map     passwd  uid             sAMAccountName
map     passwd  uidNumber       employeeID
map     passwd  gecos           cn
map     passwd  homeDirectory   "/home/${sAMAccountName}"
map     passwd  loginShell      "/bin/bash"
map     passwd  gidNumber       "2153"


filter  shadow  (memberOf:1.2.840.113556.1.4.1941:=cn=E322_ALL,ou=groups,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at)
map     shadow  uid             sAMAccountName

filter  group   (cn=E322*)
map     group   userPassword    ""
map     group   gidNumber       objectSid:S-1-5-21-527783839-1561677997-9029855232
EOF

Do not forget to have users have a home directory,

pam-auth-update mkhomedir
# Installation of libpam-ldapd most probably runs
# pam-auth-update ldap

Enable hibernation

Create a swap file (not a swap partition), enable swap and modify the kernel command line to search for a RAM image.
Use filefrag to get the offset of the swap file.
It is not necessary to modify etc/initramfs-tools/conf.d/resume.

touch /swap
chmod 600 /swap
dd if=/dev/zero of=/swap bs=1M count=32768
filefrag -v /swap | head
# Use the number in the first row, first column of the "physical offset:" columns.
# This number has two dots appended (here: 202752..).
echo GRUB_CMDLINE_LINUX_DEFAULT=\"resume=PARTLABEL=root_partition resume_offset=202752\" \
    >/etc/default/grub.d/resume.cfg
echo "/swap    swap    swap    defaults    0    0"  >>/etc/fstab

Mark customized files

cd etc
chgrp oswat hostname fstab apt/sources.list apt/apt.conf.d/90recommended_false \
    systemd/network/10-gs1.network sshd_config sshd_config.d/permitrootlogin.conf \
    default/grub.d/resume.cfg
chown oswat nslcd.conf