Maintainance and Setup of s15.fluid.tuwien.ac.at

Contents

  1. Maintainance and Setup of s15.fluid.tuwien.ac.at
    1. Installation of software packages
    2. Ssh-daemon
    3. Network setup
    4. Firewall
    5. Certificates
    6. File server
    7. Dhcp server
    8. Print server
    9. Time synchronization
    10. Privileges for oswat
    11. Resource usage metrics
    12. Netgen/NDSolve

The operation of s15 can be remotely controlled from its management interface at ms15.fluid.tuwien.ac.at.

The credentials for the management interface, and also for the root and oswat accounts on s15 are known to Werner Jandl, Sebastian Böhm, Thomas Loimer and Francesco Zonta. The user account oswat on s15 is used for administrative purposes. Although customizations can only be done by root, any modified files, e.g., in /etc/, should be owned by oswat. If, in addition, the original files are saved with the suffix .orig, it is easy to find the differences to a stock debian system. Therefore, for instance, to customize dhcpd.conf, 

su; cd /etc
cp -p dhcp/dhcpd.conf dhcp/dhcpd.conf.orig
edit dhcp/dhcpd.conf
chown oswat:oswat dhcp/dhcpd.conf
# find differences; some files, e.g., in logrotate.d,
# must be owned by root, group can be oswat
find /etc -user oswat -o -group oswat -o -name "*.orig"

The operating system on s15 is debian. Type lsb_release -a to get information on the current release, and type uname -a to get the version of the running kernel.

To keep the operating system up to date, from time to time the commands 

apt-get update
apt-get upgrade
apt-get dist-upgrade

should be issued, as root.

Installation of software packages

The list of software installed on s15 is maintained on a separate page, s15-software. Please, if any software is installed, e.g., using apt-get install, update that list.

The packages are retrieved from the official debian mirror at ftp.at.debian.org, as listed in /etc/apt/sources.list.

Ssh-daemon

As per the default configuration of the ssh-daemon, root is not allowed to log in to s15 via password authentication from a remote client. Therefore, log in with your regular account or as oswat, and su to root. Alternatively, create a public/private key pair.

The default configuration of the ssh-daemon given in /etc/sshd/sshd_config has been changed in one single instance. Clients are not allowed to pass the environment variables related to their locale settings to their shell on s15, 

# Allow client to pass locale environment variables
#AcceptEnv LANG LC_*

The machinery for authentication via ldap, i.e., by querying the database that also feeds tiss, is documented at ldap-authentication or at loginviatiss (in german).

Network setup

S15 has, apart from the managment port at 128.130.169.3, four ethernet ports, which are bonded together to one interface, visible at 128.130.169.2. The network is configured from /etc/network/interfaces using the package ifupdown, not yet via systemd configuration files.

Firewall

There is a local firewall installed on s15, which serves to stop dictionary attacks. These manifest themselves by frequent log-in attempts with common username–password combinations. The log-in attempts can be observed in /var/log/auth.log. The firewall is implemented by iptable rules. These rules count the number of log-in attempts from a given ip-number, and block that number if more than 6 attempts are tried. On a succesful login, a command in the pam stack removes the ip-number from which the succesful login originated from the list of tracked addresses.

The list of a maximum of 100 tracked or blocked ip-addresses is kept in /proc/net/xt_recent/DEFAULT. To manually unblock an ip-address, do, as root, 

# A minus (-) in front of the number
# removes that number from the table.
echo -ip.num.ber > /proc/net/xt_recent/DEFAULT

See man iptables_extensions(8), module xt_recent, and man pam_exec(3).

The iptable-rules that populate the list of addresses in proc/net/xt_recent/DEFAULT can be found in /etc/network/iptables.up.rules. The latter file was created with iptables-save > /etc/network/iptables.up.rules. The script /etc/network/if-pre-up.d/iptables loads the iptable-rules at each startup of the server. On a succesful login, a pam-module, see the line with pam_exec.so in /etc/pam.d/sshd, calls the script /etc/network/unblock.

Certificates

SSL Certificates have been issued for www.fluid.tuwien.ac.at, files.fluid.tuwien.ac.at, druck.fluid.tuwien.ac.at and s15.fluid.tuwien.ac.at. The certificates are stored under /etc/ssl/certs/s15.crt and /etc/ssl/private/s15.key. Symbolic links point to them from /etc/cups/ssl/*. The certificates have to be renewed once a year, 1st of dezember. There is a renewal id in the e-mail shipping the download link. The e-mail has "SSL certificate" in its subject line. Alternatively, create a certificat request, 

openssl req -utf8 -sha256 -new -newkey rsa:2048 -nodes \
    -out "s15_fluid_tuwien_ac_at.csr" -keyout "s15_fluid_tuwien_ac_at.key" \
    -subj "/C=AT/ST=Austria/L=Vienna/O=Technische Universität Wien/OU=E322/CN=s15.fluid.tuwien.ac.at/subjectAltName=DNS.1=s15.fluid.tuwien.ac.at/subjectAltName=DNS.2=druck.fluid.tuwien.ac.at/subjectAltName=DNS.3=files.fluid.tuwien.ac.at/subjectAltName=DNS.4=www.fluid.tuwien.ac.at"
# request a "Géant OV Multi-Domain" certificate
# Subject Alternative Names:
#     s15.fluid.tuwien.ac.at,druck.fluid.tuwien.ac.at,files.fluid.tuwien.ac.at,www.fluid.tuwien.ac.at

# Approximately two days later, download and install the new certificate,
export CERTIFICATE_URL='https://cert-manager.com/customer/ACOnet/ssl?action=download&sslId=2976980&format=pemia'
wget -O s15_fluid_tuwien_ac_at.crt "$CERTIFICATE_URL"
install -T --owner=root --group=ssl-cert --mode=640 --backup \
    s15_fluid_tuwien_ac_at.key /etc/ssl/private/s15.key
install -T --owner=root --group=root     --mode=644 --backup \
    s15_fluid_tuwien_ac_at.crt /etc/ssl/certs/s15.crt
service apache2 reload
service apache2 status

File server

The file server is implemented by a webdav cgi wrapper program from https://danrohde.github.io/webdavcgi. Currently, there is one install under /usr/local/share/webdavcgi, and the development version is installed under /usr/local/share/webdavcgi-git. Configuration files to use either version are provided under /etc/apache2/sites-available/. Look at /etc/apache2/sites-enabled to see which version is active.

For requirements, look at https://danrohde.github.io/webdavcgi/doc.html#installation . In addition, do 

apt install libdatetime-format-human-duration-perl


# clone the source tree into /usr/local/share/webdavcgi-git
root@s15:/usr/local/share# git https://github.com/DanRohde/webdavcgi.git webdavcgi-git
cd webdavcgi-git
# this script sets the appropriate permissions
./install.sh
# create the apache configuration file
cat >/etc/apache2/sites-available/files-git.conf <<EOF
<VirtualHost *>
        # Only allow secure connections to files.fluid.tuwien.ac.at
        ServerName files.fluid.tuwien.ac.at
        ServerAlias files
        ServerAlias files.fluid
        RedirectPermanent / "https://s15.fluid.tuwien.ac.at/"
</VirtualHost>

<VirtualHost *:443>
        ServerName s15
        # https://s15 would return "Your connection is not secure ...
        # ... only valid for the following names: files.fluid.tuwien.ac.at,...
        # The UseCanonicalName is also necessary here.
        # This is different from druck.conf (?!), see there.
        ServerAlias files.fluid files.fluid.tuwien.ac.at s15 s15.fluid
        RedirectPermanent / "https://s15.fluid.tuwien.ac.at/"
</VirtualHost>

<VirtualHost *:443>
        ServerName s15.fluid.tuwien.ac.at
        # Use the ServerName supplied by the client - useful when logging in,
        # otherwise the user would have to login to, e.g., files and to
        # files.fluid.tuwien.ac.at
        # Probably not needed with the Redirect
        UseCanonicalName On

        ServerAdmin thomas.loimer@tuwien.ac.at
        #DocumentRoot /var/www/html

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # The SSL-stuff
        SSLEngine on
        # The Installation Instructions from digicert require to
        # also install the DigiCertCA.crt as SSLCertificateChainFile.
        # Also the apache documentation describes this directive as obsolete
        # and recommends to use SSLCertificateFile, the latter does not work.
        SSLCertificateFile /etc/ssl/certs/s15.crt
        SSLCertificateChainFile /etc/ssl/certs/DigiCertCA.crt
        SSLCertificateKeyFile /etc/ssl/private/s15.key
        # from /etc/apache2/sites-available/ssl-default.conf
        BrowserMatch "MSIE [2-6]" \
                        nokeepalive ssl-unclean-shutdown \
                        downgrade-1.0 force-response-1.0
        # MSIE 7 and newer should be able to use keepalive
        BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown

        ScriptAlias /webdavcgi /usr/local/share/webdavcgi-git/cgi-bin/webdavwrapper
        # Never found this log.
        ScriptLog /tmp/cgi.log

        <Location /webdavcgi>
                AuthType Basic
                AuthBasicProvider ldap
                AuthName "Your upTUdate credentials"
                # mod_authnz_ldap
                # AuthLDPAUrl ldap://host:port/basedn?attribute?scope?filter
                AuthLDAPUrl ldaps://dc.intern.tuwien.ac.at/ou=people,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at?sAMAccountName?one
                AuthLDAPBindDN CN=E322_LDAP,OU=interactive,OU=exchange,OU=IT-services,OU=TU,DC=intern,DC=tuwien,DC=ac,DC=at
                AuthLDAPBindPassword <my-ldap-passwd>
                # this does not work any longer, since 2019-01-01
                #Require ldap-group cn=E322,ou=groups,ou=TU,dc=intern,dc=tuwien,dc=ac,dc=at
                Require ldap-attribute showInAddressBook="CN=E322,CN=TU Wien,CN=All Address Lists,CN=Address Lists Container,CN=TU Wien,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=intern,DC=tuwien,DC=ac,DC=at"
        </Location>

        # Correctly set $VIRTUAL_BASE and $DOCUMENT_ROOT in /etc/webdav.conf!
        # Otherwise, paths like /home//tloimer, /home///tloimer, etc, result.
        RewriteEngine   On
        RewriteRule ^/ /webdavcgi [PT,E=WEBDAVCONF:/usr/local/share/webdavcgi/webdav.conf,E=PERLLIB:/usr/local/share/webdavcgi-git/lib/perl,L]

        # Seems to be unnecessary. Probably necessary, when additional backends
        # (smb, kerberos,..) are used?
        #<Directory "/usr/local/share/webdavcgi/cgi-bin">
        #       AllowOverride None
        #       Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        #       #Require all granted
        #       Require valid-user
        #</Directory>
</VirtualHost>
EOF

Create the configuration file for the webdav-program, 

cat >/usr/local/share/webdavcgi-git/webdav.conf <<EOF
# See http://webdavcgi.sourceforge.net
## the install base is needed to find webdav-ui.* and locale files
## (don't forget the trailing slash):
$INSTALL_BASE = '/usr/local/share/webdavcgi-git/';

## the backend module (supported: FS, AFS, GFS, SMB, DBB, RCS, GIT):
$BACKEND = 'FS';

## this is an example if a user starts with home dir (http://mywebdavserver/ -> user home):
$VIRTUAL_BASE = '/';
#$DOCUMENT_ROOT = '/home/'.$ENV{REMOTE_USER}.'/';
$DOCUMENT_ROOT = '/home/';
## if you use a complex home folder structure, try this:
# $DOCUMENT_ROOT=(getpwnam($ENV{REMOTE_USER}))[7].'/';


$DBI_SRC='dbi:SQLite:dbname=/var/spool/webdav/'.$ENV{REMOTE_USER}.'.db';
$DBI_USER='';
$DBI_PASS='';
$CREATE_DB = !-e '/var/spool/webdav/'.$ENV{REMOTE_USER}.'.db';

$DEBUG=0;
@EXTENSIONS = ('Permissions', 'PosixAclManager', 'TextEditor', 'Download', 'Zip');
$LOGFILE='/var/log/webdavcgi.log';

#th /tmp/webdav would not work,
#   because /tmp/webdav is created with the current user as owner
$THUMBNAIL_CACHEDIR='/var/cache/webdav';
#
## FINE TUNING ##
$TITLEPREFIX='files/home:';
$SHOWDOTFILES = 0;
$SHOWDOTFOLDERS = 0;
$ENABLE_DAVMOUNT = 1;
#@ALLOWED_TABLE_COLUMNS = ('name','size','lastmodified','created','mode','mime','uid','gid');
@VISIBLE_TABLE_COLUMNS = ('name', 'mode', 'uid', 'gid', 'size', 'lastmodified');
push @VISIBLE_TABLE_COLUMNS, 'fileactions' if $ALLOW_FILE_MANAGEMENT;
$LANGSWITCH = '<div style="font-size:0.8em;text-align:right;border:0px;padding:0px;"><a href="?lang=default">[EN]</a>&nbsp;<a href="?lang=de">[DE]</a>&nbsp;<a href="?lang=fr">[FR]</a>&nbsp;<a href="?lang=it">[IT]</a>&nbsp;$CLOCK</div>';

## -- HEADER
## content after body tag in the Web interface
$HEADER = '<div class="header">files/home -  Web interface: You are logged in as ${USER}.<div style="float:right;font-size:0.8em;">$NOW</div></div>';

#$POST_MAX_SIZE=1073741824; # 1GB
$POST_MAX_SIZE=5000000000;

$BUF_SIZE=1073741824; # 1GB

$ENABLE_CARDDAV = 0;
$ENABLE_GROUPDAV = 0;

$ENABLE_THUMBNAIL = 0;
$ENABLE_THUMBNAIL_PDFPS = 0;

%FILECOUNTPERDIRLIMIT = ();
%FILEFILTERPERDIR = ();
EOF
# check the syntax of the configuration file
perl -c webdav.conf

webdavcgi assumes to be installed under /etc/webdavcgi. Here, the program is installed under /usr/local/share/webdavcgi-git. Look for occurences of etc/webdavcgi under /usr/local/share/webdavcgi-git and correct these, 

fgrep -r 'etc/webdavcgi'
sed -i 's/etc\/webdavcgi/usr\/local\/share\/webdavcgi-git/g' helper/mod_perl_startup.pl

Disable the old webdavcgi-site, enable the new one, 

a2dissite files.conf
a2ensite files-git.conf
# reload apache
systemctl reload apache2

Dhcp server

S15 also is the dhcp-server at our institute, that is, it provides ip-numbers to those machines that do not have a fixed ip-address, e.g., laptops. At present, there is a pool of 7 ip-numbers. Therefore, users should set their computer to a fixed ip-address as soon as possible. The configuration-file for the dhcp-server, /etc/dhcp/dhcpd.conf, has the following options set differently from the configuration file shipped with the package, now /etc/dhcp/dhcpd.conf.orig: 

# option definitions common to all supported networks...
option domain-name "fluid.tuwien.ac.at";
# tunamea.tuwien.ac.at, tunameb.tuwien.ac.at
option domain-name-servers 128.130.4.3, 128.131.4.3;
option lpr-servers 128.130.169.2;
# ntp-servers - RFC 1035 protocol
# tutimeb.tuwien.ac.at, tutimea.tuwien.ac.at, tutimec.tuwien.ac.at;
option ntp-servers 128.130.3.131, 128.130.2.3, 128.131.2.3;
# time-servers - RFC 868 protocol
# tutimeb.tuwien.ac.at, tutimea.tuwien.ac.at;
option time-servers 128.130.3.131, 128.130.2.3;

default-lease-time 7200;   # orig:  600
max-lease-time 14400;      # orig: 7200

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# the ip-numbers which can be dynamically distributed
subnet 128.130.169.0 netmask 255.255.255.128 {
  range 128.130.169.54 128.130.169.56;
  range 128.130.169.65 128.130.169.66;
  range 128.130.169.72;
  range 128.130.169.63;
  option routers 128.130.169.1;
}

Print server

S15 also is a print server. The web interface of the cups print server can be accessed at the alias of s15, https://druck.fluid.tuwien.ac.at. The print server is configured such that it can be accessed from any address within TU Wien. Therefore, one can print from laptops connected via WLAN. The configuration of the cups server is stored in the file /etc/cups/cupsd.conf.

By default, cupsd generates self-signed certificates and stores them under /etc/cups/ssl/s15.fluid.tuwien.ac.at.{crt,key}. These files were replaced by symbolic links to the correct certificate under /etc/ssl/. This also switches off generation of self-signed certificates,

The differences from the shipped version of cupsd.conf are 

--- /etc/cups/cupsd.conf.orig   2015-06-09 10:42:36.000000000 +0200
+++ /etc/cups/cupsd.conf        2017-07-25 14:30:10.440857673 +0200
@@ -13,8 +13,20 @@
 MaxLogSize 0

-# Only listen for connections from the local machine.
-Listen localhost:631
+#thomas: the /var/spool/cups directory filled up to 1.5G
+# only store a limited amount of jobs (Default: 500)
+MaxJobs 40
+
+# Listen for connections from remote machines.
+#Listen 128.130.169.2:631
+Listen *:631
 Listen /var/run/cups/cups.sock

+# With ServerName commented out, there were "BadRequests",
+# no matter what was done with Listen.
+#ServerName druck.fluid.tuwien.ac.at
+ServerAlias druck.fluid.tuwien.ac.at
+ServerAlias druck.fluid
+ServerAlias druck
+
 # Show shared printers on the local network.
 Browsing On
@@ -30,4 +42,8 @@
 <Location />
   Order allow,deny
+#  Allow from @LOCAL
+# Allow from within TU Wien
+  Allow from 128.130.0.0/15
+#  Order deny,allow
 </Location>

@@ -35,4 +51,5 @@
 <Location /admin>
   Order allow,deny
+  Allow from 128.130.169.0/24
 </Location>

@@ -41,5 +58,6 @@
   AuthType Default
   Require user @SYSTEM
-  Order allow,deny
+  Order deny,allow
+#  Allow from @LOCAL
 </Location>

@@ -53,5 +71,5 @@

   # Job-related operations must be done by the owner or an administrator...
-  <Limit Create-Job Print-Job Print-URI Validate-Job>
+  <Limit Create-Job Print-Job Print-URI Validate-Job Cancel-Job>
     Order deny,allow
   </Limit>
@@ -76,6 +94,6 @@
   </Limit>

-  # Only the owner or an administrator can cancel or authenticate a job...
-  <Limit Cancel-Job CUPS-Authenticate-Job>
+  # Only the owner or an administrator authenticate a job...
+  <Limit CUPS-Authenticate-Job>
     Require user @OWNER @SYSTEM
     Order deny,allow

The printer description files for all printers, except canon, were installed from the web-interface of cups, after installing hplip. The printer description file for canon, /etc/cups/ppd/canon.ppd was extracted from the tar-archive of the CQue printer driver, which was obtained from the canon support page, http://www.canon.at/support/products/imagerunner/ir2230.aspx

hplip bug

Some files did not print, with an error in /var/log/cups/error.log UnicodeEncodeError: \'utf-8\' codec can\'t encode character ... surrogates not allowed. Fix with 

--- /usr/share/hplip/base/sixext.py.orig        2017-05-04 18:35:44.000000000 +0200
+++ /usr/share/hplip/base/sixext.py     2017-10-20 11:11:55.722454094 +0200
@@ -107,5 +107,5 @@

     def to_bytes_utf8(s):
-        return s.encode("utf-8")
+        return s.encode("utf-8", errors="surrogateescape")

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=879115, or https://bugs.launchpad.net/hplip/+bug/1498366.

At the next update of hplip, do not forget to delete the file usr/share/hplip/base/sixext.py.orig.

Time synchronization

The computer clock is synchronized with the TU time servers, not with the ntp pool servers from debian, /etc/systemd/timesyncd.conf. 

timedatectl set-ntp true
rm /etc/localtime
cp /usr/share/zoneinfo/Europe/Vienna /etc/localtime

Privileges for oswat

The privileges for oswat were elevated slightly. 

# For cups, lpadmin; adm to see log files
usermod -a -G adm,lpadmin oswat

Resource usage metrics

Resource usage on s15 is tracked using munin: 

apt-get install munin munin-node libapache2-mod-fcgid
a2enmod fcgid
cp -n /etc/munin/munin.conf /etc/munin/munin.conf.orig
cp -n /etc/munin/apache24.conf /etc/munin/apache24.conf.orig
chown oswat:oswat /etc/munin/munin.conf
chown oswat:oswat /etc/munin/apache24.conf
cat >/etc/munin/munin.conf <<EOF
includedir /etc/munin/munin-conf.d

[s15.fluid.tuwien.ac.at]
    address 127.0.0.1
    use_node_name yes
EOF
cat >/etc/apache2/conf-available/munin.conf <<EOF
Alias /munin /var/cache/munin/www
<Directory /var/cache/munin/www>
        <RequireAny>
            Require ip 128.131.36.0/24
            Require ip 128.130.169.0/25
        </RequireAny>
        Options None
</Directory>

ScriptAlias /munin-cgi/munin-cgi-graph /usr/lib/munin/cgi/munin-cgi-graph
<Location /munin-cgi/munin-cgi-graph>
        <RequireAny>
            Require ip 128.131.36.0/24
            Require ip 128.130.169.0/25
        </RequireAny>
        <IfModule mod_fcgid.c>
            SetHandler fcgid-script
        </IfModule>
        <IfModule !mod_fcgid.c>
            SetHandler cgi-script
        </IfModule>
</Location>
EOF
service munin-node restart
service apache2 restart

Netgen/NDSolve

See https://ngsolve.org and the documentation for updating the software and setting the necessary environment variables for running the software. 

mkdir /usr/local/src/ngsuite
cd /usr/local/src/ngsuite
git clone git://git.code.sf.net/p/ngsolve/git ngsolve-src
cd ngsolve-src
git submodule update --init --recursive
mkdir ngsolve-build
cd ngsolve-build
# This will set the installation location to /opt/netgen
# For a different location, try, e.g.,
#   cmake -DCMAKE_INSTALL_PREFIX=/usr/local/share/netgen
cmake /usr/local/src/ngsuite/ngsolve-src/
make -j
make install