Non-Critical Systems

Non-critical systems are those that are managed by the infrastructure administrator team.

• Infra02
• Infra03
• Authserver
• Blog
• Board
• Bugs
• CATS
• Code
• Community
• Email
• Emailout
• Git
• IDP
• Ingress03
• Ircserver
• Issue
• Jenkins
• Lists
• MariaDB
• Monitor
• Motion
• Nextcloud
• OIDC Demo
• PostgreSQL
• Proxyin
• Proxyout
• Puppet
• Svn
• Test
• Test2
• Test3
• Testmgr
• Translations
• Web
• Webstatic
• Wiki

General

Todo

consider whether a central MySQL service should be used

Many containers contain their own instance of MySQL. It might be a better idea to centralize the MySQL setups in a single container.

A shareable MariaDB instance is available on MariaDB.

Todo

consider whether a central PostgreSQL service should be used

A shareable PostgreSQL instance is available on PostgreSQL.

Todo

setup a central syslog service and install syslog clients in each container

Setup package update monitoring for a new container

For Icinga to be able to check the update status of packages on you server you need to install NRPE, a helper service. Install the necessary packages:

sudo aptitude install nagios-plugins-basic nagios-nrpe-server

Put Monitor on the list of allowed hosts to access the NRPE service by adding the following line to /etc/nagios/nrpe_local.cfg:

allowed_hosts=172.16.2.18

Tell the NRPE service that there is such a thing as the check_apt command by creating the file /etc/nagios/nrpe.d/apt.cfg with the following contents:

# 'check_apt' command definition
command[check_apt]=/usr/lib/nagios/plugins/check_apt

# 'check_apt_distupgrade' command definition
command[check_apt_distupgrade]=/usr/lib/nagios/plugins/check_apt -d

Restart the NRPE service:

sudo service nagios-nrpe-server restart

Check that everything went well by going to https://monitor.cacert.org/, going to the APT service on the host and clicking “Re-schedule the next check of this service”. Make sure that “Force Check” is checked and click “Commit”. Now you should see a page with a green background. If not something went wrong, please contact the Monitor administrators with the details.

That’s it, now the package update status should be properly displayed in Icinga.

Checklist

• All containers should be monitored by Monitor and should therefore have icinga2 installed and managed via Puppet (older systems without Puppet have nagios-nrpe-server installed)

• All containers should use etckeeper to put their local setup into version control. All local setup should use /etc to make sure it is handled by etckeeper

• All infrastructure systems must send their mail via Emailout

• All infrastructure systems should have an system-admin@cacert.org alias to reach their admins

Todo

document how to setup the system-admin alias on the email system