Services – Linux Administration

HA Linux Mail Server – Part 0

I’ve been working on a project to learn more about RHCS in general and decided to build a HA Mail server.

Technologies Used:
– Centos 6.6
– RHCS
– Postfix
– Dovecot (IMAP)
– OpenLDAP
– GlusterFS
– MySQL
– Roundcube

Over the coming weeks I’ll be posting the steps required to set this up.

The general idea is to have two RHCS services

mail-svc

VIP for postfix/dovecot
glusterfs for delivered mail, postfix and dovecot configuration files

mysql-svc

VIP for mysql
glusterfs for mysql config and data

Each node runs the following locally

httpd + roundcube
OpenLDAP replicating from another LDAP server

While the above two services could certainly be cluster services, this is not required.

We NAT our public IP to the VIP for mail-svc.

To avoid SSH key issues, copy your SSH keys from /etc/ssh to the other node (So SSH to the public IP will not result in errors after a failover)

Run Levels

In Linux, you generally have run levels that range from 0-6.

0: Halt (shutdown) the system
1: Single user mode (major upgrades, maintenance, etc.)
2: Basic multi user mode
3: Full multi user mode
4: Unused (custom)
5: Multi user mode with GUI
6: Reboot

0,1 and 6 are always the same, but different distros do different things with the other run levels (for example, Debian-based systems use run level 2 for GUI and full multi-user mode)

You can see what services start at what run level by looking in the directory /etc/rc#.d where # is the run level number.

The run level number can be found in /etc/inittab (id:2:initdefault: for example)
If you were to look in /etc/rc2.d, you would see files that look like:

S99program

These are usually symbolic links to files in /etc/init.d (so a change to /etc/init.d/program would change the programs script for every run level).

Start-up scripts in Linux follow the form “start-stop” (at minimum). Many have “start-stop-restart-reload-check” or possible more.

A basic start-stop script would look like this:

#!/bin/bash
case $1 in
start)
program
;;
stop)
program
;;
esac

Don’t forget

chmod +x script

This is useful to know, especially if you are dealing with a machine that may need specific commands to be run when booting or shutting down in order to function properly (for example, in the radio station we need the client machines to run dhclient at S99 in order to obtain an IP address).

These scripts are not all run at once (by default). S##program where ## is the priority of the script.

S99 will run last, while S10 will run before most other things.

OpenSolaris mod_bw Apache2 Bandwidth Limiting

Install needed software

pkg install gsed wget SUNWapch22 sunstudioexpress

Yes, sunstudioexpress is really necessary. gcc will NOT work with mod_bw.
Download mod_bw

cd /tmp
wget http://ivn.cl/files/source/mod_bw-0.8.tgz
gunzip mod_bw-0.8.tgz
tar xf mod_bw-0.8.tar
cd mod_bw

Build/install the module. We need to make sure the SunStudioExpress binaries are chosen before anything else, so add it to the beginning of PATH.

export PATH=/opt/SunStudioExpress/bin:$PATH

We also need to make a symlink to /usr/ucb/echo (this is a bug with the apxs included in opensolaris, not with mod_bw)

mkdir /usr/ucb
ln -s /usr/bin/echo /usr/ucb/echo

Now cross your fingers and build the module

/usr/apache2/2.2/bin/apxs -i -a -c mod_bw.c

If it complains about .h files, you can find them here:

/usr/apache2/2.2/include
/usr/apr/1.3/include
/usr/apr-util/1.3/include

Once apxs finishes, it should automatically add the following entry to /etc/apache2/2.2/conf.d/modules-32.load

LoadModule bw_module /var/apache2/2.2/libexec/mod_bw.so

Add the following to /etc/apache2/2.2/httpd.conf

BandWidthModule On

BandWidth all 40000

MinBandWidth all 10000

ForceBandWidthModule On

Restart apache2 and check on it’s status

svcadm restart apache22

svcs -a | grep apache22

online 1:55:27 svc:/network/http:apache22

If it’s in maintenance mode, check the log file

/var/svc/log/network-http:apache22.log

Ubuntu 9.10 PXE Boot

Ubuntu 9.10 uses initrd.lz instead of initrd.gz, so PXE booting like you did in previous versions does not work.

This article assumes you’ve already downloaded the iso and have a working PXE boot server.

First, copy the contents of the iso to a directory we can work with

mkdir /tmp/910

mount -o loop ubuntu-9.10-desktop-i386.iso /tmp/910

cp -r /tmp/910 /tftpboot

Convert initrd.lz to initrd.gz

cd /tftpboot/910/casper

mkdir initrd

cp initrd.lz initrd

cd initrd

lzma -dc -S .lz initrd.lz | cpio -id

rm initrd.lz

find . | cpio –quiet –dereference -o -H newc | gzip -9 > initrd.gz

cp initrd.gz /tftpboot/910/casper

Now add the appropriate lines to pxelinux (Example: /tftpboot/pxelinux.cfg/default)

LABEL Ubuntu 9.10 i386 Unmodified Livecd KERNEL 910/casper/vmlinuz APPEND root=/dev/nfs boot=casper netboot=nfs nfsroot=192.168.0.1:/home/tftpboot/910 initrd=910/casper/initrd.gz quiet splash --

Make sure /tftpboot is shared via nfs; in /etc/exports add the following line

/tftpboot/910 192.168.0.0/24(rw,async,no_subtree_check)

Reload the NFS server

/etc/init.d/nfs-kernel-server reload

OpenSolaris LDAP Client to Linux OpenLDAP Server

The following outlines how to set up a OpenSolaris client to work with a Linux OpenLDAP server.

The following is one line

ldapclient manual -a credentialLevel=proxy -a authenticationMethod=simple -a proxyDN=cn=admin,dc=server -a proxyPassword=yourpassword -a defaultSearchBase=dc=server -a defaultServerList=192.168.1.1:389 -a serviceSearchDescriptor=passwd:ou=People,dc=server -a
serviceSearchDescriptor=group:ou=Groups,dc=server

Add the following lines to their appropriate sections in /etc/pam.conf

login   auth required           pam_ldap.so.1
other   auth required           pam_ldap.so.1
passwd auth required         pam_ldap.so.1

Test LDAP with

ldaplist -l passwd

Test user authentication using the following steps

mkdir /export/home/ldapuser

chown ldapuser /export/home/ldapuser

Edit /etc/auto_home and add the following BEFORE “+auto_home”

ldapuser localhost:/export/home/ldapuser

SSH to test

ssh ldapuser@localhost

You should be in! If not, you may have to change the following in /etc/ssh/sshd_config

PAMAuthenticationViaKBDInt no

#Change from yes to no

Restart SSH

svcadm restart ssh

If you are still unable to log in, your userPassword attribute probably needs to be changed from md5/md5crypt to crypt.

su – ldapuser

passwd

After changing your password you should be able to log in.

OpenVZ on Ubuntu 8.10

Other OpenVZ Ubuntu 8.10 guides that I’ve seen have you use the old lenny repository; since this no longer exists let’s try another way!

I used a 64-bit machine; steps will be the same for 32-bit. If you have 64-bit and just want to download the kernel I used: http://bbis.us/~will/openvz-ubuntu810.tar.gz

Now let’s build the kernel.

cd /usr/src
wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.27.tar.bz2
tar xjf linux-2.6.27.tar.bz2
cd linux-2.6.27

wget http://download.openvz.org/kernel/branches/2.6.27/2.6.27-briullov.1/patches/patch-briullov.1-combined.gz
gunzip patch-briullov.1-combined.gz

wget http://download.openvz.org/kernel/branches/2.6.27/2.6.27-briullov.1/configs/kernel-2.6.27-x86_64.config.ovz
#OR
wget http://download.openvz.org/kernel/branches/2.6.27/2.6.27-briullov.1/configs/kernel-2.6.27-i686.config.ovz

cp kernel-2.6.27-*.config.ovz .config
patch -p1 < patch*

make oldconfig

make
make modules_install install

cd /boot
mkinitramfs -o /boot/initrd.img-2.6.27 2.6.27

vi /boot/grub/menu.lst

#Replace uuid entries with your disks uuid
title           Ubuntu 8.10 OpenVZ 2.6.27
uuid            2d5b2466-4bdf-44c7-b8e5-4d46a9f927c8
kernel          /boot/vmlinuz-2.6.27 root=UUID=2d5b2466-4bdf-44c7-b8e5-4d46a9f927c8 ro quiet splash
initrd          /boot/initrd.img-2.6.27
quiet

grub-install /dev/sda

apt-get install vzctl vzquota

reboot

If everything went according to plan, you’ll boot into your new kernel. If you get a kernel panic, or things are not working as expected, you may have to

make menuconfig

Once the kernel is working, let’s create a container.

cd /var/lib/vz/private

mkdir 1

debootstrap hardy 1

When you see:

#I: Base system installed successfully.

You can continue.

vi /etc/vz/dists/default
#Change redhat to debian OR copy ubuntu.conf to default

Create the container and set an IP

vzctl set 1 –applyconfig vps.basic –save
vzctl set 1 –ipadd 192.168.100.7 –save
vzctl set 1 –nameserver 192.168.100.1 –save

vzctl start 1

At this time you cannot enter the container. You will receive the error

Unable to open pty: No such file or directory

To fix this:

vzctl exec 1 update-rc.d -f udev remove

Restart and enter the container

vzctl restart 1

vzctl enter 1

If network doesn’t work add to /etc/sysctl.conf on server (not container):

net.ipv4.conf.default.forwarding=1
net.ipv4.conf.all.forwarding=1

Ubuntu 8.10 LDAP Server with TLS

This should also work with Ubuntu 9.04

apt-get install slapd ldap-utils libnss-ldap libpam-ldap migrationtools

Much like Debian, we must perform an extra step to make our changes permanent.

dpkg-reconfigure slapd

Omit OpenLDAP server configuration? No DNS domain name: your-server Organization name: whatever Database backend to use: HDB Do you want the database to be removed when slapd is purged? No Move old database? Yes Admin password: <password> Confirm: <password> Allow LDAPv2 protocol: No

Now we will populate LDAP. You should probably add a temporary LDAP user for testing. After we populate LDAP we’ll remove it.

adduser test cd /usr/share/migrationtools

Edit migrate_common.ph and replace “padl.com” with your domain and dc=padl,dc=com with dc=YOUR,dc=DOMAIN

./migrate_base.pl > /tmp/base.ldif
./migrate_passwd.pl /etc/passwd /tmp/passwd.ldif
./migrate_group.pl /etc/group /tmp/group.ldif

Remove the top section of /tmp/base.ldif

dn: dc=dev dc: dev objectClass: top objectClass: domain

Restart LDAP and add our information to LDAP

/etc/init.d/slapd restart
ldapadd -x -W -D 'cn=admin,dc=example,dc=net' < /tmp/base.ldif ldapadd -x -W -D 'cn=admin,dc=example,dc=net' < /tmp/passwd.ldif ldapadd -x -W -D 'cn=admin,dc=example,dc=net' < /tmp/group.ldif

To enable PAM I suggest

cd /tmp wget http://st0rage.org/files/pam.d.tar tar xf pam.d.tar -C /etc

Alternativly, you can try the following Ubuntu specific method:

auth-client-config -t nss -p lac_ldap pam-auth-update ldap

Please note that I was unable to change user passwords with ‘passwd’ using this method.

Remove the test user

userdel test

Set up TLS Certificate

mkdir /etc/ldap/ssl cd /etc/ldap/ssl
openssl req -new -nodes -out req.pem -keyout key.pem #No challenge password, leave empty openssl rsa -in key.pem -out new.key.pem openssl x509 -in req.pem -out ca-cert -req -signkey new.key.pem -days 9999 mv new.key.pem server.pem cat ca-cert >> server.pem

Enable TLS on server

ldapmodify -x -D cn=admin,cn=config -W

Paste in the following:

dn: cn=config add: olcTLSCACertificateFile olcTLSCACertificateFile: /etc/ldap/ssl/server.pem - add: olcTLSCertificateFile olcTLSCertificateFile: /etc/ldap/ssl/server.pem - add: olcTLSCertificateKeyFile olcTLSCertificateKeyFile: /etc/ldap/ssl/server.pem

Press ENTER a few times to finish modifying and then Control+C when you see:

modifying entry "cn=config"

Edit /etc/default/slapd and add

SLAPD_SERVICES="ldap:/// ldaps:/// ldapi:///"

Restart slapd

/etc/init.d/slapd restart

If it fails, try:

chown -R openldap /etc/ldap/ssl /etc/init.d/slapd restart

Client configuration

vi /etc/ldap.conf
uri ldaps://server port 636 ssl start_tls ssl on tls_checkpeer no tls_cacertfile /etc/ldap/ssl/server.pem

vi /etc/nsswitch.conf
passwd: compat ldap group: compat ldap shadow: compat ldap hosts: files dns networks: files protocols: db files services: db files ethers: db files rpc: db files netgroup: ldap

Now test! Make sure to stop nscd if you’re using it

/etc/init.d/nscd stop

Test LDAP

$ id test uid=1001(test) gid=1001(test) groups=1001(test)

Remember to keep an eye on /var/log/auth.log if you run into any problems

Apache2 hangs with ‘Digest: generating secret for digest authentication’

I’m running Apache2+SSL inside a libvirtd (kvm) Virtual Machine.

After enabling SSL and trying to add another SSL site, apache should refuse to restart and the following error would show up in /var/log/apache2.log

Digest: generating secret for digest authentication

After doing some research, it turns out that the VM did not have enough entropy to generate much of anything. Increasing this is easy, but may not be completely secure.

# cat /proc/sys/kernel/random/entropy_avail
139

# apt-get install rng-tools

# rngd -r /dev/urandom -o /dev/random

# cat /proc/sys/kernel/random/entropy_avail
2220

After running rngd, the entropy will increase at a gradual rate.

If you want this to survive a reboot, you’ll need to put it in a startup script.

Apache2 Bandwidth Limiting in Ubuntu Hardy 8.04

Unfortunately there is no libapache2-mod-bw for Ubuntu, so limiting bandwidth used by apache isn’t possible by default. Follow these steps to enable it.

wget http://ivn.cl/files/source/mod_bw-0.8.tgz

tar xvzf mod_bw-0.8.tgz

cd mod_bw

apxs2 -i -a -c mod_bw.c

You will probably get this error

apxs:Error: Activation failed for custom /etc/apache2/httpd.conf file..
apxs:Error: At least one `LoadModule’ directive already has to exist..

It’s safe to ignore this.

vi /etc/apache2/mods-available/mod-bw.load

LoadModule bw_module /usr/lib/apache2/modules/mod_bw.so

vi /etc/apache2/mods-available/mod-bw.conf

BandWidthModule On
BandWidth all 40000
MinBandWidth all 10000
ForceBandWidthModule On

Set permissions

chown www-data /etc/apache2/mods-available/mod-bw*

Activate the module

a2enmod mod-bw

(To deactivate use: a2dismod mod-bw)

Restart apache2

/etc/init.d/apache2 force-reload

Download a file (I suggest doing it remotely to test it), and you should be limited to around 40 kilobytes/second total, with each user guaranteed 10 kilobytes/second. Change this number accordingly.

pine using imap/postfix on Ubuntu/Debian

If you’d like to use pine on a Ubuntu or Debian server, chances are your current email system isn’t using the old /var/mail/user way of doing things. 🙂

apt-get install alpine

ln -s /usr/bin/alpine /usr/bin/pine

Run pine to generate a ~/.pinerc file, and change the following lines

user-domain=your-domain.com

inbox-path={your-domain.com:143}INBOX

smtp-server=your-domain.com/novalidate-cert/user=$USER

If you don’t use authenticated SMTP, just comment out smtp-server

You should probably copy this file to everyone’s home directory. Make sure to update permissions!

cd /home

for x in *; do

mv /home/$x/.pinerc /home/$x/.pinerc.bak

cp /etc/skel/.pinerc /home/$x/

chown $x /home/$x/.pinerc

done