bsd.b3ta.org
Yet another *nix admin blog

I walked through routine maintenance of one of FreeBSD boxes, which was running as reverse proxy (Apache httpd + mod_security2 + mod_proxy) for OWA in my office. Everything went smoothly. But when it came to restarting of services, apache started to complain of unresolved symbol!

Cannot load /usr/local/libexec/apache22/mod_security2.so into server: /usr/local/libexec/apache22/mod_security2.so: undefined symbol: xmlFree

I reinstalled apache22, libxml2 and mod_security2 from ports. Nothing unusual was sighted. Asking our friend, google didn’t provide much of information. The latest that I managed to dig out was in 2005 and 2006. Some dudes had the same issue as me.

Anyhow I’m pretty convinced it is a bug from mod_security2-2.1.7_1 in FreeBSD’s ports. Here a quick workaround, just add the lines below to your Apache httpd.conf and the error message will go away!

LoadFile /usr/local/lib/libxml2.so
LoadModule security2_module libexec/apache22/mod_security2.so

Here’s the note with easy steps to get pound running with SSL signed by CA.

Generating Certificate Signing Request
# cd /etc/ssl
# openssl req -new -nodes -subj '/C=MY/ST=Wilayah Persekutuan/L=Kuala Lumpur/CN=myshinny.webserver.com/O=My office./OU=IT department.' -key host.key -out host.csr

After generating certificate signing request, you need to copy and paste the contain of host.csr to Verisign for signing. Once you have got your certificate signed, save it as host.crt. Note: the naming convention here is for the demonstration below.

Obtaining Verisign intermediate CA certificate
Depending on which type of certificate that you have purchased, you could obtain Verisign CA certificate from this page. Copy the certificate content and save it as verisign.pem.

Now you have 4 files: host.key, host.csr, host.crt and verisign.pem. Only 3 of them are needed for pound ssl. Prepare the certificate to use with pound. Note: In server.pem that will be created, it is important that you follow the sequence as such.

1 Your key
2 Your certificate
3 CA certificate

# cat host.key host.crt verisign.pem > server.pem

Example pound configuration, pound.cfg:-

---snip---
ListenHTTPS
        Address x.x.x.x
        Port    443
        HeadRemove "X-SSL-.*"
        HeadRemove "X-Client-Verify.*"
        Cert    "/etc/ssl/server.pem"
        CAlist "/etc/ssl/verisign.pem"
        Ciphers "ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL"
---snip---

End

Replace x.x.x.x with your server IP address. Restart pound and you are done!

On my previous post, I encountered problem with ssl-cert in chrooted environment. By installing ssl-cert via apt-get, these processes were triggered.

root 27799 0.0 0.4 4676 2256 pts/2 Ss+ 04:33 0:00 /usr/bin/dpkg --status-fd 13 --configure ssl-cert
root 27800 0.4 1.3 10016 7132 pts/2 S+ 04:33 0:00 /usr/bin/perl -w /usr/share/debconf/frontend /var/lib/dpkg/info/ssl-cert.postinst configure
root 27806 0.0 0.2 3804 1192 pts/2 S+ 04:33 0:00 /bin/sh -e /var/lib/dpkg/info/ssl-cert.postinst configure
root 27808 0.0 0.2 3820 1280 pts/2 S+ 04:33 0:00 /bin/bash -e /usr/sbin/make-ssl-cert generate-default-snakeoil
root 27812 0.0 0.2 3780 1448 pts/2 S+ 04:33 0:00 openssl req -config /tmp/tmp.OXerK27810 -new -x509 -days 3650 -nodes -out /etc/ssl/certs/ssl-cert-snakeoil.pem -keyout /etc/ssl/private/ssl-cert-snakeoil.key


Further investigation showed that process 27812 stuck. Running the openssl command manually brings up:

27830:error:0E065068:configuration file routines:STR_COPY:variable has no value:conf_def.c:629:line 5


The error message is exactly same as what Michael Prokop has found out.

The content of /tmp/tmp.OXerK27810 is:

#
# SSLeay example configuration file.
#

[ req_distinguished_name ]
commonName = myshinnybox


DIRTY SOLUTION
As I’m only interested to get packages such as Postfix, Postgresql Apache mod SSL to install, killed the “apt-get install ssl-cert” process and fired up vi and removed the line RANDFILE = $ENV::RANDFILE from /tmp/tmp.OXerK27810.


# openssl req -config /tmp/tmp.OXerK27810 -new -x509 -days 3650 -nodes -out /etc/ssl/certs/ssl-cert-snakeoil.pem -keyout /etc/ssl/private/ssl-cert-snakeoil.key
Generating a 1024 bit RSA private key
................................................++++++
.......++++++
writing new private key to '/etc/ssl/private/ssl-cert-snakeoil.key'
-----
# apt-get install ssl-cert
Reading package lists... Done
Building dependency tree
Reading state information... Done
ssl-cert is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0B of additional disk space will be used.
Setting up ssl-cert (1.0.15) ...
#


I don’t understand why the package ssl-cert exist in the first place when user can just use “openssl” command to generate ssl certificate. Yet making the situation worst, ssl-cert was added as dependency to many packages. Duh! In additional, chroot seems to be buggy under Linux.

In my previous post, FreeBSD : Compaq Presario V3417AU, I could not get broadcom wifi running with FreeBSD 7. Luckily, just before the Chinese New Year, I managed to get it working again. It’s timely for the CNY break. Sweet!!!!!

Project Evil

# fetch ftp://ftp.hp.com/pub/softpaq/sp34001-34500/sp34152.exe
# cabextract -F ‘bcmwl5*’ sp34152.exe
# ndisgen bcmwl5.inf bcmwl5.sys
# cp bcmwl5_sys.ko /boot/modules/
# kldxref /boot/modules
# kldload bcmwl5_sys

Using ndis0 with wpa_supplicant

Set up /etc/wpa_supplicant.conf with your WIFI network information (an example of WEP protected WLAN with DHCP enabled). You may enable wpa_supplicant automatically by setting /etc/rc.conf with ifconfig_ndis0="WPA mywifi DHCP". To initialise it, run /etc/rc.d/netif start ndis0

Additional configurations

i) /boot/loader.conf

if_ndis_load="YES"
bcmwl5_sys_load=”YES”
wlan_scan_sta_load=”YES”
wlan_scan_ap_load=”YES”
wlan_wep_load=”YES”
wlan_ccmp_load=”YES”
wlan_tkip_load=”YES”
wlan_xauth_load=”YES”
wlan_acl_load=”YES

ii) /etc/rc.conf

ifconfig_ndis0="WPA mywifi DHCP"

iii) /etc/wpa_supplicant.conf

ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=wheel
ap_scan=1
network={
ssid=”mywifi”
scan_ssid=1
key_mgmt=NONE
wep_tx_keyidx=1
wep_key1=your_104bit_wep_key
}

This a just a short note for my own reference.

1) Install postfix
# yum install postfix

2) Set default MTA to Postix via this command
# alternatives –set mta /usr/sbin/sendmail.postfix

3) Autostart Postfix (Optional but you really want to do it)
# chkconfig –levels 235 sendmail off
# chkconfig –levels 235 postfix on
# /etc/init.d/sendmail stop
# /etc/init.d/postfix start

If you are on FreeBSD 7.0 and have problem booting up HeX LiveCD with qemu, try this

kldload aio

and add -std-vga option to qemu command line. For example:-

qemu -boot d -cdrom hex-i386-1.0.2.iso -localtime -std-vga

Have fun!

With the release of OpenBSD 4.2, you will find that cdrom42.fs was not provided in OpenBSD official ftp sites. However, it is relatively easy to custom build your own OpenBSD 4.2 bootable installer CD. I will show you the steps in making your own puffer fish El Torito.

CREATE CD STRUCTURE
Create the OpenBSD bootable CD structure with this command,

%mkdir -p ~/OpenBSD/4.2/i386

DOWNLOAD OPENBSD FILES
Use ncftp or wget to download the necessary files off OpenBSD ftp site.

%cd ~/OpenBSD/4.2/i386 && ncftp ftp://ftp.jp.openbsd.org/pub/OpenBSD/4.2/i386
ncftp /OpenBSD/4.2/i386 > get *

Note : install42.iso is a bootable OpenBSD installer by itself. You should exclude that file.

CREATE CDROM42.FS
As this file is absent, creation of cdrom42.fs is required in order to make bootable OpenBSD iso. Bootable “El Torito” CDROMs usually use a boot loader. The loader will boot disk image located inside the iso9660 filesystem. This cdrom42.fs is the file that contains both boot loader and disk image. Not too worry. It is trivial.

Thanks to Rainer Krienke for creating a nice El Torito boot image extractor in PERL, called “geteltorito“. Grab a copy/make executable and extract El Torito boot image from the file cdemu42.iso with this simple command.

%geteltorito cdemu42.iso > cdrom42.fs
Booting catalog starts at sector: 29
Manufacturer of CD: Copyright (c) 2007 Theo
Image architecture: x86
Boot media type is: 2.88meg floppy
El Torito image starts at sector 30 and has 5760 sector(s) of 512 Bytes
Image has been written to stdout ....

CUSTOMIZATION
You can add whatever files you want them to be included. Just copy them to ~/OpenBSD/ . I normally will put stuff like ports.tar.gz, src.tar.gz, sys.tar.gz and etc.

CREATE OPENBSD BOOTABLE INSTALLER CD
mkisofs comes handy in creating our bootable CD. Just issue this command and wait for it to be generated.

%cd ~/OpenBSD && mkisofs -vrTJV "OpenBSD 4.2" -b 4.2/i386/cdrom42.fs -c boot.catalog
-o OpenBSD42.iso ~/OpenBSD/

Now you will have OpenBSD42.iso. Burn it to a blank CD-R and Volia! Do support the OpenBSD project. Buy CD/T-shirt. They look really cool!!

Our (xwings and I) poor, _soon_to_be_removed_ webserver box at Brickfields netmyne datacentre was under DDOS attacked this morning around 11:30am MYT. The box with 512MB of RAM running FreeBSD survived the attack. However, system resources were running extremely low. More than 100 instances of httpd were running! The system was extremely busy and lag. It barely responded to my ssh connection request.

My pf rule’s stateful tracking limit seemed to be too lenient and not suitable for the box with low system resource.

pass in quick on $netif inet proto tcp from any to ($netif) port {80, 443}
   \ keep state(source-track, max-src-states 100, max-src-nodes 999}

I have decided to head on to more aggressive approach, with use of pf max-src-conn-rate, table and filtering.

table  persist
block in quick on $netif from 
pass in quick on $netif inet proto tcp from any to ($netif) port {80, 443}
   \ keep state(max-src-conn 100, max-src-rate 15/5, overload  flush}

After stopping all httpd instances, I loaded new pf rule. pfctl -f /etc/pf.conf. It worked nicely. table was full of sons/daughters of bitch’s IP addresses.

pfctl -t dos -T show
221.194.136.38
220.181.19.176
72.232.190.82
61.135.162.18
202.190.250.2
64.26.63.19


Happy and back to sleep again. Grow up, kids! Shame on you. You couldn’t even kill a poor little box with 512MB of RAM! But thanks anyway for helping me to test area where I overlooked.

White Papers for Success
Decisions related to web hosting should be taken after going through the current web hosting review. Having an old record of past webhosting review magazines might help a bit. For services like ix web hosting and powweb however, one does not need to go through reviews.

Another way of fighting images spam is by Realtime Black List lookup. This tactic is probably one of method which is inexpensive to server resource.

With postfix, you could just add reject_rbl_client images.rbl.msrbl.net to smtpd_client_restrictions section of postfix’s main.cf.

For instance:

smtpd_client_restrictions = permit_mynetworks,
    reject_rbl_client images.rbl.msrbl.net,
    ….

For qmail, you could just add -r images.rbl.msrbl.net as tcpserver option of your qmail smtp startup script.

Note : images.rbl.msrbl.net - Hosts found sending mail contaning spam images. Check out MSRBL for more info.

On my previous post, I have discussed some of the anti image/PDF spam. I have tried clamav with Sanesecurity’s phishing and scam signatures.

On FreeBSD, I downloaded update shell script by Dan Larsson and made a slight modification as I do not wish to install/use rsync on production servers just to download signature files. I have added these two lines to update shell script under “http_source_urls” and commented out “rsync_source_urls“.

http://download.mirror.msrbl.com/MSRBL-Images.hdb

http://download.mirror.msrbl.com/MSRBL-SPAM.ndb

http_source_urls="
   http://www.sanesecurity.com/clamav/phishsigs/phish.ndb.gz
   http://www.sanesecurity.com/clamav/scamsigs/scam.ndb.gz
   http://clamav.securiteinfo.com/vx.hdb.gz
   http://download.mirror.msrbl.com/MSRBL-SPAM.ndb
   http://download.mirror.msrbl.com/MSRBL-Images.hdb
   http://www.malware.com.br/cgi/submit?action=list_clamav,fetch_interval=86400,target_file=mbl.db

"#rsync_source_urls="
#   rsync://rsync.mirror.msrbl.com/msrbl/MSRBL-Images.hdb
#   rsync://rsync.mirror.msrbl.com/msrbl/MSRBL-SPAM.ndb
#"

(more…)