When I usually like to do is to echo -n a description of the current task and then later echo either “Done.” or “Failed!” (or something similar).
Obviously, that somewhat conflicts with pv. So here is the solution:

# Create backup archive
_statusMessage="Compressing installation..."
echo -n $_statusMessage
if hash pv 2>&- && hash gzip 2>&- && hash du 2>&-; then
  echo
  _folderSize=`du --summarize --bytes $BASE | cut --fields 1`
  if ! tar --create --file - $BASE | pv --progress --rate --bytes --size $_folderSize | gzip --best > $FILE; then
    echo "Failed!"
    exit 1
  fi
  # Clear pv output and position cursor after status message
  tput cuu 2 && tput cuf ${#_statusMessage} && tput ed
else
  if ! tar --create --gzip --file $FILE $BASE; then
    echo "Failed!"
    exit 1
  fi
fi

echo "Done."

The snippet also shows how to conditionally use pv if it is available, which I really like.

Upon failing to find what I wanted online, I constructed this beauty:

find . \( -exec test -w {} \; -o \( -exec echo {} \; -quit \) \) | xargs -I {} bash -c "if [ -n "{}" ]; then echo {} is not writeable\!; exit 1; fi"

This will print the name of the item that is not writeable in addition to the non-zero return value needed when used in more complex scripts.

If you don’t want/need the output, you can go with the simpler version:

find . \( -exec test -w {} \; -o \( -exec echo {} \; -quit \) \) | xargs -I {} test -z "{}"


If there is a more simple solution, please let me know.

So, here is the revised approach for a bash script that has the ability to update itself:

#!/bin/bash

set -o nounset
set -o errexit

SELF=$(basename $0)

# The base location from where to retrieve new versions of this script
UPDATE_BASE=http://typo3scripts.googlecode.com/svn/trunk

# Self-update
runSelfUpdate() {
  echo "Performing self-update..."

  # Download new version
  echo -n "Downloading latest version..."
  if ! wget --quiet --output-document="$0.tmp" $UPDATE_BASE/$SELF ; then
    echo "Failed: Error while trying to wget new version!"
    echo "File requested: $UPDATE_BASE/$SELF"
    exit 1
  fi
  echo "Done."

  # Copy over modes from old version
  OCTAL_MODE=$(stat -c '%a' $SELF)
  if ! chmod $OCTAL_MODE "$0.tmp" ; then
    echo "Failed: Error while trying to set mode on $0.tmp."
    exit 1
  fi

  # Spawn update script
  cat > updateScript.sh << EOF
#!/bin/bash
# Overwrite old file with new
if mv "$0.tmp" "$0"; then
  echo "Done. Update complete."
  rm \$0
else
  echo "Failed!"
fi
EOF

  echo -n "Inserting update process..."
  exec /bin/bash updateScript.sh
}

# Update check
SUM_LATEST=$(curl $UPDATE_BASE/versions 2>&1 | grep $SELF | awk '{print $1}')
SUM_SELF=$(md5sum $0 | awk '{print $1}')
if [[ "$SUM_LATEST" != "$SUM_SELF" ]]; then
  echo "NOTE: New version available!"
fi

It also includes the update check which wasn’t discussed over at StackOverflow. In case I revise the approach yet again, you’ll find the latest version over at the typo3scripts project page.

Cannot load certificate file error:0906D06C:
PEM routines:PEM_read_bio:no start line: error:140AD009:
SSL routines:SSL_CTX_use_certificate_file:PEM lib

All research regarding the matter pointed to a broken certificate. But even after re-exporting it and comparing the contents, we could not find any problem. I had the client repeat several parts of the certificate over the phone to my while comparing them to the one I had on the server. Everything matched exactly.

So I had the client send me the certificate and after a while I found the problem. The certificate the client had was saved as UTF8. If it was the BOM or the encoding of some characters, I have no idea. But after saving it as ANSI/ASCII the problem went away and the client was able to connect.

tl;dr: UTF8 is bad for OpenVPN certificates.

So first thing we want to do is remove the vesa framebuffer module from the modprobe blacklist.

sudo editor /etc/modprobe.d/blacklist-framebuffer

Comment out or delete the following line:

blacklist vesafb

Now to enable the needed modules:

sudo editor /etc/initramfs-tools/modules

And add the following lines at the end:

fbcon
vesafb

Finally, update your initramfs images:

sudo update-initramfs -u

Now, we need to make Grub pass the required parameters to the kernel to enable the high-res console.
In case of Ubuntu 8.04 (and 8.10 I guess) or any pre-Grub2 version:

sudo editor /boot/grub/menu.lst

For Grub2 installations:

sudo editor /etc/default/grub

There will be a line like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

You’re gonna want to add the vga parameter to that:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vga=791"

Or as the original article suggests (and what I prefer as well):

GRUB_CMDLINE_LINUX_DEFAULT="verbose vga=791"

But that only changes the amount of information you get during boot time.
The number that is passed as an argument with vga signals the desired resolution and color depth. 791 is 1024×768 with 64K colors. A full list can be found here.
Now, for Grub2 installations you’ll want to update Grub:

sudo update-grub2

And then, reboot!

sudo reboot now

Well, I ignored those people and did it anyway. And what did I, shortly after, get in return? A failed RAID volume. Yay.

One day I simply shut off my computer for a little restructuring of my work area, and after I turned it on again, it told me it can’t boot, cause, well, yeah. In fact, it reported that one of the RAID disks was no longer a member of the array. So I rebooted multiple times, shut off the computer and power supply, booted back up. The volume remained failed.

So I booted into an Ubuntu Live CD and browsed the web for solutions. Cause I was certain no data could have actually been lost. The RAID controller was just stupid. The solutions I found were mostly along the lines of “re-create the volume and restore a backup”. Of course I had no backup.
So I thought I would just back up the drives contents now and try to simply re-create the volume and write the data back.

Using dd, I did exactly that:

dd if=/dev/sda of=/media/backup/sda.dd bs=2MB
dd if=/dev/sdb of=/media/backup/sdb.dd bs=2MB

This took the whole night of course. So, the next day I rebooted to re-create the RAID volume. But I was very surprised to find that the RAID controller reported the volume to be in perfect health. And surely enough, seconds later my Windows was up and running again.
I praised the gods, instantly made backups and went on with my life.

Until yesterday. When the same shit happened AGAIN. I was minding my own business, when suddenly my computer shut off (due to overheating I assume). When I started it back up again, it, again, reported the RAID volume as failed.
Again, multiple reboots didn’t do anything. And, again, I went into a Live CD to back up the RAID member disks.
And, AGAIN, today my volume comes up as healthy as ever.

I have no clue if there is any logic behind this or if this was just magic happening. I also have no idea if my actions actually had anything to do with this RAID recovery. But, sure as hell, when I come across a failed ICH9R RAID0 volume again, I’ll dd the disks and reboot the shit out of it.

And, also, I’ll be converting back to a non-RAID setup ASAP.

First, you wanna fire up your old friend Wireshark. Now set up the device you want to emulate to send the data to the IP of your machine. Now you can start a Live Capture in Wireshark (Ctrl+E). Now you command your device to send the message by whatever means needed (like pressing the correct button on the device). Now stop the Live Capture in Wireshark (Ctrl+E) and add a filter to get the packet you need.

Let’s assume the devices IP address is 10.0.0.1 and your IP address is 10.0.0.2. Now the correct filter would be

ip.src == 10.0.0.1 && ip.dst == 10.0.0.2

Now, hopefully you’ll see a single packet. In the center panel of Wireshark there should be a Data segment. This is what we want. Right-click it and select Copy -> Bytes (Hex Stream). You’ll get something like:

4e656574732c4b2c0100

(In case you’re interested, this is the payload when you press the first button on a Neets Control EU Standard device.)

Now let’s put this into code. The below example already has 2 payloads that would signal the target device to be turned on or off. For this 2 packets have been captured and can now be selected via the command line. The implementation is straight-forward.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net.Sockets;

namespace Neets_Control {
  class Program {

    private static byte[] WORKLOAD_ON   = new byte[] { 0x4e, 0x65, 0x65, 0x74, 0x73, 0x2c, 0x4b, 0x2c, 0x01, 0x00 };
    private static byte[] WORKLOAD_OFF  = new byte[] { 0x4e, 0x65, 0x65, 0x74, 0x73, 0x2c, 0x4b, 0x2c, 0x02, 0x00 };

    private static string TARGET_HOST   = "10.11.110.11";
    private static int    TARGET_PORT   = 5009;

    static void Main( string[] args ) {

      TcpClient sender = new TcpClient( TARGET_HOST, TARGET_PORT );
      NetworkStream stream = sender.GetStream();
      if( args[ 0 ] == "on" ) {
        Console.WriteLine( "Switching on" );
        stream.Write( WORKLOAD_ON, 0, WORKLOAD_ON.Length );

      } else if( args[ 0 ] == "off" ) {
        Console.WriteLine( "Switching off" );
        stream.Write( WORKLOAD_OFF, 0, WORKLOAD_OFF.Length );

      } else {
        Console.WriteLine( "Missing parameter (on/off)." );
      }

      stream.Close();
      sender.Close();

    }
  }
}

So, yeah, I can now just refer back to my introduction. Possibly this wasn’t event worth mentioning ;)

I start with a clean Ubuntu 8.04 install. So, first things first:

sudo apt-get update
sudo apt-get dist-upgrade
sudo shutdown -r now

Now that we’re up-to-date, let’s begin:

sudo apt-get build-dep php5
sudo apt-get install libmcrypt-dev libc-client-dev checkinstall

Now we have all the crap to build PHP with. Let’s get PHP itself:

cd /tmp/
wget http://us3.php.net/get/php-5.3.0.tar.gz/from/this/mirror
tar xvzf php-5.3.0.tar.gz
cd php-5.3.0/

Now all that is left is to build PHP…

./configure --prefix=/usr --with-apxs2=/usr/bin/apxs2 --with-config-file-path=/etc/php5/apache2 --with-config-file-scan-dir=/etc/php5/apache2/conf.d --mandir=/usr/share/man --disable-debug --with-regex=php --disable-rpath --disable-static --with-pic --with-layout=GNU --with-pear=/usr/share/php --enable-calendar --enable-sysvsem --enable-sysvshm --enable-sysvmsg --enable-bcmath --with-bz2 --enable-ctype --with-db4 --without-gdbm --with-iconv --enable-exif --enable-ftp --with-gettext --enable-mbstring --with-pcre-regex=/usr --enable-shmop --enable-sockets --enable-wddx --with-libxml-dir=/usr --with-zlib --with-kerberos=/usr --with-openssl=/usr --enable-soap --enable-zip --with-exec-dir=/usr/lib/php5/libexec --without-mm --with-curl=shared,/usr --with-zlib-dir=/usr --with-gd=shared,/usr --enable-gd-native-ttf --with-gmp=shared,/usr --with-jpeg-dir=shared,/usr --with-xpm-dir=shared,/usr/X11R6 --with-png-dir=shared,/usr --with-freetype-dir=shared,/usr --with-t1lib=shared,/usr --with-ldap=shared,/usr --with-ldap-sasl=/usr --with-mhash=shared,/usr --with-mysql=shared,/usr --with-mysqli=/usr/bin/mysql_config --with-pspell=shared,/usr --with-unixODBC=shared,/usr --with-xsl=shared,/usr --with-snmp=shared,/usr --with-sqlite=shared,/usr --with-mssql=shared,/usr --with-tidy=shared,/usr --with-xmlrpc=shared --with-pgsql=shared,/usr --enable-gd-native-ttf --enable-dba=shared --with-openssl-dir=shared,/usr --enable-gd-jis-conv --enable-json --with-mcrypt=shared,/usr --enable-pcntl --with-pdo-mysql --with-pdo-odbc=unixODBC,/usr --with-pdo-pgsql=shared,/usr --with-pdo-sqlite --enable-xmlreader --with-tsrm-pthreads --with-imap --with-imap-ssl

I’m sure this doesn’t enable ALL features in PHP, but it works for me so far. If I’ll ever find I was missing something, I’ll be sure to update this post.

When building make sure to pass the correct number of CPUs to make with the -j parameter to safe some time.

make -j2

And now we wait…

Now we need to add these lines to our httpd.conf (which is empty in this case)

#LoadModule directive to aid module installations
#LoadModule dummy_module /usr/lib/apache2/modules/mod_dummy.so

Now we can build our .deb for PHP.

sudo checkinstall -D --install=no --fstrans=no --maintainer=your@email --reset-uids=yes --nodoc --pkgname=php5 --pkgversion=5.3 --pkgrelease=200908060830 --arch=amd64

And that’s it. Now you have a .deb which you can install on a different system or (as checkinstall told you) just put on this system with dpkg -i.

Have fun

The first thing we’ll need is the screen package.

sudo apt-get install screen

Now let’s create a user for steam and switch to it.

sudo adduser --disabled-login steam
sudo su - steam

Let’s download gSteam, my frontend, and set it executable for only steam.

wget http://www.dirty-motherfucker.org/steam/gSteam.sh
chmod 700 gSteam.sh

Now we can install the steam client.

./gSteam.sh install steam

As noted in my post regarding the L4D server, if the installation does not seem to properly start or
hang, you can try to create a Steam.cfg file (in the homedir) with preferred content server ids.

http://www.dirty-motherfucker.org/blog/2008/12/26/running-a-source-dedicated-server-for-left-4-dead/

After you get the magic line “Steam Linux Client updated, please retry the command” you are ready to
install some games. So let’s install Team Fortress 2.

./gSteam.sh install tf

The identifiers for the games can be obtained with:

./steam -command list

The “No installation record found at…” messages can be safely ignored.
Now you will have to wait until the game is fully downloaded. This can take from “forever” to
“until the fucking end of time” in some cases. The old Steam.cfg can help here as well, but
I personally found that patience is the best tool.

Once the installation is completed a profile template for tf will be created. gSteam will try its
best to determine some default parameters. You should edit it nevertheless to at least adjust
the name of your server (and possibly the profile).

If everything worked fine you can now simply

./gSteam start tf2

and the thing should be running.

Installing other games is as simple as:

./gSteam install l4d_full

Let me just add that the script is far from fail-proof. Especially error-cases aren’t properly handled. But it saved me a lot of time so I thought I’d share it.
Hope it helps someone else ;)

As all of my administration articles (so far) this one is for Ubuntu Server 8.04.1.

Let’s start by adding a new user that will run our servers.

sudo adduser --disabled-login steam

Now to log in with the newly created user to install the servers. We need to switch the user as root as a normal user can not log into this account.

sudo su - steam

No we can install the HLDSUpdateTool.

wget http://www.steampowered.com/download/hldsupdatetool.bin
chmod +x hldsupdatetool.bin
./hldsupdatetool.bin
./steam

In case it will hang at “Checking bootstrapper version…” you can try to set a set of preferred content servers. You do that by creating a file called “steam.cfg” and adding the following line to it:

PreferredContentServerIDs = "id id id"

You can find the ids on http://store.steampowered.com/stats/content/ under “Individual Server Statisistics”. Look for Valve #NN servers and place the number of the servers you want to use inside the quotes. Then retry running steam.

Now we’ll install a game. Let’s start out with Left 4 Dead. If you want to see what games are available, run:

./steam -command list

This will also show you the correct game identifier for Left 4 Dead, which is l4d_full (left4dead was for the demo).

./steam -command update -game l4d_full -dir /home/steam

In case the installation process hangs you can also try setting the preferred content servers as mentioned above. Now, theoretically you’re good to go and you can start your server. But we’re gonna go a step further. I use a start script based on the article mentioned at the start of this article. Now let’s grab it.
But wait, in case you realize that you really don’t want to use any of my shitty scripts and that you just want to get a clean server running, in that case it can stilll be helpful to get the script. You see when you use my script and you start a profile it will print out the command line it used. That way you can build up your desired configuration according to this guide, then run the script and get the correct command line to run your server. Thus I still highly recommend getting the script.

wget http://www.dirty-motherfucker.org/blog/wp-content/uploads/2008/12/steamLauncher.sh
chmod +x steamLauncher.sh

The script allows to easily start and stop Source servers in the background. It also allows for several profiles for several games. So you can start all games through the same interface. I like it, if you don’t then maybe have a look at the original script.

So let’s create a profile for Left 4 Dead.

mkdir profiles
editor profiles/l4d.conf

Now, this is how a profile should look like:

longName                = Left 4 Dead - 1 instance      ; The name of this profile/game
gameName                = left4dead                     ; The name given by steam to the game (equals the foldername ex. cstrike|tf2|left4dead)
daemon                  = /home/steam/l4d/srcds_run     ; Usually srcds_run
updater                 = /home/steam/steam             ; Where is the updater located? (steam binary)
basePath                = /home/steam/l4d               ; The base path of the steam installation (this folder contains tf2/cstrike/left4dead folders)
userName                = steam                         ; What user the server should run as

serverName              = My L4D Server                 ; The name that will appear in the server browser
serverIp                = 11.222.33.111                 ; The IP address to bind the server socket to
serverPort              = 27010                         ; The first port to use for listening (will be treated as base port with forked l4d servers)
serverMap               = l4d_hospital01_apartment      ; The first map to load
serverMaxPlayers        = 4                             ; Maximum number of players allowed on the server
serverPriority          = 0                             ; Renice to set for the server process (-20 is high, 0 is normal, +19 is low)
forkCount               = 0                             ; How many instances to fork
additionalParams        = -nohltv -steamport 27960+## +clientport 25030+## +exec server##.cfg +sv_lan 0 ; Additional params to run the server with

Obviously in that profile you need to adjust at least the serverIp parameter. Now you can run that profile by typing.

./steamLauncher start l4d

Keep in mind though, this is not a state that we want to be comfortable with just yet. Nevertheless start it up and check if there is any trouble. If it comes up, attach to the screen session as the script tells you to.
In case you get an error saying “Cannot open terminal ‘/dev/pts/0′”, you can run “script /dev/null” and then attach to the session. There are other ways to resolve it. If you wanna find out about those they’re easy to find in Google ;)

Now, in the profile of Left 4 Dead in the additionParameters we also tell the server to execute a certain config file. Let’s create that one now.

editor l4d/left4dead/cfg/server01.cfg

In here we can now override the default map, difficulty and whatnot. For example:

hostname        My L4D Server - No Mercy - Normal Difficulty
map             l4d_hospital01_apartment
m_difficulty    normal

Setting the difficulty and map is almost pointless though as they can be changed by vote anyway. But I’m sure you can think of some other server variables that you might want to set. And this is the place to do it.

Now might be a good time to talk about forking with Left 4 Dead servers. Given that there are only 4 or 8 players on a Left 4 Dead server it’s almost a waste only running a single instance. That’s why you can fork multiple instances of servers. Let’s have a look at a Left 4 Dead profile modified to run 4 servers.

longName                = Left 4 Dead - 4 instances     ; The name of this profile/game
gameName                = left4dead                     ; The name given by steam to the game (equals the foldername ex. cstrike|tf2|left4dead)
daemon                  = /home/steam/l4d/srcds_run     ; Usually srcds_run
updater                 = /home/steam/steam             ; Where is the updater located? (steam binary)
basePath                = /home/steam/l4d               ; The base path of the steam installation (this folder contains tf2/cstrike/left4dead folders)
userName                = steam                         ; What user the server should run as

serverName              = My L4D Server                 ; The name that will appear in the server browser
serverIp                = 11.222.33.111                 ; The IP address to bind the server socket to
serverPort              = 27010                         ; The first port to use for listening (will be treated as base port with forked l4d servers)
serverMap               = l4d_hospital01_apartment      ; The first map to load
serverMaxPlayers        = 4                             ; Maximum number of players allowed on the server
serverPriority          = 0                             ; Renice to set for the server process (-20 is high, 0 is normal, +19 is low)
forkCount               = 4                             ; How many instances to fork
additionalParams        = -nohltv -steamport 27960+## +clientport 25030+## +exec server##.cfg +sv_lan 0 ; Additional params to run the server with

You can now also create server02.cfg, server03.cfg and server04.cfg in the l4d folder. This way you can start up each instance with a different map, difficulty, key, …
It remains a mystery to me how to properly send commands to the console with forked instances though. Although from what I read there are ways, but I just never had any luck with those. I also never had any luck with rcon. So basically when I play on my own servers I can’t do shit when someone is getting on my nerves. And we sure don’t want that!
So what do we do? We install SourceMod. It gives us a nice and easy way to kick players when they get on our balls. And we can be happy little server admins. During the installation of SourceMod it may be helpful to run only a single instance profile of Left 4 Dead so you can easily attach to the screen session and send of some commands to test your installation.
So let’s get to it!

First of all we require MetaMod. A very straight-forward guide can be found on the MetaMod Wiki. For completeness sake I’ll replicate the required information here though.

1. Download MetaMod at http://www.metamodsource.net/
2. Put it into the left4dead folder. (cd l4d/left4dead)
3. tar xvzf mmsource-1.7.0.tar.gz (Filename might differ)
4. In l4d/left4dead/addons create a file metamod.vdf and put the following into it:

   "Plugin"
   {
           "file"  "../left4dead/addons/metamod/bin/server_i486.so"
   }
   

5. Go back into your home directory and restart your server (./steamLauncher restart l4d)
6. Attach to the screen session and type “meta version”. Everything other than “Unknown command” is a good sign.

If the installation of MetaMod fails, refer to the MetaMod Wiki or bug me about it.

Now we’re gonna install SourceMod. Again a proper guide can be found at http://wiki.alliedmods.net/Installing_SourceMod.

1. Grab one of the latest snapshots of SourceMod from http://www.sourcemod.net/snapshots-1.2.php
2. Just like we did with MetaMod, put it into the l4d/left4dead folder.
3. Extract it via tar xvzf sourcemod-1.2.0-hg2469.tar.gz (Filename may differ again)
4. Go back into your home directory and restart your server (./steamLauncher restart l4d)
5. Attach to the screen session and type “meta list”. You should see that SourceMod is loaded.

Now last but not least we need to set ourself up as an admin in SourceMod. We do that by editing

l4d/left4dead/addons/sourcemod/configs/admins_simple.ini

Right at the bottom we will now put the following line:

"STEAM_YOURID" "99:z"

If you’re like me then you’ll have no clue what your Steam ID is. So open up steam and connect to any server in any game. Then open up the console and type “status”. In the list that appears there should be your nick somewhere. And close to it, you’ll find your Steam ID.

Now you should be an admin on your Left 4 Dead servers. You can join your own servers easily by invoking “openserverbrowser” in the Left 4 Dead console. To enjoy being an admin it helps to bind “sm_admin” to some key. Then you can open the admin menu and kick those pesky players who just can’t stop running ahead to get themselves killed.

In case this isn’t clear. You’ll have to forward some ports for your servers to work. Which can easily be accomplished by adding two lines to your /etc/ufw/after.rules file. Similar to these two:

# Steam TF2 Server
-A ufw-after-input -d 11.222.33.111 -m udp -p udp --dport 26900:27040 -m conntrack --ctstate NEW,RELATED,ESTABLISHED -j ACCEPT
-A ufw-after-input -d 11.222.33.111 -m tcp -p tcp --dport 27000:27020 -m conntrack --ctstate NEW,RELATED,ESTABLISHED -j ACCEPT

I hope I haven’t left any open questions. In case I did, please let me know ;)