Following on from the issue(s) I had with my OpenVPN server, I was still not happy/confident that in the event of a reboot or restart for any reason (wether deliberate or unintentional) all the necessary processes and services would startup successfully without some post boot intervention.
This in mind, I decided to create another server to transfer the live service(s) onto so I could get some much needed downtime on the existing server. Owing to the lack of another physical machine to do this with, I decided to create an virtual machine on our ESX cluster.
The initial steps were pretty easy, create a VM with x1 Vcpu, 1GB RAM, 30GB vdisk and x2 network interfaces. I installed Ubuntu server 9.04 i386 from the .iso and enabled LAMP and SSH. Installation completed and the system rebooted. Watching the console I saw that everything started at bootup time as it should.
Next step was to copy the websites across from the live server to this one. I installed NFS and mounted /var/www from the live server and copied all the sites across along with the relevant config files. I modified the config files to allow for the change of ip address and then restarted the system.
And that was when it started to go wrong. I only caught a glimpse of the error the first time I restarted the system. After reboot, I logged in a checked and apache was not running. Looking in /var/log/syslog did not show any clues why, even the error message itself did not seem to have been captured.
So I rebooted again and watched the console carefully, and this time saw the error :
apache2: no listening sockets available
along with
could not bind to address x.x.x.x:80 (where x was the ip address of the server)
Googling this made mention several times of other processes or programs perhaps using and blocking the socket/port in question, but this was happening at boot time, nothing else really had a chance to be up and running yet ? to test, I tried starting apache from the command prompt after bootup and it started fine, so what was going on
The main difference between this server and the live one was that this one was in a VM. Looking at the runlevel start scripts I noticed apache gets in there really early with S02apache2. Given my previous post where OpenVPN was trying to start before bridging on the live server, I wondered if perhaps the interface that Apache was trying to bind to was perhaps not quite ready at the time it tried during the boot process.
So I moved S02apache2 to S09apache2 for all runlevels and rebooted the VM again. Result, Apache was now loading as part of the boot process with no errors or manual intervention required.
So if you are also having issues with processes that do not start at boot time, but start fine after boot when you initiate them from the command prompt, you may just need to move them to a little late in the boot process to give other things time to start up beforehand.
I don’t profess to be the best system admin in the world, but I always get to the cause eventually :o)
Tags: apache, virtulisation, vmware