CCIE.gresz.ca

I found this interesting post on building a dual-processor, quad-core ESXi 4 server for Cisco UC applications.  I really was not looking to invest more money right now in new hardware, so for fun I tried to install ESXi 4.  Last year I tried running ESXi 3, without any luck.

I installed ESXi to a 1 GB flash drive following these instructions and using the Unix command “dd’ on my Mac.

Everything seemed to boot OK, except my Intel dual-port PCI Ethernet 09213P cards were not recognized.  I still have to double-check if the SATA controller works - but I didn’t want to risk losing any data that is currently there.

I guess I will have to replace my two dual-port Ethernet cards with some single PCI and PCIe cards that are supported by ESXi 4.

I was asked sometime last year about how my UC suite performed on my whitebox VMware 2 server running on Ubuntu.  I was unable to properly answer this question until I started getting into full labs and running a CUCM pub, sub, Unity and CUPS.

The short answer is that the CUCM pub and sub ran quite well on the server.  The performance started degrading when I turned on a third server such as Unity.  The CUCM annunciator, UCCX prompts, and Unity speech became broken and garbled.  When I turned up 4 servers (for example CUCM pub, sub, Unity, and CUPS) the system was more or less paralyzed.

My guess that this has something to do with either only having on 5400 RPM 500 GB hard drive, or that VMware server does not like oversubscribing processor cores.  (One core for the Ubuntu Linux OS, one for CUCM sub, one for CUCM pub, leaving only room for one more server).

In the end I was mostly studying with a single CUCM pub, Unity, CUPS running at the same time.

Well, I sat my first lab attempt and was quite disappointed my my result.  It was a bit of a wake-up call, especially the Open Ended Questions where I thought all four were quite easy and that I answered them all correctly.  Unfortunately Cisco disagreed and I ended up with only 2 right answers…

Anyway, a bit of holiday time, some catching up on work projects and then back on the horse to work on my CCIE voice weaknesses.  I’m probably not going to schedule another attempt until sometime next spring.

Someone wrote in the comments that they were having trouble installing ESXi on their HP DL380 G3 from an .iso CD/DVD.  I can’t remember if I had problems with this or not, as I eventually installed using PXE.  In the DL380 BIOS I set the OS to “Linux”, drive controller to Raid5 (as it was old equipment and I found that the drives were failing often).

I built a DHCP/PXE server on an existing DL380 running Debian Etch by following this document: http://www.debian-administration.org/articles/478

I then followed a document on how to extract VMWare ESXi files from the .iso and add it to the PXE server: http://communities.vmware.com/docs/DOC-6824

The resulting tree in my Debian server looks like this:

stgreszc@stor:/var/lib/tftpboot$ tree
.
|– VMwareESXi
| |– binmod.tgz
| |– boot.cfg
| |– cim.tgz
| |– environ.tgz
| |– ldlinux.sys
| |– license.tgz
| |– mboot.c32
| |– oem.tgz
| |– safeboot.c32
| |– syslinux.cfg
| `– vmkernel.gz
|– boot.txt
|– debian
| `– etch
| `– i386
| |– initrd.gz
| `– linux
|– pxelinux.0
`– pxelinux.cfg
`– default

To make your life easier, I uploaded my /etc/dhcp3 directory and my /var/lib/tftpboot directories.

You still need to register at the VMWare website to acquire your licenses.

Here is a video showing the PXE install via the HP iLO (remote console).  It really only takes a couple of minutes.  If the video stops, click on the bubble or text:

When I was originally trying to install the UC7 applications, I encountered a problem when trying to install the subscriber (subsequent node) to the publisher (first node).  I received an error “Unable to set the system clock using the NTP server on the first node in the cluster”:

I’m not completely sure what the underlying problem was, but I suspect that there might have been some issues due to the Debian ntp server package being installed on the host.  When I was getting these errors, I was using Debian Etch and VMware Server 1.6 on HP DL380.  On those same servers I am now running VMware ESXi and had no problems installing and synchronizing the pub and sub.  I also installed a pub/sub successfully on VMware server 2 running on Ubuntu 8.10 (no NTP package installed).

Another thing to be aware of is that you must add the subscriber details in the CUCM admin pages of the publisher before installing the subscriber.  You do this under “system->server”.  I changed the publisher from hostname to IP address since I’m not using DNS.

Finally, you can always check IP connectivity by logging into the subscriber using SSH (you can at this point in the install) and the command “utils network ping <IP or hostname of pub>”:

I have not yet tried synchronizing the CUCM pub to an NTP server, as when I installed I always used the hardware clock.  I will try to sync the publisher when I integrate all the UC7 nodes.

Yesterday I finished installing the 5 UC7 nodes in VMware Server 2.  The last ones to get going were CUCCX7 (IPCCX) and CUPS7.

I set up VMware to start all the UC7 applications when the host Ubuntu server starts up.  (Select the main host server in “Inventory” then in the “Commands” window select “Edit virtual machine startup/shutdown”):

Finally you can see all 5 servers running in VMware.  I don’t think that the processor and memory usage is highly accurate though.   I haven’t looked that closely, and I think I should have enough processor and memory resources.

Some people have asked about memory requirements for the UC7 suite.  You can install CUCM and CUC with 1024 M.  CUPS requires just under 1300 M or it won’t install.  CUCCX requires 2048 M to install, but after installation you can bring the VM guest memory back down to 1024 M.

This is what VMware shows as the resources being used by the various guest servers running in an idle state:

CUCM7a (pub): 245 M, 0.083 GHz

CUCM7b (sub): 163 M, 0.058 GHz

CUPS7: 364 M, 0.092 GHz

CUC7: 419 M, 0.245 GHz

CUCCX7: 153 M, 0.012 GHz

I began installing the UC7 suite of applications on my home VMware Server 2, running on Ubuntu 8.10.  I installed 2 nodes of CUCM, CUPS, CUC, and CUCCX.  The installation in VMware Server 2 is a little different than ESXi, so I recorded the installation of the first CUCM node.  If the video stops, click on the bubble or text.

I started installing Cisco Unified Presence Server 7 (CUPS 7) and Cisco Unified Contact Centre Express 7 (CUCCX 7) in VMWare ESXi on a second HP DL380.

The CUPS installation was identical to the CUCM7 and Unity install, except the version of CUPS that I installed required an 80 GB drive and just under 1.3 GB of RAM.

For CUCCX I am using a Cisco IP Telephony version of Windows Server 2003.1.1 which was originally installed on an MCS-7835H1 and converted with VMware Converter.  Because I am running VMware ESXi, I needed to convert the image to a .ovf file using ovfTool from VMware.

After downloading the Linux tarball, the commands were quite simple (after some googling).  To convert the .vmx to .ovf:

sudo ~/ovftool/ovftool CIPT2003.vmx CIPT2003.ovf

After that, I needed to send the VMware image to my ESXi server using this command:

sudo ~/ovftool/ovftool CIPT2003.ovf vi://root@10.52.101.12/vm?name=IPCCX

where the 10.52.101.12 is the VMware server and IPCCX is the name of the new guest.  This seems to have worked quite well!  I just need to patch the OS and install CUCCX7.

stgreszc@stor:/vm-images/UC-software/IVR/UCCX-vm$ sudo /home/stgreszc/ovftool/ovftool /vm-images/vm-cm-os/CM\ OS\ 2003.1.1/CIPT2003.vmx CIPT2003.ovf
Exporting /vm-images/vm-cm-os/CM OS 2003.1.1/CIPT2003.vmx to CIPT2003.ovf
Converting disk /vm-images/vm-cm-os/CM OS 2003.1.1/CIPT2003.vmdk: 100% done.
Export completed successfully!

stgreszc@stor:/vm-images/UC-software/IVR/UCCX-vm$ sudo /home/stgreszc/ovftool/ovftool ./CIPT2003.ovf vi://root@10.52.101.12
Deploying ./CIPT2003.ovf to vi://root@10.52.101.12
Password:
[2010-01-27 14:07:27.223 0xb4aa36c0 warning 'ovftool'] (vim.fault.OvfElementInvalidValue) {
dynamicType = ,
faultCause = (vmodl.MethodFault) null,
lineNumber = 85,
name = “Connection”,
value = “bridged”,
msg = “”,
}
[2010-01-27 14:07:27.223 0xb4aa36c0 warning 'ovftool'] (vim.fault.OvfElementInvalidValue) {
dynamicType = ,
faultCause = (vmodl.MethodFault) null,
lineNumber = 95,
name = “Connection”,
value = “bridged”,
msg = “”,
}
Deploying disk [vm2:storage1] CIPT2003/CIPT2003.vmdk: 100% completed.
Deployment completed successfully!

At work I use the free “bare-metal” VMware ESXi virtualization software a lab that I set up using old MCS servers (HP ProLiant DL380 G3).  This video shows the installation of Unity Connection 7 using the VMware Infrastructure Client.  (Make sure to click the comments bubble if the video stops).

I installed Ubuntu 8.10 Server 64-bit as my underlying OS and compiled VMWare Server 2 to host my UC7 applications.

The Ubuntu Server installation is quite straight forward. I installed it from a USB stick, following the instructions here and downloading the required boot files here.

There is a good install guide on howtoforge: perfect-server-ubuntu-8.10

I just selected the basic installation options and added OpenSSH server. All the hardware was detected correctly, including the two Intel dual-port PCI Ethernet cards. This gives me a total of 5 Ethernet ports on my server including the built-in port on the motherboard.

Installing VMware server 2 was quite easy too. There is a good guide on howtoforge: how-to-install-vmware-server-2-on-ubuntu-8.10.

Read the rest of this entry »