This is really going the last post of my series on Oracle VM Server / VM Manager on inexpensive hardware.
Last week a second Dell Power Edge arrived, followed by a little NAS/iSCSI System, the ES-2100 from Eurostor, which is rebranded Thecus N5200 Pro. I do link Eurostor because i made some very nice contact with their tech support.
After running the Oracle VM Server on a 2.4 GHz Core 2 Duo Xeon with 4 Gb Ram for about 70 days non-stop, we decided to do the next step: Incarnating a second server with a shared storage.
The one server runs an paravirtualized OEL5 with 2 GB Ram which itself runs an Oracle 11g test instance with medium load, a hardware virtualized Windows XP with 512 MB Ram that runs a Jetty with a few services and since 2 weeks a hvm Debian that serves as a mailrelay for that Exchange of ours… Which has a now really less load as SpamAssasin takes care of all.
Setting up the second Dell was flawless, nothing new.
The iSCSI was another thing… First i deleted the RAID6 as we decided to go for RAID5. Stupid me set disk usage to 100%, went for the weekend, came back on monday and saw: Wow, no space for the iSCSI target. Damn it, all timeplans went bazoo… So deleting the RAID once again and back to start, this time with 20% for Disk Usage (you never know) and 80% for one iSCSI target (if this was my machine, i really had a purpose for 1.5TB storage… but here.. *sigh*).
So, another 8 hours later, i bought a cheap 8 port Gigabit switch, set up the ES-2100 for link aggregation and connected it to both Oracle VM Servers.
I roughly followed the steps described here, but as i changed some steps, let me describe them:
rpm -Uvh iscsi-initiator-utils-6.2.0.742-0.5.el5.i386.rpm
iscsiadm -m discovery -t sendtargets -p 139.185.48.249
Example of removing a node:
iscsiadm -m node -p 10.2.0.250:3260,3 -T iqn.1992-04.com.emc:cx.apm00070202838.b0 -o delete
Listing the remaining:
iscsiadm -m nodeand having a new partition under /proc/partitions after
service iscsi restart
The ocfs2 cluster configuration is as simple as described in the linked Oracle document. I recommend adding names and ipaddresses corresponding to the one in /etc /ocfs2/cluster.conf to /etc /hosts, as the o2cb services won’t start otherwise. One thing Oracle forgot to mention is to open port 7777 on both machines in the iptables configuration.
At first i made the mistake to mkfs.ocfs2 the device and forgot to create a partition. This worked for whatever reason, but i destroyed the filesystem and created a partition with fdisk (new partition, primary, the whole thing).
Next, i didn’t follow Oracle but decided the following:
/dev/sdb1 /OVS ocfs2 defaults 1 0
So in the end we have a safe setup with hardware costs under 5k € and a setup time from about 6 or 7 days which brought some good knowledge and know-how. I think we wouldn’t have achivied this based on a VMWare solution brought by external consultants. Maybe that would have ended like the last Dilbert in that series 
.
To bring some variety to this blog, here’s a picture of the current setup:
I one of the google visitors or the 2 readers have any questions, feel free to ask 
For backup and testing purpose i keep a virtualized version of my webserver running under Parallels Desktop for Mac on my mac.
Recently Dell had some very convenient offers on their PowerEdge machines and i could barely resist to buy one, but finally, i managed to: cut the crap, i don’t want to have even more boxes standing around at home.
But i was thinking again: Whats better than one backup? Redundant backups
So i downloaded a copy of Parallels Workstation to install it on my PC at work. Equipt with a DVD and my image file from Parallels Desktop, i fired up the PC Version and after fiddling around with some pathes i can confirm that the Parallels Desktop 3.x vms are compatible with Parallels Workstation 2.x vms, at least a vm with a Debian OS inside.
But to me, performance was poor. My mac is a 2Ghz Core2Duo Mac Book with 3GB Ram, my Pc is a 2.66 Core2Duo Dell with 2GB Ram and Parallels Workstation was way slower on the PC than unter OS X. In addition: I didn’t want to spend another 50$ for Parallels (i already bought versions 2 and 3 for Mac, should be enough), so i thought about alternatives on my PC running Windows XP as host.
I already knew about QEMU which in contrast to Parallels and VMWare is a processor emulator and not a virtualization tool and therefore must be slower. I recommend the QEMU Manager for Windows Users, as this thingy already contains the kqemu virtualization extension.
Installation is dead simple and to my surprise, it was enough to convert my Parallels extending image with the Parallels Image Tool to a plain disk which i could use without further changes with QEMU.
Debian is very stable against the few “hardware” changes. The only problem i had that Debian wouldn’t find my eth0 device although the Realtek 8139too module was loaded. Solutions: The ethxxx devices are bound to the hardware (mac) address. I could look up the mac address in the Parallels configuration file, add this number in the QEMU config and voila, everything was up and running.
Somewhat complicated is bridged networking with QEMU. First you need the TAP-Win32 driver which is brought to you by the great OpenVPN project. The most simple way to install is through OpenVPN Gui for Windows.
After install add a new TAP device through the startmenu entry and then, the clou: There are some howtos which recommend enable bridged networking through the Internet Connection Sharing facilities of windows but it’s much simpler than that: Under network connections, select your default LAN Connection, the tap device, right click and choose bridge networks.
After that, change the network mode in your QEMU vm from user networking to tap networking and you’re ready to go.
Performance was quite good (at least at Parallels speed) but i guessed, there was room for more so i look for VMWare. Unfortunately, the QEMU image tool qemu-img.exe repeatedly crashed while converting my Parallels Image to the vmdk format so this was a dead end for me.
Although the Parallels image worked fine with QEMU and later on also in our Oracle VM Server as a hw virtualized XEN machine, i was not content and i looked out for VirtualBox which is available under a GPL license without USB support which i don’t need, but is able to use Intel VT-x and AMD-V technology.
I needed to convert the Parallels image from a raw disk format to innoteks vdi format, following the steps explained here.
All the tools mentioned there are also available under a Windows installation of QEMU and Virtualbox. Commands as follow:
qemu-img.exe convert foobar.hdd foobar.bin VBoxManage.exe convertdd foobar.bin foobar.vdi
For bridged networking unter VirtualBox the same applies as to QEMU without the need for OpenVPN, you can add a TAP device right from within VirtualBox . If you already have the bridged described above, you can add the VirtualBox interface with a right click to that bridge. Also, to have Debian not change the eth number, add the same mac address in the network tap of VirtualBox.
All this said and done, booting the system was blazing fast and also the backup of my Daily Fratze project which is pulled via rsync from my server, runs extremly smooth and i have absolutely no hassle setting up Ruby on Rails, RMagick and MySQL under the Windows “Operating System” but can use Debian or any other distro with a sane environment.
After fiddling around with both VMWare and VirtualBox, i’d prefer VirtualBox over VMWare Server (which is also available for free {i.e. for giving VMWare your personal data}). VirtualBox has less overhead then VMWare Server, is simple to configure and as i said, amazingly fast.
When the OS X version leaves beta, i’ll switch from Parallels on my Mac to VirtualBox, at least for my server live backup, so i can make it redundant with the one some kilometres away (call me paranoid if you like) at my workplaces PC.