xm top

Share This

Found at xkcd, here.

Share This

Last week a second Dell Power Edge arrived, followed by a little NAS/iSCSI System, the ES-2100 from Eurostor, which is rebranded Thecus N5200 Pro. I do link Eurostor because i made some very nice contact with their tech support.

After running the Oracle VM Server on a 2.4 GHz Core 2 Duo Xeon with 4 Gb Ram for about 70 days non-stop, we decided to do the next step: Incarnating a second server with a shared storage.

The one server runs an paravirtualized OEL5 with 2 GB Ram which itself runs an Oracle 11g test instance with medium load, a hardware virtualized Windows XP with 512 MB Ram that runs a Jetty with a few services and since 2 weeks a hvm Debian that serves as a mailrelay for that Exchange of ours… Which has a now really less load as SpamAssasin takes care of all.

Setting up the second Dell was flawless, nothing new.

The iSCSI was another thing… First i deleted the RAID6 as we decided to go for RAID5. Stupid me set disk usage to 100%, went for the weekend, came back on monday and saw: Wow, no space for the iSCSI target. Damn it, all timeplans went bazoo… So deleting the RAID once again and back to start, this time with 20% for Disk Usage (you never know) and 80% for one iSCSI target (if this was my machine, i really had a purpose for 1.5TB storage… but here.. *sigh*).

So, another 8 hours later, i bought a cheap 8 port Gigabit switch, set up the ES-2100 for link aggregation and connected it to both Oracle VM Servers.

I roughly followed the steps described here, but as i changed some steps, let me describe them:

• Installed the iscsi tools with:

  rpm -Uvh iscsi-initiator-utils-6.2.0.742-0.5.el5.i386.rpm

• Discovering and removing unused services like that:

  iscsiadm -m discovery -t sendtargets -p 139.185.48.249

  Example of removing a node:

  iscsiadm -m node -p 10.2.0.250:3260,3 -T iqn.1992-04.com.emc:cx.apm00070202838.b0 -o delete

  Listing the remaining:

  iscsiadm -m node

  and having a new partition under /proc/partitions after

  service iscsi restart

The ocfs2 cluster configuration is as simple as described in the linked Oracle document. I recommend adding names and ipaddresses corresponding to the one in /etc /ocfs2/cluster.conf to /etc /hosts, as the o2cb services won’t start otherwise. One thing Oracle forgot to mention is to open port 7777 on both machines in the iptables configuration.

At first i made the mistake to mkfs.ocfs2 the device and forgot to create a partition. This worked for whatever reason, but i destroyed the filesystem and created a partition with fdisk (new partition, primary, the whole thing).

Next, i didn’t follow Oracle but decided the following:

• Unmount the /OVS on the first server (the one with all the vms)
• Adding the following stanza to /etc /fstab:

  /dev/sdb1               /OVS ocfs2   defaults        1 0

• mount -a
• Mount the old /OVS to somewhere else and rsychned it to the new location. I reached transferrates around 20MB/s with concurrent writes from the other server. Not but for a inexpensive device like that little iscsi thingy.
• Added the the new server to the pool.
• Rebooted both servers, just to be sure that they come back healty.
• Restarted all vms, which worked greated over the iscsi.
• Tested load-balancing and live migration and what can i say: Wow, it works. Fast and flawless. Great thing.

So in the end we have a safe setup with hardware costs under 5k € and a setup time from about 6 or 7 days which brought some good knowledge and know-how. I think we wouldn’t have achivied this based on a VMWare solution brought by external consultants. Maybe that would have ended like the last Dilbert in that series .

To bring some variety to this blog, here’s a picture of the current setup:

I one of the google visitors or the 2 readers have any questions, feel free to ask

See the other posts here: 1, 2, 3, 4, 5.

Share This

1, 2, 3, 4

Share This

Recently Dell had some very convenient offers on their PowerEdge machines and i could barely resist to buy one, but finally, i managed to: cut the crap, i don’t want to have even more boxes standing around at home.

But i was thinking again: Whats better than one backup? Redundant backups

So i downloaded a copy of Parallels Workstation to install it on my PC at work. Equipt with a DVD and my image file from Parallels Desktop, i fired up the PC Version and after fiddling around with some pathes i can confirm that the Parallels Desktop 3.x vms are compatible with Parallels Workstation 2.x vms, at least a vm with a Debian OS inside.

But to me, performance was poor. My mac is a 2Ghz Core2Duo Mac Book with 3GB Ram, my Pc is a 2.66 Core2Duo Dell with 2GB Ram and Parallels Workstation was way slower on the PC than unter OS X. In addition: I didn’t want to spend another 50$ for Parallels (i already bought versions 2 and 3 for Mac, should be enough), so i thought about alternatives on my PC running Windows XP as host.

I already knew about QEMU which in contrast to Parallels and VMWare is a processor emulator and not a virtualization tool and therefore must be slower. I recommend the QEMU Manager for Windows Users, as this thingy already contains the kqemu virtualization extension.

Installation is dead simple and to my surprise, it was enough to convert my Parallels extending image with the Parallels Image Tool to a plain disk which i could use without further changes with QEMU.

Debian is very stable against the few “hardware” changes. The only problem i had that Debian wouldn’t find my eth0 device although the Realtek 8139too module was loaded. Solutions: The ethxxx devices are bound to the hardware (mac) address. I could look up the mac address in the Parallels configuration file, add this number in the QEMU config and voila, everything was up and running.

Somewhat complicated is bridged networking with QEMU. First you need the TAP-Win32 driver which is brought to you by the great OpenVPN project. The most simple way to install is through OpenVPN Gui for Windows.

After install add a new TAP device through the startmenu entry and then, the clou: There are some howtos which recommend enable bridged networking through the Internet Connection Sharing facilities of windows but it’s much simpler than that: Under network connections, select your default LAN Connection, the tap device, right click and choose bridge networks.

After that, change the network mode in your QEMU vm from user networking to tap networking and you’re ready to go.

Performance was quite good (at least at Parallels speed) but i guessed, there was room for more so i look for VMWare. Unfortunately, the QEMU image tool qemu-img.exe repeatedly crashed while converting my Parallels Image to the vmdk format so this was a dead end for me.

Although the Parallels image worked fine with QEMU and later on also in our Oracle VM Server as a hw virtualized XEN machine, i was not content and i looked out for VirtualBox which is available under a GPL license without USB support which i don’t need, but is able to use Intel VT-x and AMD-V technology.

I needed to convert the Parallels image from a raw disk format to innoteks vdi format, following the steps explained here.

All the tools mentioned there are also available under a Windows installation of QEMU and Virtualbox. Commands as follow:

qemu-img.exe convert foobar.hdd foobar.bin
VBoxManage.exe convertdd foobar.bin foobar.vdi

For bridged networking unter VirtualBox the same applies as to QEMU without the need for OpenVPN, you can add a TAP device right from within VirtualBox . If you already have the bridged described above, you can add the VirtualBox interface with a right click to that bridge. Also, to have Debian not change the eth number, add the same mac address in the network tap of VirtualBox.

All this said and done, booting the system was blazing fast and also the backup of my Daily Fratze project which is pulled via rsync from my server, runs extremly smooth and i have absolutely no hassle setting up Ruby on Rails, RMagick and MySQL under the Windows “Operating System” but can use Debian or any other distro with a sane environment.

After fiddling around with both VMWare and VirtualBox, i’d prefer VirtualBox over VMWare Server (which is also available for free {i.e. for giving VMWare your personal data}). VirtualBox has less overhead then VMWare Server, is simple to configure and as i said, amazingly fast.

When the OS X version leaves beta, i’ll switch from Parallels on my Mac to VirtualBox, at least for my server live backup, so i can make it redundant with the one some kilometres away (call me paranoid if you like) at my workplaces PC.

Share This

Honestly, i was afraid of installing Oracle the first time under a linux environment but i must say, the installation was pleasantly easy, following the steps described here. No surprises at all. My host has a fixed IP adress, no problem with DHCP (although a don’t quite get why this should be any problem at all).

Fun started after stopping the database.

First: SELINUX is enabled per default under OEL5. I found the following tutorial which recommends turning SELINUX off. I disagree with that.

After installation complets, the following commands are necessary to enable Oracle to load its shared objects and to some text relocation with it:

chcon -t textrel_shlib_t /opt/app/oracle/product/11.1.0/db_1/lib/libnnz11.so
chcon -t textrel_shlib_t /opt/app/oracle/product/11.1.0/db_1/lib/libclntsh.so.11
chcon -t textrel_shlib_t /opt/app/oracle/product/11.1.0/db_1/lib/libnque11.so
chcon -t textrel_shlib_t /opt/app/oracle/product/11.1.0/db_1/lib/libsqlplus.so

otherwise you’ll some errors like

/opt/app/oracle/product/11.1.0/db_1/bin/ocrconfig.bin: error while loading shared libraries: /opt/app/oracle/product/11.1.0/db_1/lib/libnnz11.so: cannot restore segment prot after reloc: Permission denied

(Paths may vary) when starting the database.

Speaking of starting, Oracle comes with no batteries included, i.e. a startup script is missing.

I started with the one described here and added some lines to start the enterprise manager controll, see resources below.

In the meantime, the first import should be finished. The performance of 11g seems ok, on a virtual machine with a minimum of 1GB Ram. Maybe i’ll post some more impressions in a few weeks.

Resources

• Oracle® Database Installation Guide 11g Release 1 (11.1) for Linux
• ORACLE-BASE – Oracle Database 11g Release 1 (11.1) Installation On Enterprise Linux 4.5 and 5.0
• ORACLE-BASE – Automating Database Startup and Shutdown on Linux
• Sample Oracle Startup Script for Oracle 11G on OEL5

Share This

The OEL5 is delivered as 5 iso files, cd images they are. The Oracle VM Manager is only able to install the patched paravirtualized guest system from one installation medium. Damn it, they praise their system as blazing fast under their own VM and didn’t manage to deliver the stuff as one dvd image?

After a lot of googling i found the following script. Its a CentOS script to combine their installation mediums into on, big iso file. I thought, give it a try as CentOS is based on RedHat as well is OEL5. It ate a lot of diskspace but in the end, i got a working DVD that i mounted via the loop device.

mount -o rw,loop /OVS/tmp/oel5dvd.iso /tmp/oel5/

The next step requires the installation medium available either via http, ftp or nfs. Starting nfsd and exporting the fs is done easily via:

/etc /init.d/nfs start
exportfs *:/tmp/oel5/

I had all the iso and the resulting dvd on my Oracle VM Server. The default installation is rather strict with the iptable rules.

Although Oracle VM Manager will except an nfs://127.0.0.1/… path and also do the initial patching, the installation will fail afterwards.

To get NFS working, i followed these instructions to pin all needed ports and open them in iptables.

Installation itself was after that straigt forward. I deselected all prepackaged environments since the server was destined to be a database server.

I didn’t realize that this was going to be a problem.

There is no centralized repository in the RHEL or OEL world like the one in Debian or Ubuntu. Either hunt the rpms down or subscribe to the services both manufactures offers, in my case the Oracle Unbreakable Linux Network. Hello? I just wanted to install a gcc, for example.

Their is no possibility in the gui to reuse your installation medium as a repository for “YUM” or “Pirut”.

Bummer! After a lot of searching i found this, but this method didn’t work for me. Nevertheless, i’ve learnd how to configure repositories for yum. Pirut did find the installation medium but wasn’t able to install a package due to an “uropen error unknown url type: media”.

My solution was to copy all rpms from the /Server directories of the original installation disk along with “comps-rhel5-server-core.xml” from the first disc found in /Server/repodata/ to disk, running

createrepo -g comps-rhel5-server-core.xml .

and finally exporting this directory as a nfs share, configuring it to yum with the following:

[oel5]
name=Oracle Enterprise Linux 5
baseurl=file:///tmp/oel5_repository
enabled=1
gpgcheck=0

After a “yum clean all”, pirut can add and remove packages from the “original” medium without any hassle.

I installed the following packages:

yum install binutils
yum install compat-libstdc++
yum install elfutils-libelf
yum install elfutils-libelf-devel
yum install gcc
yum install gcc-c++
yum install glibc
yum install glibc-common
yum install glibc-devel
yum install glibc-headers
yum install libaio
yum install libaio-devel
yum install libgcc
yum install libstdc++
yum install libstdc++-devel
yum install make
yum install sysstat
yum install unixODBC
yum install unixODBC-devel

preparing the next post which is about installing Oracle 11g on Oracle Enterprise Linux 5.

Resources:

• CentOS Build and mkdvdiso, local mirror of the mkdvdiso.sh script
• F7 Network File System (NFS), local PDF Mirror
• markwilson.it » Trying to get Red Hat Enterprise Linux to accept a DVD-based repository, local PDF Mirror
• YUM – Yellow dog Updater, Modified / Pirut
• Sample YUM DVD Repository file

Share This

The product isn’t as new as Oracle wants you to believe, it’s based on Xen 3.1.1 “with many, many performance enhancements done by our engineers”. We’ll see.

After unpacking our shiny new Dell (The rabat action was entitled “My First Server”, sweet… kind of), i booted of CD, left most settings on default and made my way to the non-graphical Installer. No surprises here, Oracles Xen Version runs on their own Linux distribution (“Oracle Unbreakable Linux / Enterprise Linux”) which itself is based on Red Hat Enterprise Linux (ok, tante isn’t wrong about re branding stuff and so…). It’s quickly said that the stuff is available without re branding in it’s original form but to many people it is important that these products are certified by Oracle.

Oracle VM comes in two packages, a server and a manager component, which needs to be installed on separate machines.

Oracle VM Manager runs on both Oracle Enterprise Linux 4 and 5. I set up a VMWare Server machine on my workstation for that as i didn’t want to ruin a near perfect desktop. Installation of the manager is simple with one little caveat: The script prompts for the oc4j admin password which is “oracle”. It doesn’t want to know a new one but the default.

VM Manager installs an OC4J, an Oracle XE Database and the manager app itself which is a J2EE app.

Within this thingy you define server pools and servers.

For the later you have to install one. Put in the disc, but the machine, accept a view defaults and you’re ready to go. I put two ethernet cards in our dell for accessing the server itself and the other for the virtual machines.

I had some weird problems configuring the ntp daemon, that was all.

My first experiment was to install a fully hardware virtualized Windows XP. Creating the vm, adding diskspace and so on with Oracle VM Manager is as easy as with any other tool like the ones from VMWare. One noteworthiness is that you need to import your iso files from any install media to the server. This was easily done via ftp. I loaded the machine with XP and for later, all 5 disks of Oracle Enterprise Linux 5.

I spend nearly a day to get the bridged networking with xp up and running. I learned a lot about networking under Xen (especially this page was useful) but in the end, the on and only reason why i couldn’t reach the Windows XP domU was the bastard firewall build into XP. Bummer!

XP runs reasonable fast within the hvm environment. The only thing that felt like ages was creating the virtual disk.

From within the manager application one is able to reach a VNC server running on the Oracle VM Server entitled console. You even see the bootscreens and so on from this VNC. I first saw this on the DOAG last month, it certainly didn’t work out of the box. The Oracle VM Manager application was missing the jar files with the Tight VNC applet.

Although the VNC server is reachable with any VNC client, it’s nice to start right from within the browser. To install the applet properly on the manager machine, do the following:

1. Download the TightVNC rpm at the Oracle OVM RPM repository (also, there’s a plugin for linux clients)
2. Install it with

   rpm -ivh tightvnc-java-1.3.9-1.noarch.rpm

3. Go to the OVS homedirectory which is

   /opt/oc4j/j2ee/home/applications/OVS/webapp1/Class

   Make sure there is a file “VncViewer.jar”. Mine was there after installing the rpm.

4. Sign the “VncViewer.jar” file with the following commands:
   

   /opt/oc4j/java/jdk1.5.0_11/bin/keytool -genkey -alias vncviewer -validity 365
   /opt/oc4j/java/jdk1.5.0_11/bin/jarsigner VncViewer.jar vncviewer

5. Clean your browser cache and you’re ready to go.

The next step on the list was installing a paravirtualized Oracle Enterprise Linux. I hoped the templates at oracle.com were paravirtualized but no, they all are hvm templates. Why the heck? So this is where the fun starts and the next post takes place…

So far, i like the concept of Oracle VM Server. The software makes it easy to start with one server which acts as a virtualisation server, a server pool master and an utility server. If your needs grow, you throw some hardware in, define a new server in the manager application, add it to the pool and you’re done. From within the manager app, you can move the virtual machines around as you like. You certainly need to have for that to work some sort of shared storage available, but i don’t think that moving the /OVS tree of the first server to some kind of iSCSI or any other SAN is a big problem.

I know that Oracle builds their product upon Open Source but in this case, i think the package works great. Buying some relatively cheap server hardware, installing and configuring two products (Oracle VM Server and VM Manager) was far less expensive than a VMWare ESX3 test installation. Where Oracle fails is their Oracle Enterprise Linux, at least in some points. But as i said, in another post.

At one point a had a problem with the Manager App. Somehow my server pools went missing, i clicked around and finally found “Restore from Manager”. Hm, well… *click* and my virtual machines were gone. After searching awhile i found out that some configuration files for VMS exists under /etc /xen on the server but i guess they aren’t use. Any changes to them were ignored. All the vm starting, stopping and restarting is done from the manager app. So be careful when fiddling around with the config files on the server!

Share This

Der VM Server ist stupide einfach zu installieren. Alles, was danach folgt, ist eine Qual, wenn man einmal den Komfort von apt-get, debian und co. kennen gelernt hat. Ich habe gerade das Gefühl, ich werde wahnsinnig dabei.

Es ist unter Oracle Enterprise Linux ohne weiteres nicht möglich, nachträglich noch weitere Pakete aus den Installations CDs zu installieren. Dazu braucht man einen Account im Oracle Unbreakable Linux Netzwerk. Ich will die Schuld nicht alleine Oracle anlasten, bei RedHat läuft es wohl genauso.

Worauf ich hinaus will: Der Erfahrungsbericht folgt, allerdings später, als geplant.

Share This

Nicht nur hat Dell erfreulich schnell geliefert, es war sogar Versandkosten frei. Aber das beste: Man kennt das ja mit Windowskisten: Man baut sie auf, nimmt sie in Betrieb und hat erst mal damit zu tun, die Ausrufungszeichen in der Systemsteuerung zu beseitigen, sprich Treiber nachzuinstallieren sowie unnütze Shareware / Demoversionen von irgendwelchem Rotz zu deinstallieren. Nicht so auf dem Vostro 200. Alle Komponenten waren unter XP sauber installiert und ich musste keine einzige, dämliche Demo entfernen (ok, Virenscanner und ein paar andere, für mich überflüssige Sachen habe ich entfernt, aber Dell weiß ja nicht, das wir einen Firmenweiten Virenscanner haben).

Meine diesbezüglichen Erfahrungen mit Geräten vom “Schrauber um die Ecke” oder aus großen Elektronikketten waren bisher genau Gegenteilig, jedesmal war annähernd soviel Handarbeit gefragt, dass ich es direkt hätte selber machen können. Selbst mein altes Thinkpad (IBM R52) war nicht so ordentlich vorkonfiguriert.

Nicht, das meine Java Programme bis jetzt so schrecklich langsam waren, Eclipse war es dafür – bei sehr vielen Projekten – um so mehr. Das hat natürlich jetzt ein Ende.

Spannend wird die weitere Entwicklung: Evaluierung von Oracle VM, einer Virtualisierungslösung auf XENSource Basis. Der Manager der Oracle VM wird zur Zeit ebenfalls nur für Linux ausgeliefert (ok, nur für Oracle Unbreakable Linux), aber das ist ein Schritt weit in die Richtung, auch Linux auf den Arbeitsplatz zu bringen. Und so läuft auf meiner Workstation seit einigen Tagen OEL5 in einer VMWare und auch dort muss ich sagen: Rasant schnell.

Somit finde ich jetzt auch hoffentlich einen Übergang: In den nächsten Wochen werde ich von meinen Erfahrungen mit Oracle VM berichten, es gibt also mal wieder etwas neues neben J2EE und Rails auf diesem Blog.

Share This