I am a network engineer by profession, but with the proliferation of SDN and OpenFlow, I have had to spend a lot of time re-learning a lot of system admin skills that I’d shelved years ago. Now, I’ve been a virtualization user forever. From VMware (Fusion, ESX), VirtualBox, to Parallels, I’ve used them at least in testing if not in production environments. I’d not really spent any mentionable amount of time with XEN, qEMU or KVM, but some projects I was working on suggested it for the virtualization mechanism, so I figured I’d try to pick it up. I think the hosting provider of this blog is using it to provide the VPS that this site runs on, so it is clearly production quality. In looking around for a straightforward how-to, a few things became obvious pretty quickly:
KVM is very robust.
KVM and qEMU have got a lot of documentation, but not all of it is straightforward.
They have a lot of pieces needed to make it work efficiently.
It wasn’t a plug and run experience, by any means. I wanted a headless, remotely manageable system that ran on a bare minimum install (i.e. no GUI). After a late evening of hacking at it, here is what I finally had to do to get it running and build a functional VM on a fresh install of CentOS 6.3.
Install all the dependancies. And there were many. Coming from using the VirtualBox CLI, little things that I expected didn’t work if I didn’t have this or that installed.
yum install -y qemu-kvm.x86_64 qemu-kvm-tools.x86_64 \ kvm libvirt bridge-utils tunctl python-virtinst.noarch avahi
Restart the deamons
/etc/init.d/messagebus restart /etc/init.d/avahi-daemon restart /etc/init.d/libvirtd restart
Make the new daemons start at boot:
/sbin/chkconfig messagebus on /sbin/chkconfig avahi-daemon on
vi /etc/sysconfig/network-scripts/ifcfg-eth1
Your file should look something like this when done:
DEVICE="eth1" HWADDR="c0:ff:ee:c0:ff:ee" # (leave this as your MAC address) NM_CONTROLLED="yes" BRIDGE=br0 ONBOOT="yes" UUID="a9c26927-7650-42e9-a86a-1ae1227eac4b" #leave this as your UUID)
vi /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE="br0" TYPE="Bridge" BOOTPROTO="static" IPADDR="10.10.10.50" # Use your own IP address NETMASK="255.255.255.128" GATEWAY="10.10.10.1" # Provide your gateway IPV6INIT="yes" IPV6_AUTOCONF="yes" NM_CONTROLLED="yes" ONBOOT="yes"
service network restart
You’re going to use virt tools and virsh to manipulate the guests. I tried it a few other ways and this seems far more supportable. I wanted to install a CentOS 6.3 guest, I grabbed the install media from a local mirror. I have a volume mounted as /services that I keep stuff in.
mkdir /services/vm/template-host cd /services/vm/template-host wget http://cosmos.cites.illinois.edu/pub/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-bin-DVD1.iso
Now use the virt-install command to boot the system with the following parameters:
2G of RAM
50G disk named disk.img
Console redirected to a VNC instance on port 5901
Network interface attached to br0
4 CPUs
CDROM points to install image
virt-install --name template-host --ram 2048 --disk /home/vm/template-host/disk.img,size=50 \ --graphics vnc,port=5901 --network bridge=br0 --vcpus=4 --os-type=linux \ --cdrom=/home/vm/Install_Media/CentOS-6.3-x86_64-bin-DVD1.iso
Now, to connect to the console, you’ll need to tunnel VNC over ssh. This can be adjusted, but that’s outside of the scope of this.
I like to redirect real port numbers for my own sanity.
ssh -N -p 22 -c 3des buraglio@vmhost -L 5901/localhost/5901
The above command will redirect localhost port 5901 to port 5901 on the host vmhost
Connect to the VNC server:
Once connected you should drop right into the console of the KVM instance. Install as a normal system at that point.
Once installed It’ll appear as a normal host console.
I found virsh to be the most useful for manipulating the virtual machine.
[root@behemoth Install_Media]# virsh list Id Name State ---------------------------------------------------- 9 template-host running
Helpful commands I found after 30 minutes of poking around and playing with virsh:
virsh list –show the list of virtual machines
virsh destroy <vm name> –hard shut down a host (I believe)
virsh undefine <vm name> –remove the VM from registration –this one was hard for me to find.
Here is a complete list of the commands.
Pingback: Networking Field Day 5, participation.
I really like your blog. You write about very interesting things. Thanks for all your tips and information
Hello! I really like this blog. Tell me please – from where do you have information for ths blog?
Fixed a typo in the ifcfg-br0 section.
I am in the exact same boat right now, having a network security background and a fair amount of experience with “desktop” hypervisors like Fusion/Parallels but also some ESXi. I discovered KVM through my interest in SDN/OpenFlow and have been hooked ever since.
In your opening statement you mentioned you were looking for a headless, remotely managable solution without a GUI.
If you ever want to dive into a web GUI for the management, you should check out archipelproject.org (awesome system, really clever, controlling hypervisors through XMPP/Jabba!) and also oVirt.
Great find! Thanks for the link. A web client would also work well as it implies that it can run on a headless box for remote management. I’m going to look into this for sure for the OpenFlow lab I’m working on.
hi there,
I really like this howto. I was folloving it on a cenOS6.4 up to the “connect to the console” part. At that point I haven’t find a way how to VNC connect to the KVM-guest fom a windows machine. Is there some easy way to do that???
Saniel, you need to give your VM an IP address that is routable through the host system. Then you can connect to that IP.
hmmm…but how to give a VM an IP when there is no guest OS yet 🙂
I need a way how to connect to a VM-guest console to do the install of an OS. My scenario is…an centOS6.4 KVM server-no GUI and a win7 workstation.
you want to look into SPICE (it’s basically something like RDP). Libvirt has support for it. I am not sure though you would be able to connect to a new machine over the network without the help of some additional libvirt management tools.
Take a look at http://www.ovirt.org/
oh.. and http://spice-space.org/ of course.
Nick,
Great post. I’ll have to give KVM on Centos a shot. KVM is definitely rough around the edges. Simple bridge/NAT networking is easy, although installing on headless server complicates matters. I didn’t want to go the VNC route as you did. I couldn’t get the console output to a terminal the way you can using ‘-display curses’ in qemu.
Another frustrating aspect for me was trying to set up a network topology such a two VMs talking to one another through a virtual router. This is easy with ESXi, Workstation, and virtualbox. Not so much with KVM.
Jeff, you can do the same kind of network setup with KVM. Just hook up all the virtual machines to the same bridge and make either your host or another virtual machine the router.
cryptochrome,
Thanks for the tip. It would work as far as basic network connectivity. I am not sure if the topology would affect the results of my TCP throughput testing and tuning. Maybe software is software with two private network segments or a single bridged one.
Possibilities are endless. In fact, you can do way more than you can do with ESXi when it comes to networking (out of the box). And if the Linux bridge is too basic for your needs, you can always use Open vSwitch. Just remember you have Linux’ powerful networking stack on the host machine. Just point your VMs to the host as default gateway and take it from there. One bridge, 1000 bridges, however you like it. Pull in some IPTables for security and other packet mangling. Happy days. Doing the same with ESXi is either impossible or $$$.
Stop and delete would have been a far better names than destroy and undefine.
Currently looking at setting up OpenStack/KVM/libvirt.
Agreed. Openstack is good stuff. Check out Brent’s blog over at networkstatic.net for some good info in it. Ill be adding some openstack stuff hopefully soon as well. It’s been on my list for a while.