VMware Server needs exactly two kernel modules running on the host system (there are separate modules for guest systems). These are the vmmon and the vmnet modules. Unfortunately, the vmmon and vmnet packages included in the VMware server distribution package don't compile with the newest kernels. When I upgraded my laptop to Gutsy Gibbon a few weeks ago, I forgot to check if VMware server supports the new kernel. And, of course, it doesn't yet. But luckily I found the vmware-any-any package, a patched version of the host kernel modules that works with newer kernel versions.

You can download the package from here:

http://knihovny.cvut.cz/ftp/pub/vmware/?M=D

Download the tarball and unpack it:

mkortela@T60:~/Desktop$ tar zxvf vmware-any-any-update114.tar.gz
vmware-any-any-update114/
vmware-any-any-update114/services.sh
vmware-any-any-update114/runme.pl
vmware-any-any-update114/update.c
vmware-any-any-update114/vmnet.tar
vmware-any-any-update114/update
vmware-any-any-update114/vmblock.tar
vmware-any-any-update114/vmmon.tar

You also need the source code of your running kernel. On Ubuntu, you can get it by running:

mkortela@T60:~/Desktop$ sudo apt-get install linux-source

There's a "runme.pl" script included which should be able to compile the modules and patch your system, but I couldn't get it working. Instead, I unpacked the the vmmon.tar and vmnet.tar and compiled them myself:

mkortela@T60:~/Desktop/vmware-any-any-update114$ tar xf vmmon.tar
mkortela@T60:~/Desktop/vmware-any-any-update114$ tar xf vmnet.tar
mkortela@T60:~/Desktop/vmware-any-any-update114$ cd vmmon-only
mkortela@T60:~/Desktop/vmware-any-any-update114/vmmon-only$ make
.
.
.
mkortela@T60:~/Desktop/vmware-any-any-update114$ cd ../vmnet-only
mkortela@T60:~/Desktop/vmware-any-any-update114/vmnet-only$ make
.
.
.

The result is two files which can be loaded into the kernel as modules: vmmon.ko and vmnet.ko. You can simply insert those into the kernel with the insmod command:

mkortela@T60:~/Desktop/vmware-any-any-update114$ sudo insmod vmnet-only/vmnet.ko
mkortela@T60:~/Desktop/vmware-any-any-update114$ sudo insmod vmmon-only/vmmon.ko

After that, VMware server starts up nicely!

But one thing is still left. To be able to load the modules at startup properly, they must be copied to the kernel module tree under /lib. First I created a directory for the modules, and then copied them there. After that operation, the command depmod must be run in order to create an up-to-date list of available modules:

mkortela@T60:~/Desktop/vmware-any-any-update114$ sudo mkdir /lib/modules/2.6.22-14-generic/vmware-server
mkortela@T60:~/Desktop/vmware-any-any-update114$ sudo cp vmmon-only/vmmon.ko /lib/modules/2.6.22-14-generic/vmware-server/
mkortela@T60:~/Desktop/vmware-any-any-update114$ sudo cp vmnet-only/vmnet.ko /lib/modules/2.6.22-14-generic/vmware-server/
mkortela@T60:/lib/modules/2.6.22-12-generic/vmware-server$ sudo depmod

That's it! Now you should be able to run VMware server, and all the modules should be loaded at runtime. Please note that if you are doing a fresh install of VMware server, you must now run the vmware-config-network.pl script to configure the network.

You can make sure that the modules load properly by unloading them with rmmod and then loading them again with modprobe:

mkortela@T60:/lib/modules/2.6.22-12-generic/vmware-server$ sudo rmmod vmmon
mkortela@T60:/lib/modules/2.6.22-12-generic/vmware-server$ sudo rmmod vmnet
mkortela@T60:/lib/modules/2.6.22-12-generic/vmware-server$ sudo modprobe vmmon
mkortela@T60:/lib/modules/2.6.22-12-generic/vmware-server$ sudo modprobe vmnet

Vmmon should print a status message like this in your dmesg:

[  999.429124] /dev/vmmon[8602]: VMCI: Driver initialized.
[  999.432385] /dev/vmmon[8602]: Module vmmon: registered with major=10 minor=165
[  999.432756] /dev/vmmon[8602]: Module vmmon: initialized